If you're using Vista, you don't have Outlook Express.
If you're using Vista's Windows Mail, and install Windows Live Mail, you
can't remove Windows Mail. You can remove the accounts from it, or just
stop using it.
Under which version of Windows? If it's Vista, Outlook Express won't run
there anyway so you might as well remove it. Windows Mail is a part of
Vista and very hard to remove without also removing the rest of Vista,
but you can keep it from doing much by removing all the accounts under
it. You can also delete all shortcuts pointing to it.
If it's XP, you'll find more people who can answer if you ask in this
newsgroup: