have you looked to see if the icon is hidden in the Windows System Tray?
For Windows XP: Right-click on the Start button, select properties, select
the taskbar tab then Customize. locate the "you have new unopened items"
entry and set it to always show.
if you are talking about a general set reminder then try starting outlook
once with the /cleanreminders switch added. consult help under "command-line
switches" to find switch use.