Use the correct language. Start - Control Panel - Regional Settings - Languages - Details - Add.
I can't tell what you're set for (as it's unknown which keys you are referring to - is @ on the 2 key or is " on the 2 key). If your language is correct then the markings on your keyboard aren't. Australian and NZ keyboards have @ on 2, UK have " on 2. The only difference in the keyboards is what is painted on the keys.
