It kinda depends on the extension you associated. If it's a .dll, or.exe,
or .sys, or .lnk, for example, then no, you don't want to change it (well,
you do, but to what? there are a number of Windows files that open with
Unknown Application, because of the nature of the file extension. Its
complexities may vary). If it's a file extension that windows doesn't use,
like one you made up, say, .xyz, then yes, you can remove it in the
registry.
If you are trying to change an extension back to it's original state,
associated with its original Unknown Application, you might
have to state which file extension it is, if you need directions as to how
to restore it.