This is what I've always found:
If Windows in Taskbar isn't checked, Alt-Tab should switch apps, and only
one Excel should appear in the Alt-Tab box (only one Excel icon on Taskbar
too). Once in Excel, Ctrl-F6 should switch document windows. So also does
Ctrl-Tab, which is easier.
If Windows in Taskbar is checked, Alt-Tab should switch between all running
apps, and also each open workbook. Each workbook appears as an app.
Ctrl-F6 or Ctrl-Tab still works to switch between workbooks, if you're in
any of the Excel windows. Also, an extra Excel icon appears in the Alt-Tab
box. There's one with a big X, then one with a small x for each workbook.
I have no idea why the extra one is there.