Depends on how you want to debug - do you want the XPE device to be
running WinDBG, or do you want to run WinDBG on another machine and debug
the XPE box?
In the first case, it might be possible, but I haven't gotten that working
yet.
In the second case, modify the XPE device's BOOT.INI to add a /debug
switch to the boot entry for XPE. You can set /baudrate and /debugport as
necessary, then connect a null modem cable between the XPE device and your
debug host and launch WinDBG on the host.