Losing signal through windows wouldn't matter. Radio waves go out from the
antenna in all directions through solid walls too, except metal. All of the
signal except direct lines from router to computers' WLAN antennas is lost
anyway. Reflection from walls (some paints, metal components) is more
important. You don't want reflected signals coming back to the router as
interference. No windows near my computer and my WLAN adapter is below
ground level but I still recieve signals from routers located in other homes
nearby.
If there are other WLANs nearby your computers could be trying to connect
to those periodically instead of your own network. Set up your WLAN with a
unique SSID and make sure that on each computer only your own wireless
network is set for automatic connection. Also on each computer set them to
connect only to infrastructure networks. Try setting your router to use
channel 1 or 11 rather than the default or automatic channel select.
Check your router logs to see what is happening there. If there are a lot of
bad data or attacks being registered, frequent reconnects to your ISP, or
spontaneous reboots, you may need to make changes. Update the router
firmware if a newer version is available (do this via a hardwire connection,
not wireless). If the router reports the addresses of computers that
repeatedly send bad data or attacks see if your router allows blocking those
addresses. Make sure some sort of security protocol is being used, at least
WEP preferably WPA-PSK.
Limit the number of connections being opened by P2P programs. Many routers
cannot cope with them unless you do. If the situation doesn't settle down,
stop using any P2P or IRC software for a few days and see if your connection
stabilizes.