4GB shows as 2GB in Vista Ultimate 32bit

[Mark]

As stated in my other posts, software can overcome the barrier by providing
additional bits for addressing, or paging schemes, but these all produce
overhead. Should you turn these schemes on but not improve the amount of
memory you have in your system, then you may actually see a decrease in
available memory due to the increased overhead.

These re-addressing schemes do not free up the reserved space. They only
allow re-mapping of additional memory because hardware is still treated as
hardware and reserves memory so as not to violate the boundaries and overall
maintain compatibility.

"A 32-bit system can only address 4GB of RAM."

True for some processor architectures, but not the Intel 32-bit
architecture. By use of segment tables and the PAE extension, it can
support up to 64GB of RAM.
The intertwining of LDTs, GDTs, PAE and page tables provide a myriad of
addressing modes which are simplified by the restrictions chosen by the
Operating System (not to mention the support chips which have their own
restrictions.

 

[cvp]

It's not software, it's the Intel 386+ architecture that provides more
addressing modes than any operating system can sensibly use. Windows in all
its versions uses (intentionally) a very small subset of those capabilities
to make its life easy, but the Intel 48-bit (16 plus 32) addressing lookup
is there for all "32-bit" Intel processors and is used for every instruction
execution - no extra overhead. That 48-bit lookup is reduced by PAE to 4
plus 32 (hence 64GB). The paging scheme is built in to that and has to live
within that 64GB address space before mapping to real memory which is
generally limited by the capabilities of support chips etc. but obviously
cannot exceed that 64GB. The "reserved memory" has to be part of that real
memory, but can appear anywhere within the virtual address space that the
operating system (and bios) allows.

It was only your incorrect statement "A 32-bit system can only address 4GB
of RAM." that I was addressing.

http://www.intel.com/design/pentiumii/manuals/24319202.pdf covers the nitty
gritty...

As stated in my other posts, software can overcome the barrier by
providing additional bits for addressing, or paging schemes, but these all
produce overhead. Should you turn these schemes on but not improve the
amount of memory you have in your system, then you may actually see a
decrease in available memory due to the increased overhead.

These re-addressing schemes do not free up the reserved space. They only
allow re-mapping of additional memory because hardware is still treated as
hardware and reserves memory so as not to violate the boundaries and
overall maintain compatibility.

 

[Mark]

Part of the issue is we started with Vista, moved to general 32-bit
addressing and are now talking Intel architecture.
You are correct: Software can be written to bypass the 32-bit address limit.
It was done back when the data bus was only 8-bits wide and transferred
information in 4-bit nibbles. An 8-bit machine was capable of addressing 128
GB, but who had the money? Much less, there wasn't any software that could
make use of it.

The 4GB limit with 32-bit is simply a function of math and not a real
limit. It simply minimizes overhead for addressing. And, the 48-bit Intel
architecture can address much more than 64GB with the same math function.
But beyond the 32-bit limit, paging is currently used. 16-bits for paging
would realize an almost infinite amount of memory that could be accessed.
Instead, these additional bits are used to create protected-mode memory,
handle interrupts, address the device with the memory we are attempting to
access, and pass the instructions to the device on what to do with the
address being forwarded. PAE can take 4-bits from this because it is not all
currently being used for these other functions. (This is how we address
things like the swap file, cache memory, etc. which all violate the 4GB
limit imposed by 32-bit addressing.) Use of paging requires that a paging
table be established: overhead. Use of multiple paging tables requires
paging directories. More overhead. Change the size of your swap file and see
how much free RAM changes. (Don't get rid of it... you need it.)

Additionally, PAE forces addressing to be handled as 64-bit addresses to
make use of those additional 4-bits, requiring 64-bit drivers in a 32-bit
OS. This requires an address be broken into two 32-bit strings making the
CPU work double to address the same memory for those drivers and
applications using those drivers. Again, "overhead." As a result, PAE was
"disabled" by MS in 32-bit OS's as of WinXP SP2. (It runs, but it doesn't do
anything.)

I understand the physical architecture can handle more, but the software
is limited to 32-bit by design of the software and hardware for
compatibility with non-Intel architecture (most of which also has additional
bits for addressing the hardware.)

So, back to my original comment:
There are workarounds, but they create overhead which would lower usable
memory if the user doesn't increase the amount of physical RAM available.

 

[Paul Clemmons]

Geez guys, I didn't think this was going to cause arguments, but thanks for
all the info. What is interesting is no one from MSFT has chimed in on this
one. Not that I am complaining. Memory allocation (or lack thereof in some
cases) has been a problem since DOS (as some of us are old enough to
remember that DOS 3.3+ was awesome and DOS 6.2 was bloated to take into
account the memory issues of the day)

I have already indicated to the customer that if he wants to take full
advantage of the 4GB (8GB when he gets done with it) he must move to a
64-bit OS, simply because of what was stated earlier, with 16EB of
addressable memory, it is going to take a while.

Makes you wonder if one of the future generations of windows will have to be
loaded from a BlueRay disk and require a minimum of 1TB of memory and 10TB
of drive space.

Paul
"the amazing thing was I used to program dBase III+ in 512KB of memory space
and it ran wonderfully"

 

[Mark]

This was actually a very good thread. Lots of questions AND answers on a
topic that keeps coming back to the newsgroup. In this case, I don't think
it's arguments... it's discussion.

Overall, the point... it's not a Vista limit.

Geez guys, I didn't think this was going to cause arguments, but thanks
for all the info. What is interesting is no one from MSFT has chimed in
on this one. Not that I am complaining. Memory allocation (or lack
thereof in some cases) has been a problem since DOS (as some of us are old
enough to remember that DOS 3.3+ was awesome and DOS 6.2 was bloated to
take into account the memory issues of the day)

I have already indicated to the customer that if he wants to take full
advantage of the 4GB (8GB when he gets done with it) he must move to a
64-bit OS, simply because of what was stated earlier, with 16EB of
addressable memory, it is going to take a while.

Makes you wonder if one of the future generations of windows will have to
be loaded from a BlueRay disk and require a minimum of 1TB of memory and
10TB of drive space.

Paul
"the amazing thing was I used to program dBase III+ in 512KB of memory
space and it ran wonderfully"

 

[cvp]

The 4GB limit with 32-bit is simply a function of math and not a real

limit.

OK I still think you're missing the point. "32-bit" has to do with the data
path width. It has no connection (on Intel 386 architecture) with the
virtual address space or the real memory space. You may recall that some of
the early 386-architecture processors were actually 16-bit. Paging does
nothing to reduce or increase the real memory limits. However the size and
usage of each entry within the page table do. They are limited to 36-bits
used for real addresses. That's where the 64GB real memory restriction comes
from. ALL address accesses go through the 48-bit lookup so there's no
overhead in using the mechanisms.
The Windows implementation on 32-bit Intel architecture does limit the
virtual address space (for ease of software architecture, to allow 16-bit
Intel architecture compatibility, and an unfullfilled desire to support
other less flexible processor architectures).

 

[Mark]

I think we've both got it.
Unfortunately, there are mixed discussions occurring.
Allow me to rehash.

The 32-bit data path is limited to 4GB of addressing.
(It's a math function of 2 raised to the 32nd power.)

Hardware that is designed to be directly read by the processor reserves
space from that 4GB and does not allow alteration of that space. Otherwise,
the physical 4GB limit would be breached. So, video cards that reserve
memory actually "disable" a portion of the system RAM to allow direct access
to the video card RAM.
(I think this was the original question.)

The actual 4 GB being addressed at any one time by the 32-bit system
can be altered as long as reserved areas (most) are not altered.

Increasing the data path width to 36 bits should allow the use of
68,719,476,735 bytes (64GB.)
The extra 4 bits is not universal amongst the platforms and would create
incompatibility issues.

The extra 16-bits on the address registers are used for purposes other
than addressing memory and are used every clock iteration just as the other
32-bits are used for address functions. If the full 16 bits were used to
page an actual 32-bit address, it would set a memory limit of 16,384GB (4096
pages of 4GB each.)
(http://www.intel.com/design/pentiumii/manuals/24319202.pdf )

Instead, various schemes are used to artificially increase the data path
which raises the 4GB limit, but the OS is still operating on a 32-bit limit.
(Even though the architecture may be there to support higher bit paths.)

Paging tables are used to point to different areas of memory beyond the
32-bit addressing limit allowing the 4GB window to capture a different 4GB
area. Paging limits the memory accessible at any moment to 20-bits (1MB),
plus a 12-bit offset (or 4096 pages of 1MB each... a total of 4GB,) but it
is a flexible 4GB. Paging, as it currently exists, does not make use of the
extra 4-bits on the address bus. This memory addressing is slower than
direct memory addressing since it has to collect two pieces of information
to create an address. Overhead grows rapidly in this system as you create
tables, directories, algorithms to anticipate loading of pages, etc. As a
result, hardware prefers to have direct access (no paging) and must reserve
memory which in turn lowers the available memory from the 4GB limit.
(http://webster.cs.ucr.edu/AoA/Windows/HTML/MemoryArchitecturea3.html)

PAE makes use of the 4-bits, increasing the address path to 36 bits, but
to get the information, all PCI adapters (including 32-bit PCI adapters)
must be able to address the full physical address space. For 32-bit PCI
adapters, this means that they must be able to support the Dual Address
Cycle (DAC) command to permit them to transfer 64-bit addresses to the
adapter or device (that is, addresses above the 4 GB address space).
Unfortunately, Microsoft found that not all PCI buses on a system board
support DAC, which is required for a 32-bit PCI adapter to address more than
4 GB of memory. Furthermore, there is no way for a DAC-capable PCI device
(or its associated driver) to know that it is running on a non-DAC-capable
bus.
End result, PAE requires 64-bit device drivers to work on the 32-bit OS.
Since 64-bit addresses are not able to transmit on the 48-bit architecture,
they are folded into two iterations of 32-bit addresses eating more CPU time
to process a single address. This is an overall loss, but PAE support is
left available to support Data Execution Prevention still found on some
hardware.
(http://www.microsoft.com/whdc/system/platform/server/PAE/PAEdrv.mspx)

PAE used with AWE (only on server platforms) allows use of non-paged
memory above the 4GB limit with a 32-bit OS. (Win 2003 Server, Datacenter,
Advanced Datacenter)
(http://msdn2.microsoft.com/En-US/library/aa366796.aspx)

 

[cvp]

Well, I would argue with a couple of pieces, but it's not worth it since
it's really all history now that 64-bit addressing is here (at least in the
processors and OS). Now all we have to do is get the hardware platforms (and
the add-ins) to support the extra real memory (4GB and 8GB restrictions
abound at the moment).

 

[Mark]

Thanks for the discussion!

Well, I would argue with a couple of pieces, but it's not worth it since
it's really all history now that 64-bit addressing is here (at least in
the processors and OS). Now all we have to do is get the hardware
platforms (and the add-ins) to support the extra real memory (4GB and 8GB
restrictions abound at the moment).

 

[MICHAEL]

Yes, I've enjoyed reading the discussion, too.


-Michael

* Mark:

 

[Rick Raisley]

No, you're misunderstanding what happens. The video card does *not*
take 768MB out of your 4GB of system RAM; it does not take *any* of
your system RAM.

Whether you say it "uses" or "takes", it's 768 MB I don't have available,
right?

What it uses (not takes) is some of the 4GB *address space* that all
32-bit versions of Windows have. That 4GB of address space is the same
for all of us, *regardless* of how much memory we have: 512MB, 1GB,
2GB, 4GB, etc.

What I think it amounts to (and yes, it's been a very interesting discussion
in this thread) is that the 768 MB address space is taken from the 4 GB
address space. BUT - If you only have 2 or 3 MB of actual memory, then none
of the memory is lost due to the video card? Is that right? It just uses the
unused upper address space, and since that isn't required for memory, it
doesn't reduce memory any? (I realize there are items other than the 768 MB
video card in this instance that similarly take up memory addresses or
actual memory.)

 

[Jim]

Whether you say it "uses" or "takes", it's 768 MB I don't have available,
right?


What I think it amounts to (and yes, it's been a very interesting
discussion
in this thread) is that the 768 MB address space is taken from the 4 GB
address space. BUT - If you only have 2 or 3 MB of actual memory, then
none
of the memory is lost due to the video card? Is that right? It just uses
the
unused upper address space, and since that isn't required for memory, it
doesn't reduce memory any? (I realize there are items other than the 768
MB
video card in this instance that similarly take up memory addresses or
actual memory.)

The video card uses RAM on the motherboard if, and only if, it states that
it uses shared memory. The shared memory is not available to Windows.

Otherwise, the card only needs a very limited amount of system RAM for
interface purposes.

Jim

 

[Ken Blake, MVP]

What I think it amounts to (and yes, it's been a very interesting discussion
in this thread) is that the 768 MB address space is taken from the 4 GB
address space. BUT - If you only have 2 or 3 MB of actual memory, then none
of the memory is lost due to the video card? Is that right? It just uses the
unused upper address space, and since that isn't required for memory, it
doesn't reduce memory any? (I realize there are items other than the 768 MB
video card in this instance that similarly take up memory addresses or
actual memory.)

Yes.

 

[Rick Raisley]

The video card uses RAM on the motherboard if, and only if, it states that
it uses shared memory. The shared memory is not available to Windows.

Otherwise, the card only needs a very limited amount of system RAM for
interface purposes.

Well, yeah, UNLESS you happen to have 4 GB of memory, and then, for all
practical purposes, the full video card memory is /taken away/ from
available RAM. Rather ironic, that all these years I've avoided "shared"
memory like the plague, only to find on my state-of-the-art PC that I "kinda
have it"! ;-) I realize that's not actually true, that shared memory would
be much slower and all, but I still lose it nonetheless!