Breaking the 2GB barrier

[Frank Rizzo]

I will soon be deploying an application on Windows 2003 R2 64-bit
Standard Edition. The app will also be compiled for 64-bit. One of the
reasons is that the application will really benefit from having more
than 2GB available to it.

The
http://www.microsoft.com/windowsserver2003/evaluation/features/comparefeatures.mspx
page states that Windows 2003 R2 64-bit Standard Edition can handle up
to 32GB of RAM. My question is whether anyone has actually seen an
application break the 2GB barrier on this OS?

Regards

 

[Guest]

Frank, on a 64-bit windows OS, if your app is compiled to 64-bit, there "IS
NO" 2 GB barrier.
Peter

 

[Willy Denoyette [MVP]]

|I will soon be deploying an application on Windows 2003 R2 64-bit
| Standard Edition. The app will also be compiled for 64-bit. One of the
| reasons is that the application will really benefit from having more
| than 2GB available to it.
|
| The
|
http://www.microsoft.com/windowsserver2003/evaluation/features/comparefeatures.mspx
| page states that Windows 2003 R2 64-bit Standard Edition can handle up
| to 32GB of RAM. My question is whether anyone has actually seen an
| application break the 2GB barrier on this OS?
|
| Regards

What barrier are you talking about?
When talking about per process user address space, following are the
restrictions imposed by the OS versions..
32 bit Windows :
all systems: 2 GB
extended : 3 GB using 4GT tuning (/3GB flag in boot.ini) valid for
XP,W2K3,W2K AS, NT4 EE
extended : up to 3GB (/USERVA= oflag in boot.ini) XP and W2K3
"extended" requires the application to be "largeaddressaware" enabled.

64 bit Windows:
32 bit applications : 4GB. (.NET application compiled with
/platform:x86)
64 bit applications : 7152 or 8192 GB (.NET applications compiled with
/platform:anycpu or X64).
anycpy is the default.

When talking about the 2GB maximum object size for .NET application, the
same restriction applies for both 32 and 64 bit systems.

It's important to note, that a 32 bit application has a 4GB address space
available on 64 bit windows, this is the prefered operating environment if
you don't need anything more than 4GB. That means that you should compile
with the x86 platform flag, to prevent running in x64 mode.
If you really need more than this, you'll have to compile your application
with the x64 flag, to prevent your application to run on X86 (32 bit
windows).

Willy.

 

[John Vottero]

[snip]

It's important to note, that a 32 bit application has a 4GB address space
available on 64 bit windows, this is the prefered operating environment if
you don't need anything more than 4GB. That means that you should compile
with the x86 platform flag, to prevent running in x64 mode.

That's the first time I've heard that. Why is 32bit the prefered mode under
64 bit windows?

 

[Jon Skeet [C# MVP]]

[snip]

It's important to note, that a 32 bit application has a 4GB address space
available on 64 bit windows, this is the prefered operating environment if
you don't need anything more than 4GB. That means that you should compile
with the x86 platform flag, to prevent running in x64 mode.

That's the first time I've heard that. Why is 32bit the prefered mode under
64 bit windows?

Presumably because all the references will only be 32 bits instead of
64, thus taking up half the space. I'm not sure I necessarily buy it
though - I don't know enough about the execution model for 32 bit apps
on a 64 bit processor, but I'd have *thought* you'd lose some of the
speed of the processor, working in a mode it's not as tailored for. I
could easily be wrong on that bit though.

 

[Willy Denoyette [MVP]]

| | >
| [snip]
| >
| > It's important to note, that a 32 bit application has a 4GB address
space
| > available on 64 bit windows, this is the prefered operating environment
if
| > you don't need anything more than 4GB. That means that you should
compile
| > with the x86 platform flag, to prevent running in x64 mode.
|
| That's the first time I've heard that. Why is 32bit the prefered mode
under
| 64 bit windows?
|

If you don't need the extended addressing space, and if your application
only does some moderate floating point and large integer calculations, you
will achieve some better performance when running in 32 bit mode.
The reason is simple:
- references are everywhere in .NET and the are 4 bytes in 32 bit, 8 bytes
in 64 bit.
- code tends to be bigger also, with an average of 50-60% due to the extra
prefix and the 8 byte immediate values.
Both of these results in a larger processor cache occupation (both
instruction and data) compared to 32 bit. Or technically spoken, a 64 bit
application incurs a larger cache miss rate compared with the same
application running in 32 bit mode. Note that cache lines are 64 bytes on
AMD irrespective the operating mode. The L2 and L1 cache are shared
resources, that means that running multiple (simultaneously active) 64 bit
processes will compete for the cache and increase the L2 and L1 cache misses
even more.

- 64 bit JIT is a 'version 1' product, not optimized to the level of the 32
bit equivalent which is kind of 'version 3'. Or otherwise stated, the JIT32
generates some better code than his 64 bit cousin.

Benchmarks I have run (on AMD) indicate a moderate performance advantage for
32 bit non-math bound applications, while for a highly math bound
application a 64 bit version definitely takes the lead.
Note that your mileage may vary, all depends on the processor type and it's
memory hierarchy.

Willy.

 

[Chris Mullins]

It's not. No way, no how. If you're building .Net code, the preferred mode
is "Any CPU" so that the platform just doesn't matter. There are times (if
you're doing Interop, especially) when you would NEED to run x86 or x64 due
to dependency issues - and in these circumstances it's important that you
compile the right type of code.

Also, .NET Deployment projects have a need to be compiled for a specific
environment.

There's no concept of a "Any CPU" MSI file. This makes things more than a
bit frustrating, especially as MS-Build won't build MSI files. We've had to
create a full build system that pokes at files, resets platforms, used
devenv.exe to perform builds, and all sorts of other craziness...

Presumably because all the references will only be 32 bits instead of
64, thus taking up half the space.

Our 64-bit compiled code is pretty much the same size as our 32-bit compiled
code.

For example, our 64-bit compiled (release mode) app is: 4095488 . The 32-bit
compiled (release mode) app is: 4091904

Memory footprints when things are running are also very close. We calculed a
"12 kb per user" (I don't remember the exact number) a while back on x86 for
a specific type of high-demand user (lots of roster items, managed groups,
etc) and when we moved to x64 we didn't see much in the way of change.

I'm not sure I necessarily buy it
though - I don't know enough about the execution model for 32 bit apps
on a 64 bit processor, but I'd have *thought* you'd lose some of the
speed of the processor, working in a mode it's not as tailored for. I
could easily be wrong on that bit though.

I've signed all sorts of "Don't give out benchmark specifics without written
permission", so I can't go into any detail, but the current generation of
x64 chips, from a variety of vendors, running in x64 mode, are damn fast.

.... and with no memory limititations beyond the cost of the memory,
applications can be as memory hungry as they need to be.

 

[Frank Rizzo]

|I will soon be deploying an application on Windows 2003 R2 64-bit
| Standard Edition. The app will also be compiled for 64-bit. One of the
| reasons is that the application will really benefit from having more
| than 2GB available to it.
|
| The
|
http://www.microsoft.com/windowsserver2003/evaluation/features/comparefeatures.mspx
| page states that Windows 2003 R2 64-bit Standard Edition can handle up
| to 32GB of RAM. My question is whether anyone has actually seen an
| application break the 2GB barrier on this OS?
|
| Regards

What barrier are you talking about?
When talking about per process user address space, following are the
restrictions imposed by the OS versions..
32 bit Windows :
all systems: 2 GB
extended : 3 GB using 4GT tuning (/3GB flag in boot.ini) valid for
XP,W2K3,W2K AS, NT4 EE
extended : up to 3GB (/USERVA= oflag in boot.ini) XP and W2K3
"extended" requires the application to be "largeaddressaware" enabled.

64 bit Windows:
32 bit applications : 4GB. (.NET application compiled with
/platform:x86)
64 bit applications : 7152 or 8192 GB (.NET applications compiled with
/platform:anycpu or X64).
anycpy is the default.

When talking about the 2GB maximum object size for .NET application, the
same restriction applies for both 32 and 64 bit systems.

It's important to note, that a 32 bit application has a 4GB address space
available on 64 bit windows, this is the prefered operating environment if
you don't need anything more than 4GB. That means that you should compile
with the x86 platform flag, to prevent running in x64 mode.
If you really need more than this, you'll have to compile your application
with the x64 flag, to prevent your application to run on X86 (32 bit
windows).

Willy, is this also true for 32-bit .net apps, compiled under vs2003?
Thanks

 

[Willy Denoyette [MVP]]

Willy, is this also true for 32-bit .net apps, compiled under vs2003?

Note that vs2003 only "produces" X86 code, more exactly , the V1.1 JIT compiler only emits
X86 code.
Anyway, all X86 code that runs on X64 (WOW64) has access to 4GB of user memory, provided
that you have set the LARGEADDRESSAWARE header flag.
LARGEADDRESSAWARE can be set by means of the editbin.exe tool from the platform SDK, like
this:
editbin /LARGEADDRESSAWARE <progamfile>

Willy.

 

[Frank Rizzo]

Willy, is this also true for 32-bit .net apps, compiled under vs2003?

Note that vs2003 only "produces" X86 code, more exactly , the V1.1 JIT
compiler only emits X86 code.
Anyway, all X86 code that runs on X64 (WOW64) has access to 4GB of user
memory, provided that you have set the LARGEADDRESSAWARE header flag.
LARGEADDRESSAWARE can be set by means of the editbin.exe tool from the
platform SDK, like this:
editbin /LARGEADDRESSAWARE <progamfile>

Thank you, this clears it up a bit. And, if I compile my app as x86 on
vs2005, do I also need to apply the editbin /LARGEADDRESSAWARE trick or not?

Thanks.

 

[Frank Rizzo]

Note that vs2003 only "produces" X86 code, more exactly , the V1.1 JIT

compiler only emits X86 code.
Anyway, all X86 code that runs on X64 (WOW64) has access to 4GB of user
memory, provided that you have set the LARGEADDRESSAWARE header flag.
LARGEADDRESSAWARE can be set by means of the editbin.exe tool from the
platform SDK, like this:
editbin /LARGEADDRESSAWARE <progamfile>

Another question. If I run editbin /LARGEADDRESSAWARE on my vs2003
compiled app, can it now access more than 2GB on the Windows Server 2003
(32-bit standard or enterprise edition)? The Windows Server is setup
with the /3GB switch.

Thanks.

 

[Willy Denoyette [MVP]]

Thank you, this clears it up a bit. And, if I compile my app as x86 on vs2005, do I also
need to apply the editbin /LARGEADDRESSAWARE trick or not?

Thanks.

You have to apply /LARGEADDRESSAWARE to all X86 binaries that need to address > 2GB , on all
PAE enabled OS.
All 64 bit windows are PAE enabled by default, X86 binaries (largeaddressaware) can address
up to ~4GB on these systems.
X64 binaries can address more than you will be able to fit in any existing box.
32 bit OS SKU's need to be enabled through the boot.ini file, here you have two options, or
you specify /3GB or you set the "userva" variable to the value you need (up to max. 3GB).

Willy.

 

[Willy Denoyette [MVP]]

Another question. If I run editbin /LARGEADDRESSAWARE on my vs2003 compiled app, can it
now access more than 2GB on the Windows Server 2003 (32-bit standard or enterprise
edition)? The Windows Server is setup with the /3GB switch.

Sure, but beware, no objects can be larger than 2GB and fragmentation (win32 heap and/or GC
heap) can spool the party.

Willy.

 

[Willy Denoyette [MVP]]

| | >> Note that vs2003 only "produces" X86 code, more exactly , the V1.1 JIT
compiler only
| >> emits X86 code.
| >> Anyway, all X86 code that runs on X64 (WOW64) has access to 4GB of user
memory, provided
| >> that you have set the LARGEADDRESSAWARE header flag.
| >> LARGEADDRESSAWARE can be set by means of the editbin.exe tool from the
platform SDK, like
| >> this:
| >> editbin /LARGEADDRESSAWARE <progamfile>
| >
| > Another question. If I run editbin /LARGEADDRESSAWARE on my vs2003
compiled app, can it
| > now access more than 2GB on the Windows Server 2003 (32-bit standard or
enterprise
| > edition)? The Windows Server is setup with the /3GB switch.
| >
|
|
| Sure, but beware, no objects can be larger than 2GB and fragmentation
(win32 heap and/or GC
| heap) can spool the party.
|
| Willy.
|

......can spoil the party ;-).
The extra 1GB user VAS, is separated from the lower 2GB user VAS by a 64KB
inaccessible memory block, and the modules (native code dll's) loaded by the
OS loader are still in top of the 1st 2GB address space, worse, some of them
are loaded at lower address locations.
This means that you still don't have a "2GB contagious" block of memory in
your process when running on 32 bit windows.

Willy.