StringBuilder OutOfMemory

[Arnd Iffland]

Hi,

i want to use a StringBuilder to create a large string.

After instatiation, the MaxCapacity is 2.147.483.647, what is large
enough for my purpose.

During my procedure i get a SystemOutOfMEmoryException at
System.String.GetStringForBuilder

The length of the StringBuilder is at this moment 17.825.779 and the
capacity 17.825.792.

Why do i get this exception? What can i do to get the MAxCapacity?

There is no other StringBuilder in the procedure.
The procedure is a WebService build with WebDeveloper Express and .Net
2.0.50727.42

Best regards,

Arnd

--

 

[Scully]

Hi Arnd,

can you show me some code where the exception occurs? Maybe, I can be
of help.

Cheers.

 

[Hans Kesting]

Hi,

i want to use a StringBuilder to create a large string.

After instatiation, the MaxCapacity is 2.147.483.647, what is large
enough for my purpose.

During my procedure i get a SystemOutOfMEmoryException at
System.String.GetStringForBuilder

The length of the StringBuilder is at this moment 17.825.779 and the
capacity 17.825.792.

Why do i get this exception? What can i do to get the MAxCapacity?

There is no other StringBuilder in the procedure.
The procedure is a WebService build with WebDeveloper Express and .Net
2.0.50727.42

Best regards,

Arnd

StringBuilder doesn't reserve MaxCapacity in memory, but a lower value
that you can specify when you instantiate it (or accept the default
value of 16 chars).
When you want to add text, so that the current capacity is not enough
anymore, then StringBuilder will try to *double* it's capacity.

So in your case it will want to go from 17MB to 34MB. I don't know if
it needs a *continuous* block of memory.

try it with a
StringBuilder sb = new StringBuilder(<your target capacity>);


Hans Kesting

 

[Willy Denoyette [MVP]]

| Hi,
|
| i want to use a StringBuilder to create a large string.
|
| After instatiation, the MaxCapacity is 2.147.483.647, what is large
| enough for my purpose.
|
| During my procedure i get a SystemOutOfMEmoryException at
| System.String.GetStringForBuilder
|
| The length of the StringBuilder is at this moment 17.825.779 and the
| capacity 17.825.792.
|
| Why do i get this exception? What can i do to get the MAxCapacity?
|
| There is no other StringBuilder in the procedure.
| The procedure is a WebService build with WebDeveloper Express and .Net
| 2.0.50727.42
|
| Best regards,
|
| Arnd
|
| --
|

You can't create a StringBuilder with "MaxCapacity is 2.147.483.647". The
largest StringBuilder on the current CLR versions available is ~1G
characters, but even such a SB can't be created on 32 bit Windows, you need
to run 64 bit windows for this. On 32 bit Windows the largest SB you are
able to create is subject of the amount of free "contiguous" virtual address
space, the size of free space highly depends on the amount of fragmentation
of both the 'process heap' and the 'large object heap', but it general it's
much smaller than ~1.2 GB - 1.6Gb (or ~600K - 800K characters). If your heap
is highly fragmented (which I guess it's the case here), it's even possible
that you can't allocate a 17.825.779 characters SB.

Willy.

 

[Kevin Spencer]

Hi Arnd,

It's hard to say for sure what caused your OOM exception. There are many
possible factors involved. In fact, there are no methods of StringBuilder
that are documented to throw an OOM exception. So, it is entirely possible
that the StringBuilder did not throw the exception, but that the exception
was thrown as a result of a StringBuilder operation that resulted in a
memory allocation exceeding the amount of available memory on the system.

What I mean is, it looks to me like the problem is not related to the
MaxCapacity or Capacity properties of the StringBuilder, asevidenced by your
statistics. However, using a StringBuilder can cause memory allocation to
occur at different points, and this memory allocation may have caused the
exception.

For example, the default Capacity of a StringBuilder is 16 characters. When
you append more than 16 characters, the Capacity is doubled, and is doubled
again each time the length of the string exceeds the Capacity. Each time
this occurs, more memory must be allocated. Since it is allocated piecemeal,
you may be experiencing the result of memory fragmentation. This can be
avoided by setting the Capacity to a high number before beginning the
building of the String.

Another possibility lies in the usage of the ToString method of the
StringBuilder. The first time this is called, a reference to the actual
string in the buffer is returned. If you append to the String afterwards,
the existing string is copied to the buffer, so that the string reference
returned by the previous ToString call is not altered by your future
StringBuilder modification code. This means that additional memory is
allocated to hold the existing string buffer, and the new buffer.

You can read more about the technical details here:

http://www.codeproject.com/dotnet/strings.asp

--
HTH,

Kevin Spencer
Microsoft MVP
Professional Chicken Salad Alchemist

A lifetime is made up of
Lots of short moments.

 

[Ignacio Machin \( .NET/ C# MVP \)]

hi


post the code of the method

 

[Arnd Iffland]

You can't create a StringBuilder with "MaxCapacity is 2.147.483.647".
The largest StringBuilder on the current CLR versions available is
~1G characters, but even such a SB can't be created on 32 bit
Windows, you need to run 64 bit windows for this. On 32 bit Windows
the largest SB you are able to create is subject of the amount of
free "contiguous" virtual address space, the size of free space
highly depends on the amount of fragmentation of both the 'process
heap' and the 'large object heap', but it general it's much smaller
than ~1.2 GB - 1.6Gb (or ~600K - 800K characters). If your heap is
highly fragmented (which I guess it's the case here), it's even
possible that you can't allocate a 17.825.779 characters SB.

Willy.

Thanks for the explanation!

--

 

[Willy Denoyette [MVP]]

Try this:

string s = new string('a', 1024*1024*32);
StringBuilder sb = new StringBuilder(1024*512); // start with a SB of
512K chars.
for (int i = 0; i < 100; i++)
{
Console.WriteLine(sb.Capacity);
sb.Append(s);
}


The output should look like:
524288
33554432
67108864
134217728
134217728
268435456
268435456
268435456
268435456

notice the SB expanding, but as the SB expands, the preceeding buffer cannot
be re-used as it's contents must be preserved in order to copy the contents
to the new buffer. When the SB has expanded a second (and third and
fourth...) time, the preceeding "free buffers" are simply too small to hold
the new required SB buffer (they can be used to hold other objects however).
This results in a max. SB of 268435456 chars. in this sample.

That is why you should pre-allocate your SB when you have to deal with very
large strings. To prove this, just change the initial size into 620Mb
(1024*1024*620) and all should be fine for this simple console application,
but no-one will guarantee that it will do for a more complex program, nor
will it work for a Windows Forms application.


Willy.

"Ignacio Machin ( .NET/ C# MVP )" <ignacio.machin AT dot.state.fl.us> wrote
in message | hi
|
|
| post the code of the method
|
|
| --
| --
| Ignacio Machin,
| ignacio.machin AT dot.state.fl.us
| Florida Department Of Transportation
|
| | > Hi,
| >
| > i want to use a StringBuilder to create a large string.
| >
| > After instatiation, the MaxCapacity is 2.147.483.647, what is large
| > enough for my purpose.
| >
| > During my procedure i get a SystemOutOfMEmoryException at
| > System.String.GetStringForBuilder
| >
| > The length of the StringBuilder is at this moment 17.825.779 and the
| > capacity 17.825.792.
| >
| > Why do i get this exception? What can i do to get the MAxCapacity?
| >
| > There is no other StringBuilder in the procedure.
| > The procedure is a WebService build with WebDeveloper Express and .Net
| > 2.0.50727.42
| >
| > Best regards,
| >
| > Arnd
| >
| > --
| >
|
|