Using multiple _alloca()'s to build up a buffer

[Guest]

I usually try to stay away from _alloca(). However, I'm considering using it
for a logging function. Our current logging function maintains its own
buffer which it grows to fit the string being logged. To work correctly in a
multi-threaded environment there are locks around the code which log the
string. We could remove the locks and just allocate on the heap, but then we
could run into heap contention and heap fragmentation. We could also just
keep the instance buffer in there along with the locks.

Another approach would be to use the stack to store the buffer, e.g.
_alloca. The problem is that I don't know the size the buffer needs to be.
The logging API is exposed via a method that allows for a variable number of
arguments (...) and we use _vsnprintf() to format the buffer. I don't want
to create an extremely large buffer to handle any possible input since that
could blow up our stack quite easily. So I'm wondering if _alloca() can be
chained as follows:

#define ALLOCA_BLOCK_SIZE (1024)

void EventLogLogger::LogArg(short type, long severityLevel, const char
*format, va_list args)
{
char *buffer;
int size = 0;

while (true)
{
size += ALLOCA_BLOCK_SIZE;
buffer = (char *) _alloca(ALLOCA_BLOCK_SIZE);
if (_vsnprintf(buffer + this->_len + 2, size - this->_len + 2, format,
args) >= 0)
break;
}
strcpy(buffer, this->_application);
buffer[this->_len] = ':';
buffer[this->_len + 1] = ' ';
OutputDebugString(buffer);
}

So the chaining above allocates the same block size each time (1024) but I'm
assuming buffer points to a block of memory whose size is 'size' (n * 1024).
Is this correct? This code appears to work. What it does is simply output
the string to the debug monitor, prepending <application: > where application
is a string supplied to the constructor of my class.

In reading the docs for _alloca() it mentions something about not being able
to use _alloca() in exception handling clauses, though it then says you can
use it from within an EH routine. What does this mean? I just can't have it
in the catch clause but I can call a function from the catch clause which
uses _alloca()?

 

[Doug Harrison [MVP]]

I usually try to stay away from _alloca(). However, I'm considering using it
for a logging function. Our current logging function maintains its own
buffer which it grows to fit the string being logged. To work correctly in a
multi-threaded environment there are locks around the code which log the
string. We could remove the locks and just allocate on the heap, but then we
could run into heap contention and heap fragmentation. We could also just
keep the instance buffer in there along with the locks.

Another approach would be to use the stack to store the buffer, e.g.
_alloca. The problem is that I don't know the size the buffer needs to be.
The logging API is exposed via a method that allows for a variable number of
arguments (...) and we use _vsnprintf() to format the buffer. I don't want
to create an extremely large buffer to handle any possible input since that
could blow up our stack quite easily. So I'm wondering if _alloca() can be
chained as follows:

#define ALLOCA_BLOCK_SIZE (1024)

void EventLogLogger::LogArg(short type, long severityLevel, const char
*format, va_list args)
{
char *buffer;
int size = 0;

while (true)
{
size += ALLOCA_BLOCK_SIZE;
buffer = (char *) _alloca(ALLOCA_BLOCK_SIZE);
if (_vsnprintf(buffer + this->_len + 2, size - this->_len + 2, format,
args) >= 0)
break;
}
strcpy(buffer, this->_application);
buffer[this->_len] = ':';
buffer[this->_len + 1] = ' ';
OutputDebugString(buffer);
}

So the chaining above allocates the same block size each time (1024) but I'm
assuming buffer points to a block of memory whose size is 'size' (n * 1024).
Is this correct? This code appears to work. What it does is simply output
the string to the debug monitor, prepending <application: > where application
is a string supplied to the constructor of my class.

That'll work fine. How large can the buffer grow? The default stack
reservation is 1 MB, and your program will not terminate gracefully if it's
exceeded. So you might want to impose a limit. Also, any linear growth
policy will be O(N^2) in the final size of the buffer, and all the copying
can really kill performance. Though it's probably not a big issue here, a
better strategy is exponential growth, say, by a factor of 1.5 each time.
This is what classes like std::vector do to give amortized constant time
push_back.

In reading the docs for _alloca() it mentions something about not being able
to use _alloca() in exception handling clauses, though it then says you can
use it from within an EH routine. What does this mean? I just can't have it
in the catch clause but I can call a function from the catch clause which
uses _alloca()?

Yes, but I'm not clear as to what the difference is between the catch block
context and functions called from that context. Note that the compiler is
serious enough about this to generate errors for direct calls of _alloca in
forbidden contexts.

 

[Guest]

Yes, I might add some sort of exponential grow as opposed to linear growth.

In terms of the stack size, doesn't windows/crt grow that for me? I thought
there was some code that caught a stack overflow and grew the stack for me,
assuming there was room to grow. And if this is true, would it be correct to
say that allocating from the stack or the default heap (not talking about
multiple private heaps) each has the same likelihood for success? From what
I remember the stack and heap both work from the same chunk of memory each
working towards each other (stack growing towards higher addresses and the
heap growing towards lower addresses). If this is true then I would suspect
that if an alloca() would fail a malloc() would also fail.

 

[Doug Harrison [MVP]]

Yes, I might add some sort of exponential grow as opposed to linear growth.

In terms of the stack size, doesn't windows/crt grow that for me? I thought
there was some code that caught a stack overflow and grew the stack for me,
assuming there was room to grow.

The default stack size is 1 MB, but that's just a memory reservation. The
OS commits memory to the stack as the stack is used, and when the memory
reservation is exceeded, it raises EXCEPTION_STACK_OVERFLOW.

And if this is true, would it be correct to
say that allocating from the stack or the default heap (not talking about
multiple private heaps) each has the same likelihood for success? From what
I remember the stack and heap both work from the same chunk of memory each
working towards each other (stack growing towards higher addresses and the
heap growing towards lower addresses). If this is true then I would suspect
that if an alloca() would fail a malloc() would also fail.

No, that's not correct. It is very possible for alloca to fail when malloc
has tens of megabytes available to it. It is also possible for malloc to
fail when alloca could succeed. I think it would be quite unusual for them
both to fail at the same time for ordinary allocations. In modern Windows,
the stack and heap don't grow toward one another and compete for the space
in-between. You need to think in terms of address spaces. By default, the
stack has 1 MB of reserved address space, while malloc has the rest of the
address space (less things like the process image area), and these regions
of address space don't compete with one another, except for virtual memory
to back them up.

 

[Guest]

Thanks. By the way, I realized afterwards that I got the directions wrong,
at least in the old model. The heap would grow towards higher addresses and
the stack would grow towards lower addresses (at least I think that's right
now).

So you're saying that the 1MB is reserved and as soon as that total 1MB is
committed and we grow past it I will get a stack fault? So if I think I'll
have a deep stack I should reserve more stack space (with the linker flag)?

 

[Doug Harrison [MVP]]

Thanks. By the way, I realized afterwards that I got the directions wrong,
at least in the old model. The heap would grow towards higher addresses and
the stack would grow towards lower addresses (at least I think that's right
now).

Stacks still grow downwards.

So you're saying that the 1MB is reserved and as soon as that total 1MB is
committed and we grow past it I will get a stack fault? So if I think I'll
have a deep stack I should reserve more stack space (with the linker flag)?

Yep, and you can test it with infinite recursion.

 

[Pavel A.]

I usually try to stay away from _alloca(). However, I'm considering using it
for a logging function.

You may want to consider the Windows trace (a.k.a. WMI Trace ).

--PA