StackOverflowException.

[Robert Bouillon]

I have a StackOverflowException that's NOT caused by a recursive function.
I'm having a hard time with this exception as it takes about an hour to
reproduce, and I'm not sure what's causing it.

I have a serial stream class that reads bytes from a serial port. The
information is stored in a local byte array of about 4k. When I'm done
processing the data, the byte array is assigned to an object property that
serves to retain a record of the bytes transferred. I think this may be the
problem.

I know that locally declared byte arrays are stored on the stack. Am I
mistaken in thinking that assigning a byte array to an object property boxes
the array (Because the object exists on the heap)? If so, then this is my
problem. How do I box the array while keeping it strong-typed?

If assigning a byte array to a class property (on the same thread) DOES box
the array, then I'm at a loss to the problem. Are there any tips or tricks
to determining what exactly is being stored on the stack?

Of course because error handling requires stack space, and my stack is
invalid due to an overflow, it's very difficult trying to track down this
bug.

EXAMPLE CODE (Short version: not actual code. Just to demonstrate how my
code handles the array):

do{
byte[] buffer = new byte[4196];
p_Stream.Read(buffer,0,4196);
DoStuff(buffer);
DataRecord dr = new DataRecord();
dr.Data = buffer;
} while(i++<500)

Thanks
--ROBERT

 

[Willy Denoyette [MVP]]

I have a StackOverflowException that's NOT caused by a recursive function.
I'm having a hard time with this exception as it takes about an hour to
reproduce, and I'm not sure what's causing it.

I have a serial stream class that reads bytes from a serial port. The
information is stored in a local byte array of about 4k. When I'm done
processing the data, the byte array is assigned to an object property that
serves to retain a record of the bytes transferred. I think this may be
the
problem.

I know that locally declared byte arrays are stored on the stack. Am I
mistaken in thinking that assigning a byte array to an object property
boxes
the array (Because the object exists on the heap)? If so, then this is my
problem. How do I box the array while keeping it strong-typed?

If assigning a byte array to a class property (on the same thread) DOES
box
the array, then I'm at a loss to the problem. Are there any tips or tricks
to determining what exactly is being stored on the stack?

Of course because error handling requires stack space, and my stack is
invalid due to an overflow, it's very difficult trying to track down this
bug.

EXAMPLE CODE (Short version: not actual code. Just to demonstrate how my
code handles the array):

do{
byte[] buffer = new byte[4196];
p_Stream.Read(buffer,0,4196);
DoStuff(buffer);
DataRecord dr = new DataRecord();
dr.Data = buffer;
} while(i++<500)

Thanks
--ROBERT

I have no definitive answer for your problem, but your assumption that
arrays are stack allocated is wrong, arrays are reference types so they are
heap allocated.

How can you be so sure it's not because of a recursive function?.

Willy.

 

[Robert Bouillon]

Willy,

Thanks for the reply

My call stack is only 27 calls deep: 17 of them are Windows Forms calls (via
"Show non-user code"). I checked all of my threads as well and they're all
behaving normally. To-date, on StackOverflows caused by recursion, I've
always gotten an exception. I've tried to reproduce this error 3 times, and
only once did I get this exception. The other 2 times I got obscure errors
and was unable to view my local variables, leading me to believe that lack
of stack space caused my VM to go screwy.

I narrowed it down to what I believe may be the culprit. I recently added a
"Status" subclass to my Rs232Stream class. The Status object monitors the
Serial Port in an infinite loop, on background thread, making calls to
win32. Disabling this thread alleviates the problem.

Here's a snippet (C#):

=====================================
/// <summary>
/// Monitors the Serial Port opened by the parent stream for events
/// </summary>
/// <remarks>
/// Sits directly on the m_StatusThread
/// <see cref="Parent"/>
/// <see cref="m_StatusThread"/>
/// </remarks>
private void Monitor()
{
InitEventMask();

while(true)
{
if(!WaitCommEvent(p_Parent.hPort, m_UnmanagedCurrentEvent,
m_UnmanagedOverlappedHandle))
{
NativeError err = Win32Exception.Check(false);
if (err!= NativeError.ERROR_IO_PENDING)
throw new IOException("Error setting comm event handle", new
Win32Exception());
}

m_StatusChangeEvent.WaitOne();
Requery(p_Parent.hPort);
}
}

public Rs232Status(Rs232Stream parent)
{
p_Parent = parent;

Overlapped wo = new Overlapped();
wo.hEvent = m_StatusChangeEvent.Handle;

m_UnmanagedOverlappedHandle =
Marshal.AllocHGlobal(Marshal.SizeOf(wo));
Marshal.StructureToPtr(wo, m_UnmanagedOverlappedHandle, true);

m_UnmanagedCurrentEvent =
Marshal.AllocHGlobal(Marshal.SizeOf(Type.GetType("System.UInt32")));
Marshal.WriteInt32(m_UnmanagedCurrentEvent,0);
}

/// <summary>
/// Checks the status of the Flow Control Lines
/// </summary>
/// <param name="portHandle">Handle to the opened port</param>
/// <remarks>Called by Requery</remarks>
public void UpdateLines()
{
ModemStatus ms;
while(!GetCommModemStatus(p_Parent.hPort,out ms))
if(Win32Exception.Check(false)==NativeError.ERROR_IO_PENDING)
continue;
else
throw new Win32Exception("Error during GetCommModemStatus");
{...} //---> Event Calls and Field Updates
}
===============================
InitEventMask encapsulates the Win32 SetCommMask call. Requery is a lot of
bloated code that checks the Event Flags and calls some OnXXXX methods to
fire events and update some local fields.

It's pretty cut-and-dry. It's an infinite loop that makes a few minor calls
to unmanaged IO code. No unmanaged memory is allocated outside of the ctor,
so I'm really at a total loss as to where I'm leaking stack memory.

I don't mind posting the entire status class here, but I know how painful
that is so I'm going to just post the calls I make to unmanaged code or the
marshaller. Perhaps you'll see a function that manipulates memory or catch
something I'm missing?

[DllImport("kernel32.dll")]
EscapeCommFunction
ClearCommError
GetOverlappedResult
WaitCommEvent
SetCommMask
GetCommModemStatus

[.NET Interop]
Marshal.ReadInt32
Marshal.StructureToPtr

Here's an idea: My EscapeCommFunction has the following declaration:
[DllImport("kernel32.dll")]
private static extern bool EscapeCommFunction(IntPtr hFile, CommFunction
dwFunc);

CommFunction is a private managed struct I use in the class. I prefer to let
..NET interop marshal my variables for me to save time / code. Is it possible
that in this scenario, the dwFunc is never being destroyed? I'm not
destroying it explicitly.

Thanks.
--ROBERT

P.S. Good to know about arrays. I could have sworn I read something that
said blittable-typed arrays were stored on the stack. At least that gives
stackalloc a little more prominence I wish there was a way that
strong-typed streams could access stackalloc arrays, but we all know the
rules about pointers in the CLR :-/

 

[Guest]

Streams are meant to be used as a transport mechanism from one endpoint to
another. In most systems, the final endpoint is a persistant store, not a
perpetual cache. The array should be held on the heap, so this should not be
a problem, but you may be eating up the stack with object references that
never go out of scope. That is where I would check.

---

Gregory A. Beamer
MVP; MCP: +I, SE, SD, DBA

***************************
Think Outside the Box!
***************************

 

[Robert Bouillon]

Sorry for being so abstract. I'm using a persistent store, and only caching
the data temporarily on the heap when debugging. I'll explain a little
further:

The stream is passed to a Protocol Class that handles the transport and
stores the byte array as a property in a Packet class. The buffer used by
the Protocol Class is passed to the Packet class and a new buffer is created
for the next Packet. The Packet class handles converting the binary data to
strong-typed data to be used by .NET.

Normally, when the information from the packet is processed the packet is
discarded (Falls out of scope). When debugging, the Packet is stored in a
strong-typed collection that can be browsed through a Windows Forms control.

--ROBERT

Streams are meant to be used as a transport mechanism from one endpoint to
another. In most systems, the final endpoint is a persistant store, not a
perpetual cache. The array should be held on the heap, so this should not be
a problem, but you may be eating up the stack with object references that
never go out of scope. That is where I would check.

---

Gregory A. Beamer
MVP; MCP: +I, SE, SD, DBA

***************************
Think Outside the Box!
***************************

I have a StackOverflowException that's NOT caused by a recursive function.
I'm having a hard time with this exception as it takes about an hour to
reproduce, and I'm not sure what's causing it.

I have a serial stream class that reads bytes from a serial port. The
information is stored in a local byte array of about 4k. When I'm done
processing the data, the byte array is assigned to an object property that
serves to retain a record of the bytes transferred. I think this may be the
problem.

I know that locally declared byte arrays are stored on the stack. Am I
mistaken in thinking that assigning a byte array to an object property boxes
the array (Because the object exists on the heap)? If so, then this is my
problem. How do I box the array while keeping it strong-typed?

If assigning a byte array to a class property (on the same thread) DOES box
the array, then I'm at a loss to the problem. Are there any tips or tricks
to determining what exactly is being stored on the stack?

Of course because error handling requires stack space, and my stack is
invalid due to an overflow, it's very difficult trying to track down this
bug.

EXAMPLE CODE (Short version: not actual code. Just to demonstrate how my
code handles the array):

do{
byte[] buffer = new byte[4196];
p_Stream.Read(buffer,0,4196);
DoStuff(buffer);
DataRecord dr = new DataRecord();
dr.Data = buffer;
} while(i++<500)

Thanks
--ROBERT

 

[Robert Bouillon]

Moved the thread to microsoft.public.dotnet.framework as "Stack Memory Leak
in C#".

Please feel free to keep continue responding here if you wish.

--ROBERT

 

[Stu Smith]

A couple of things you may want to check, since you're making calls to
unmanaged code:

1) Check your interop declarations are exactly correct. Getting parameter or
return types wrong will corrupt the stack. I've seen all sorts of 'random'
errors (eg ExecutionEngineException, StackOverflowException) caused by one
issue (return type was long instead of int, or vice-versa).

2) Check objects aren't being collected or moved while they're referenced by
an unmanaged function.
a) If the unmanaged function needs a pointer to a managed object, you
may need to pin that object.
b) If the unmanaged function takes some sort of reference to an object,
that object may not have any roots left (functions, both managed and
unmanaged, don't count as a GC root). You may need to keep the object alive
after the unmanaged call using GC.KeepAlive(). That goes for 'this' too --
it can be collected while in a member function, if it's not used!

Stu

Willy,

Thanks for the reply

My call stack is only 27 calls deep: 17 of them are Windows Forms calls (via
"Show non-user code"). I checked all of my threads as well and they're all
behaving normally. To-date, on StackOverflows caused by recursion, I've
always gotten an exception. I've tried to reproduce this error 3 times, and
only once did I get this exception. The other 2 times I got obscure errors
and was unable to view my local variables, leading me to believe that lack
of stack space caused my VM to go screwy.

I narrowed it down to what I believe may be the culprit. I recently added a
"Status" subclass to my Rs232Stream class. The Status object monitors the
Serial Port in an infinite loop, on background thread, making calls to
win32. Disabling this thread alleviates the problem.

Here's a snippet (C#):

=====================================
/// <summary>
/// Monitors the Serial Port opened by the parent stream for events
/// </summary>
/// <remarks>
/// Sits directly on the m_StatusThread
/// <see cref="Parent"/>
/// <see cref="m_StatusThread"/>
/// </remarks>
private void Monitor()
{
InitEventMask();

while(true)
{
if(!WaitCommEvent(p_Parent.hPort, m_UnmanagedCurrentEvent,
m_UnmanagedOverlappedHandle))
{
NativeError err = Win32Exception.Check(false);
if (err!= NativeError.ERROR_IO_PENDING)
throw new IOException("Error setting comm event handle", new
Win32Exception());
}

m_StatusChangeEvent.WaitOne();
Requery(p_Parent.hPort);
}
}

public Rs232Status(Rs232Stream parent)
{
p_Parent = parent;

Overlapped wo = new Overlapped();
wo.hEvent = m_StatusChangeEvent.Handle;

m_UnmanagedOverlappedHandle =
Marshal.AllocHGlobal(Marshal.SizeOf(wo));
Marshal.StructureToPtr(wo, m_UnmanagedOverlappedHandle, true);

m_UnmanagedCurrentEvent =
Marshal.AllocHGlobal(Marshal.SizeOf(Type.GetType("System.UInt32")));
Marshal.WriteInt32(m_UnmanagedCurrentEvent,0);
}

/// <summary>
/// Checks the status of the Flow Control Lines
/// </summary>
/// <param name="portHandle">Handle to the opened port</param>
/// <remarks>Called by Requery</remarks>
public void UpdateLines()
{
ModemStatus ms;
while(!GetCommModemStatus(p_Parent.hPort,out ms))
if(Win32Exception.Check(false)==NativeError.ERROR_IO_PENDING)
continue;
else
throw new Win32Exception("Error during GetCommModemStatus");
{...} //---> Event Calls and Field Updates
}
===============================
InitEventMask encapsulates the Win32 SetCommMask call. Requery is a lot of
bloated code that checks the Event Flags and calls some OnXXXX methods to
fire events and update some local fields.

It's pretty cut-and-dry. It's an infinite loop that makes a few minor calls
to unmanaged IO code. No unmanaged memory is allocated outside of the ctor,
so I'm really at a total loss as to where I'm leaking stack memory.

I don't mind posting the entire status class here, but I know how painful
that is so I'm going to just post the calls I make to unmanaged code or the
marshaller. Perhaps you'll see a function that manipulates memory or catch
something I'm missing?

[DllImport("kernel32.dll")]
EscapeCommFunction
ClearCommError
GetOverlappedResult
WaitCommEvent
SetCommMask
GetCommModemStatus

[.NET Interop]
Marshal.ReadInt32
Marshal.StructureToPtr

Here's an idea: My EscapeCommFunction has the following declaration:
[DllImport("kernel32.dll")]
private static extern bool EscapeCommFunction(IntPtr hFile, CommFunction
dwFunc);

CommFunction is a private managed struct I use in the class. I prefer to let
.NET interop marshal my variables for me to save time / code. Is it possible
that in this scenario, the dwFunc is never being destroyed? I'm not
destroying it explicitly.

Thanks.
--ROBERT

P.S. Good to know about arrays. I could have sworn I read something that
said blittable-typed arrays were stored on the stack. At least that gives
stackalloc a little more prominence I wish there was a way that
strong-typed streams could access stackalloc arrays, but we all know the
rules about pointers in the CLR :-/

 

[Robert Bouillon]

Stu,

Thanks for the reply. I actually found the problem and posted the solution.
Oddly enough, MS's news servers sometime just won't take the post. Maybe
it's outlook express.

At any rate, it was actually a problem with my P/Invoke declaration.
Detailed information on making proper P/Invoke calls takes a lot more
digging than I thought. I added a couple of "In" attributes, and changed or
removed some out and ref statements accordingly. So far so good.

--ROBERT

A couple of things you may want to check, since you're making calls to
unmanaged code:

1) Check your interop declarations are exactly correct. Getting parameter or
return types wrong will corrupt the stack. I've seen all sorts of 'random'
errors (eg ExecutionEngineException, StackOverflowException) caused by one
issue (return type was long instead of int, or vice-versa).

2) Check objects aren't being collected or moved while they're referenced by
an unmanaged function.
a) If the unmanaged function needs a pointer to a managed object, you
may need to pin that object.
b) If the unmanaged function takes some sort of reference to an object,
that object may not have any roots left (functions, both managed and
unmanaged, don't count as a GC root). You may need to keep the object alive
after the unmanaged call using GC.KeepAlive(). That goes for 'this' too --
it can be collected while in a member function, if it's not used!

Stu

Willy,

Thanks for the reply

My call stack is only 27 calls deep: 17 of them are Windows Forms calls (via
"Show non-user code"). I checked all of my threads as well and they're all
behaving normally. To-date, on StackOverflows caused by recursion, I've
always gotten an exception. I've tried to reproduce this error 3 times, and
only once did I get this exception. The other 2 times I got obscure errors
and was unable to view my local variables, leading me to believe that lack
of stack space caused my VM to go screwy.

I narrowed it down to what I believe may be the culprit. I recently

added

a
"Status" subclass to my Rs232Stream class. The Status object monitors the
Serial Port in an infinite loop, on background thread, making calls to
win32. Disabling this thread alleviates the problem.

Here's a snippet (C#):

=====================================
/// <summary>
/// Monitors the Serial Port opened by the parent stream for events
/// </summary>
/// <remarks>
/// Sits directly on the m_StatusThread
/// <see cref="Parent"/>
/// <see cref="m_StatusThread"/>
/// </remarks>
private void Monitor()
{
InitEventMask();

while(true)
{
if(!WaitCommEvent(p_Parent.hPort, m_UnmanagedCurrentEvent,
m_UnmanagedOverlappedHandle))
{
NativeError err = Win32Exception.Check(false);
if (err!= NativeError.ERROR_IO_PENDING)
throw new IOException("Error setting comm event handle", new
Win32Exception());
}

m_StatusChangeEvent.WaitOne();
Requery(p_Parent.hPort);
}
}

public Rs232Status(Rs232Stream parent)
{
p_Parent = parent;

Overlapped wo = new Overlapped();
wo.hEvent = m_StatusChangeEvent.Handle;

m_UnmanagedOverlappedHandle =
Marshal.AllocHGlobal(Marshal.SizeOf(wo));
Marshal.StructureToPtr(wo, m_UnmanagedOverlappedHandle, true);

m_UnmanagedCurrentEvent =
Marshal.AllocHGlobal(Marshal.SizeOf(Type.GetType("System.UInt32")));
Marshal.WriteInt32(m_UnmanagedCurrentEvent,0);
}

/// <summary>
/// Checks the status of the Flow Control Lines
/// </summary>
/// <param name="portHandle">Handle to the opened port</param>
/// <remarks>Called by Requery</remarks>
public void UpdateLines()
{
ModemStatus ms;
while(!GetCommModemStatus(p_Parent.hPort,out ms))
if(Win32Exception.Check(false)==NativeError.ERROR_IO_PENDING)
continue;
else
throw new Win32Exception("Error during GetCommModemStatus");
{...} //---> Event Calls and Field Updates
}
===============================
InitEventMask encapsulates the Win32 SetCommMask call. Requery is a lot of
bloated code that checks the Event Flags and calls some OnXXXX methods to
fire events and update some local fields.

It's pretty cut-and-dry. It's an infinite loop that makes a few minor calls
to unmanaged IO code. No unmanaged memory is allocated outside of the ctor,
so I'm really at a total loss as to where I'm leaking stack memory.

I don't mind posting the entire status class here, but I know how painful
that is so I'm going to just post the calls I make to unmanaged code or the
marshaller. Perhaps you'll see a function that manipulates memory or catch
something I'm missing?

[DllImport("kernel32.dll")]
EscapeCommFunction
ClearCommError
GetOverlappedResult
WaitCommEvent
SetCommMask
GetCommModemStatus

[.NET Interop]
Marshal.ReadInt32
Marshal.StructureToPtr

Here's an idea: My EscapeCommFunction has the following declaration:
[DllImport("kernel32.dll")]
private static extern bool EscapeCommFunction(IntPtr hFile, CommFunction
dwFunc);

CommFunction is a private managed struct I use in the class. I prefer to let
.NET interop marshal my variables for me to save time / code. Is it possible
that in this scenario, the dwFunc is never being destroyed? I'm not
destroying it explicitly.

Thanks.
--ROBERT

P.S. Good to know about arrays. I could have sworn I read something that
said blittable-typed arrays were stored on the stack. At least that gives
stackalloc a little more prominence I wish there was a way that
strong-typed streams could access stackalloc arrays, but we all know the
rules about pointers in the CLR :-/