Pinvoke question: Am I causing this stack overflow?

G

Guest

Hi, TIA for any help.
(Thanks, Mattias Sjögren for your help yesterday! )

I am developing a C# wrapper for a 3rd party API. It processes buffers from
a video stream (file) and is structured where the initialization function
passes in two callback methods (my code) that the API calls to successively
read and write buffers. The C++ sample I am modeling this after works and
processes thousands of buffers, but when interface to my C# wrapper the API
gets a stack overflow every time the 18th buffer is read and the SUCCCESS
status is returned from the callback.

Q: Is there somethng in the way I am calling/being called by pinvoke that
is causing this stack overflow? I am attaching two posts
1) Facade class that calls the native methods (TpFacade.cs)
2) Driver class that calls the facade class. (ProcessStream.cs)

The stack overflow occurs when the GetStreamBuf method returns SUCCESS for
the 18th time.

Thanks for any suggestions!

+tom allen
Cupertino, CA.
 
G

Guest

//File: TpFacade.cs
using System;
using System.Collections.Generic;
using System.Text;
using System.Runtime.InteropServices; // DllImport

namespace TableProcessorFacade
{
public class TpFacade
{
public const int PACKET_SIZE = 188;
public const int SDT_SECTION_SIZE = 4096 * 100;

// TYPE OF THE TRANSPORT STREAM, Actual -> Actual TS, Other -> Other
TS (DVB SI ONLY)
public enum TS_TYPE { ActualTS, OtherTS } ;

// TYPE OF THE EIT TABLE (DVB SI ONLY)
public enum EIT_TYPE { EventSchedule, PresentFollowing } ;

//Interop Overview
//http://msdn2.microsoft.com/en-us/library/eaw10et3(VS.80).aspx
//
//Marshaling Data with Platform Invoke
//http://msdn2.microsoft.com/en-us/library/fzhhdwae(VS.80).aspx
//
//Article: Using P/Invoke to Call Unmanaged APIs from Your Managed
Classes
//http://msdn2.microsoft.com/en-us/library/aa719104(vs.71).aspx
//
//Default Marshaling for Delegates (Ensure that delegates are not
garbage collected after the InitProcessor call returns.)
//http://msdn2.microsoft.com/en-us/library/7esfatk4(VS.80).aspx
//
//Managed and Unmanaged Threading
//http://msdn2.microsoft.com/en-us/library/5s8ee185(VS.80).aspx
//
//Passing structures
//http://msdn2.microsoft.com/en-us/library/awbckfbz(VS.80).aspx
//
/////////////////////////
//HOWTO: DLL method names & ordinal location
//To find ordinal:
// dumpbin /exports tableprocessor.dll
//ordinal for InitProcessor is 33:
// 33 20 000193D0 ?InitProcessor@@YAHP6AHPAEH@Z1HHPAD@Z
//
/////////////////////////
//Newsgoup

//http://msdn.microsoft.com/newsgroup...13ef&mid=dc6d7478-bce3-45ad-8ec0-b77f95141f93
//
////////////////////////
//Types
// Note: UCHAR (in Wtypes.h) maps to SByte (signed byte) which
represents an 8-bit signed integer.
// The SByte type is not CLS-compliant. The CLS-compliant
alternative type is Int16.
// Warning: In some other contexts the name "UCHAR" refers to
"Unicode character", which is a 16-bit structure.
//
// Here substituting Byte array for UCHAR array.
//
// UCHAR maps to Byte (for our purposes here)
// BOOL maps to Booean (or Int32)
// int maps to Int32
// char* maps to String (decorate with Ansi, not Unicode)

//TableProcessor API
//File tableprocessor.h
// Defines the TableProcessor API functions exported from
tableprocessor.dll.
// #ifdef TPROCESSOR_EXPORTS
// #define TPROCESSOR_API __declspec(dllexport)
// #else
// #define TPROCESSOR_API __declspec(dllimport)
// #endif

#region //Managed function prototypes

// C++ signature
// InitProcessor - Intialize and start the Table Processor.
// Accepts two user defined callback functions to read and write TS
packets.
// Returns 0 on success and 1 on failure. NOTE: To exit the
tableprocessor module
// return 1 (failure) on any of the callback functions else return 0
// \param GetStreamBuf - function pointer to read the TS packet
// \param PutStreamBuf - function pointer to write the TS packet
// \param enableLogging - Set to TRUE if logging is to be enabled in
the API else FALSE
// \param logLevel - Level of the logging information to be
displayed, defaults to 0 (0 to 10)
// \param logFile - file to save the logging information to */
////int TPROCESSOR_API InitProcessor(int (*GetStreamBuf)(UCHAR *Buf,
int buf_len),
//// int (*PutStreamBuf)(UCHAR *Buf,
int buf_len),
//// //int streamType,
//// BOOL enableLogging=FALSE,
//// //BOOL displayLog,
//// int logLevel=0,
//// char* logFile=NULL );

public delegate Int32 GetStreamBufCallback(
[Out][MarshalAs(UnmanagedType.LPArray,SizeParamIndex = 1)]
Byte[] Buf, Int32 buf_len);
public delegate Int32 PutStreamBufCallback(
[In][MarshalAs(UnmanagedType.LPArray,SizeParamIndex = 1)] Byte[]
Buf, Int32 buf_len);

[DllImport("tableprocessor.dll", CharSet = CharSet.Ansi, EntryPoint
= "#33", CallingConvention=CallingConvention.StdCall)]
private static extern int InitProcessor(
[MarshalAs(UnmanagedType.FunctionPtr)] GetStreamBufCallback GsCb,
[MarshalAs(UnmanagedType.FunctionPtr)] PutStreamBufCallback PsCb,
Boolean enableLogging,
Int32 logLevel,
String logFile
);

// C++ signature
// GetActualPresentEit - Returns the present/following EIT sections
for actual Transport stream - DVB.
// present/followig EIT Sections.
// Returns 0 on success and 1 on failure.
// param buf - char buffer containing the EIT sections.
//int TPROCESSOR_API GetActualPresentEit(UCHAR *buf) ;

[DllImport("tableprocessor.dll", CharSet = CharSet.Ansi, EntryPoint
= "#33", CallingConvention=CallingConvention.StdCall)]
private static extern int GetActualPresentEit(
[Out] [MarshalAs(UnmanagedType.LPArray, SizeConst =
SDT_SECTION_SIZE)] Byte[] Buf);

#endregion //Managed function prototypes

#region //Facade functions

public static bool InitProcessorFacade(
GetStreamBufCallback GsCb,
PutStreamBufCallback PsCb,
Boolean enableLogging,
Int32 logLevel,
String logFile
)
{
//Return "0 -> SUCCESS"
return (0 == InitProcessor(
GsCb,
PsCb,
enableLogging,
logLevel,
logFile
));
}//InitProcessorFacade

//TODO [In, Out] vs ref
public static bool GetActualPresentEitFacade([In, Out] Byte[] Buf)
{
//Return "0 -> SUCCESS")
return (0 == GetActualPresentEit(Buf));
}//GetActualPresentEitFacade

#endregion //Facade Functions

}//Class TpFacade

}//namespace TableProcessorFacade

-- Tom Allen
 
G

Guest

//File: ProcessStream.cs
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading; //AutoEvent
using System.Timers; //ModTimer
using System.IO; //FileStream

using TableProcessorFacade; //Facade for TableProcessor API

namespace MicrosoftCorp_MSTV_Metadata
{
public class ProcessStream
{
//Processing flags
Boolean done;
Boolean pModsAllCompleted, fModsAllCompleted;

const int SUCCESS = 0; //return callback status to API
const int FAILURE = 1; //return callback status to API
static TpFacade.TS_TYPE TsType = TpFacade.TS_TYPE.ActualTS;
static TpFacade.EIT_TYPE EitType = TpFacade.EIT_TYPE.PresentFollowing;

System.IO.FileStream fsInput;
System.IO.FileStream fsOutput;

static TpFacade.GetStreamBufCallback GetStreamBufCb;
static TpFacade.PutStreamBufCallback PutStreamBufCb;

//Progress indicators
short ReadProgressRollover65535 = 0;
short WriteProgressRollover65535 = 0;
char CheckEitTablesRollover256 = (char)0;

//Threading objects
//lock for thread sync with API processing

//ms-help://MS.VSCC.v80/MS.MSDN.v80/MS.NETDEVFX.v20.en/cpref12/html/T_System_Threading_AutoResetEvent.htm
static AutoResetEvent m_autoEventProcessingDone;
static System.Timers.Timer ModTimer;

public void Process(RunParmsDataset.RunFileSetRow fSet,
ProposedEventPair evtPair, StreamEventsDataset.EventServiceRow EventTriplet,
bool WriteStreamEventsLog, bool WritePacketsBeforeModifiedEitEvent,
System.Int64 SkipBytesBeforeEitCheck)
{
//processing flags
done = false;
pModsAllCompleted = false;
fModsAllCompleted = false;

//Disable API logging
string apiLogFileString = String.Empty;
bool m_EnableLogging = false;
int m_LogLevel = 0;

//Open files
//TODO Validate input file path better.
string InputPath = Path.GetFullPath(fSet.InputFilePath);
string OutputPath = Path.GetFullPath(fSet.OutputFilePath);
fsInput = File.OpenRead(InputPath);
fsOutput = File.OpenWrite(OutputPath);

//Maintain reference to callback delegates to avoid garbage
collection following return from InitProcessorFacade.
GetStreamBufCb = GetStreamBuf;
PutStreamBufCb = PutStreamBuf;

//TODO Exception handling, reporting, logging.

//prep, then call API InitProcessor
//The "done" clause of GetBuffer releases this lock.
m_autoEventProcessingDone = new AutoResetEvent(false);

//TODO: Ensure that delegates are not garbage collected after
the InitProcessor call returns.
// "If, however, the callback function can be invoked after the
call returns,
// the managed caller must take steps to ensure that the
delegate remains uncollected
// until the callback function finishes. For detailed
information about preventing garbage collection,
// see Interop Marshaling with Platform Invoke. "
//HandleRef Sample
//http://msdn2.microsoft.com/en-us/library/hc662t8k(VS.80).aspx

BeginProcessing(apiLogFileString, m_EnableLogging, m_LogLevel);

m_autoEventProcessingDone.WaitOne();

//Processing completed for current file set
}//Process

private void BeginProcessing(string apiLogFileString, bool
m_EnableLogging, int m_LogLevel)
{
// initialize the API
if (TpFacade.InitProcessorFacade(GetStreamBufCb, PutStreamBufCb,
m_EnableLogging, m_LogLevel, apiLogFileString))
{
return;
}
}//BeginProcessing

#region TableProcessor API callbacks

// Callback function used by the API to read packet from the
application
public Int32 GetStreamBuf(Byte[] buf, Int32 bufSize)
{
if (fsInput != null)
{
try
{
if (fsInput.Read(buf, 0, bufSize) == TpFacade.PACKET_SIZE)
{
Console.Write("->{0}", ReadProgressRollover65535);
Console.Out.Flush();

//stability - wait for API threads
System.Threading.Thread.Sleep(200);

//progress indicator
if (ReadProgressRollover65535 == 0)
{
Console.Write("XX->");
Console.Out.Flush();
}
ReadProgressRollover65535++;
return SUCCESS; //return callback
status to API
}
else
{
Console.Write("Done reading input (failed read).");
Console.Out.Flush();
done = true;
return FAILURE; //return callback
status to API
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
done = true;
throw;
}
}
else
{
done = true;
Console.Write("Done reading input (eof).");
Console.Out.Flush();
return FAILURE;
}
}//GetStreamBuf


// Callback function used by the API to write packet to the
application
public Int32 PutStreamBuf(Byte[] buf, Int32 bufSize)
{
try
{
if (!(buf == null))
{
Console.Write("<-");
Console.Out.Flush();
fsOutput.Write(buf, 0, bufSize);

//TODO Stack overflow occurs regardless of this call.
//ModifyEit();
return SUCCESS; //Success == 0
}
m_autoEventProcessingDone.Set();
return FAILURE; //stop API thread?
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
//Processing completed. Release sync lock.
m_autoEventProcessingDone.Set();
throw;
}
}

private void ModifyEit()
{
if (!done)
{
//TODO Why allocate this here?
byte[] buf = new byte[TpFacade.SDT_SECTION_SIZE]; //4096*100
//Accumulate EIT event tables
//Get the ActualPresentEit table
if (TpFacade.GetActualPresentEitFacade(buf))
{
Console.Write("\nFound GetActualPresentEit\n");
Console.Out.Flush();
}
}
}//PutStreamBuf
#endregion //TableProcessor API callbacks

}//class ProcessStream
}

--Tom Allen
 
B

Ben Voigt [C++ MVP]

Tom Allen said:
Hi, TIA for any help.
(Thanks, Mattias Sjögren for your help yesterday! )

I am developing a C# wrapper for a 3rd party API. It processes buffers
from
a video stream (file) and is structured where the initialization function
passes in two callback methods (my code) that the API calls to
successively
read and write buffers. The C++ sample I am modeling this after works and
processes thousands of buffers, but when interface to my C# wrapper the
API
gets a stack overflow every time the 18th buffer is read and the SUCCCESS
status is returned from the callback.

Q: Is there somethng in the way I am calling/being called by pinvoke that
is causing this stack overflow? I am attaching two posts
1) Facade class that calls the native methods (TpFacade.cs)
2) Driver class that calls the facade class. (ProcessStream.cs)

The stack overflow occurs when the GetStreamBuf method returns SUCCESS for
the 18th time.

Thanks for any suggestions!


You didn't specify a calling convention for the delegate types as far as I
can see, so you might be trashing your caller's stack (actually trashing his
EBP, ESP registers).
 
G

Guest

Ben,

Thanks much for the reply, and for taking the time to scan the code. Your
suggestion makes sense.

Sadly, today was my last day on this contract. I have forwarded the
information to my boss and I hope she will be able to test it out and post a
reply to this thread. Thanks again.

+tom allen
Cupertino, CA.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top