VB6's Nothing & C# RCW Error

[Tyler]

I am attempting to extend a legacy VB6 application by making it use a .NET
component written in C# exposed through COM interop. Everything appeared to
be going well (VB application creates the .NET component instead of the
legacy VB6 component and invokes some methods successfully) until I hit a
snag.

For one method that is being invoked by the VB6 application, the method
takes 2 parameters (string values) and a custom interface. Depending on
processing, this custom interface may be a valid object, or may be
"Nothing". When this method is invoked on a legacy VB6 object implementing
this interface, no error is experienced. When this method is invoked on my
..NET component, I get the following error:
Object reference is not set to an instance of an object (0x8000403).
I have determined that the error is due to the custom interface parameter (I
can avoid the problem when it is caught in the VB6 debugger by creating an
instance of an object that implements the interface).

My question is, what causes the runtime callable wrapper that is generated
by C# to fail the request with the 0x8000403 error when the custom interface
is "Nothing" and, more importantly, is there anyway I can disable this error
or work-around the problem without having to change the VB6 legacy
application?

I hope this information is sufficiently detailed to allow a reader to
determine whether this is an obvious error with an obvious solution (just
one that I don't know). However, if it is not and an example is required,
please let me know and I will generate one. Any assistance is appreciated.

Thanks, Tyler

 

[Dmitriy Lapshin [C# / .NET MVP]]

Hi Tyler,

Does any of the parties (either the legacy app or the .NET component) assume
that this second parameter is always not null?
Could you post the C# code working with the parameter?

 

[Steven Cheng[MSFT]]

Hi Tyler,

Thank you for posting here. Regarding on the issue, I am
finding proper resource to assist you and we will update as soon as posible.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS IS",
with no warranties, and confers no rights.)

 

[Tyler]

I am currently in the process of creating a sample that I can post to
demonstrate the problem as it does not appear that this issue is a "known"
or "common" problem - although I expect I am just missing something obvious.
I will post this sample when I have it ready.

Tyler

 

[Tyler]

I have had a difficult time trying to create a sample application to
demonstrate my problem. However, possibly the following code snippets will
provide some insight?

The method is declared in the VB class as follows:
* Public Function Request(RequestString As String, _
* ByRef ResponseString As String, _
* Optional ByRef MyData As MyDataCollection =
Nothing) As Boolean


Later in the VB class, the method is invoked on an instance of a class
implementing the interface (my .NET component) as follows:
* Dim tvsRequest as String
* tvsRequest = "Hi"
* Dim tvsResponse as String
* Dim tvData as MyData
* IF NOT MyDataObj.Request(tvsRequest, tvsResponse, tvData) THEN
and this line generates the error message (0x80004003).

However, if I modify the line to be something like the following:
* IF NOT MyDataObj.Request("Hi", tvsResponse, tvData) THEN
the problem goes away.

Alternatively, if I add a line like the following before the IF statement,
the problem goes away too:
* SET tvData = New MyData

Can someone tell me what is going on?

Thanks, Tyler

 

[Tyler]

The legacy app does not assume it is always not null. Depending upon the
path it takes through the code, this parameter may have a value, or it may
not.

The C# code tests the parameter for validity before doing anything with it.

The problem I have is that the legacy VB component invokes the method on the
C# .NET object (exposed through COM interop) and an exception is thrown -
the call never makes it to the C# code. I know this because I use the C#
debugger to set breakpoints at the beginning of all my C# object's methods.
All other breakpoints are hit, but the breakpoint for this method is not
hit.

That leads me to believe that the COM interop wrapper is doing some checking
and rejecting the legacy VB6 request before it makes it to the .NET
component. If you see the other thread started on this topic, it appears to
be very flakey - I can get rid of the error by passing a literal string
instead of a string variable, for example.

I hope that helps to explain the problem I am experiencing.

Thanks, Tyler


Even though it never gets to it, the C# code is as follows (prototype
generated by referencing the VB6 COM library):

public bool Request(ref string rfvsRequest, ref string rfvsResponse, ref
MyData rfvData)
{
if( rfvData == null )
{
return false;
}

...
}

 

[Peter Huang]

Hi Tyler,

How to you declare the MyData in C# and VB?
Is it a Class or a Type?

It would be better if you can provide a simple reproduce sample for us to
reproduce the problem so that we can troubleshooting the problem at my side.
I will appreicated your efforts.


Best regards,

Peter Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.

 

[[MSFT]]

Hello,

Regarding the issue, our COM Interop team is investigating and will reply
you very soon.

Thanks,

Luke

 

[Rob Maushardt]

Hi Tyler,

Thanks for posting the clarification details and code snippets. BTW, I
didn't see you specify, so I'm assuming you are using v1.1 of the .NET
Framework. The Optional ByRef makes this interesting. There are some
peculiarities around this in COM Interop, so let's set them out before we
dig into the issue of implementing a C# component exposed to COM that
implements an interface defined like you show that accepts a custom
interface as that Optional ByRef.

First, C# doesn't have an equivalent to "optional" parametres. There is
the System.Runtime.InteropServices.OptionalAttribute that allows you to
declare methods that accept optional parametres, and you can declare them
in C#, you can even define them methods in C#, it's just that when you call
them, C# ignores the attribute (the language simply doesn't support that
Attribute). So that's one thing we need to keep in mind with this scenario.

Another thing that's interesting is how you mention passing the string
literal in the first parametre is changing all of this. I can't see how
that would affect the Optional parametre. I would need to repro this to
explain exactly what is happening there. By default, all params in VB6 are
ByRef, so the signature you posted is:

Public Function Request( _
RequestString As String, _
ByRef ResponseString As String, _
Optional ByRef MyData As MyDataCollection = Nothing _
) _
As Boolean
End Function

And that is equivalent to:

Public Function Request( _
ByRef RequestString As String, _
ByRef ResponseString As String, _
Optional ByRef MyData As MyDataCollection = Nothing _
) _
As Boolean
End Function

So, it's not clear to me how that string literal could affect the issue
with the "Optional ByRef MyData as MyDataCollection = Nothing" param. I'd
like to focus on this one right now because the NullReferenceException
("Object reference not set to an instance of an object.") should be coming
from the MyDataCollection param instead of the ref strings. I should
clarify that in an unmanaged client, the equivalent to a
NullReferenceException is the HRESULT you mentioned:

Error Result: 0x80004003 ( -2147467261 )
ID Defined as: E_POINTER
Source Error file: Winerror.h
Message Text: Invalid pointer

Because this originates in managed code, the message text you get back is
"Object reference not set to an instance of an object.".


To try and repro this, I built a C# DLL that implements the interface
defined by the VB6 DLL by defining this method:

public bool Request(
ref string rfvsRequest,
ref string rfvsResponse,
ref MyData rfvData) {
if( rfvData == null ) {
return false;
}
rfvsResponse = "Received: " + rfvsRequest;
return true;
}

When I run this from a VB6 client, I always get back false. By setting a
breakpoint in this method, I can see that rfvData is <undefined value> but
I don't get a NullReferenceException/E_POINTER. I was testing this with
v1.1 of the .NET Framework. All things being equal, it's likely that my
repro differs from your application in some way. So, to be certain of the
root cause here, you may want to open a case with Microsoft PSS to debug
this issue with your application.

That said, there are a few likely scenarios you can test and hopefully
isolate the root cause. First, I wanted to mention the
[OptionalAttribute]. You add this to the C# method signature so that
compilers that support optional parametres will do the right thing. In
your VB6 code, it always supplies something (sorry about the wording)
meaning that it supplies an instance of MyData or the VB6 Nothing keyword
kicks in and supplies an empty Variant. Even without this attribute, the
client code you posted should allow the CLR to handle this correctly, so I
just mention it here as something to think about for the future maintenance
and servicing of this code if it will ever be called from code that
supports Optional params (VB.NET does) and doesn't include a default value.

The other thing thing that may need to do here is to use the
MarshalAsAttribute declaring to COM Interop that this is an IUnknown
interface pointer you are passing, and then casting to the particular COM
interface when you use it. Something like this:

public bool Request(
ref string rfvsRequest,
ref string rfvsResponse,
[MarshalAs(UnmanagedType.IUnknown)] ref Object rfvData) {

MyAssembly.MyData obj1; // var to "extract" the COM interface

if( rfvData == null ) {
return false;
} else {
obj1= (MyAssembly.MyData) rfvData; // cast to the
type you need.
//...
return true;
}
}

Like I said, since I can't repro the exact problem you are seeing, I can't
be sure this is exactly what you need. It was not needed for me to get
this working, but there must be something different between my repro test
and your application. I do hope this helps get you on track to solve this
problem. If you are able to get this down to a repro you can post, I'd be
happy to continue investigating. And, as I mentioned, if it's not possible
for you to create a repro, you might consider opening a case with PSS to
debug this.

Cheers,
Rob

Rob Maushardt
Microsoft COM Team


This posting is provided "AS IS" with no warranties, and confers no rights.

 

[Tyler]

Thank you very much Rob,

I made some of the changes you mentioned (the optional OptionalAttribute)
and a few others that I read about online in the past week and it is now
working as I expected.

Thanks for your assistance!

Tyler