Using an array of bytes with fixedoffset in a struct?

[sexauthor]

I'm converting a VB6 application over that called a 3rd party DLL with
specific data structures. The VB6 code defined custom types for those
data structures (ie: one with the specific data types, then one as a
word array, and one as a byte array, for use in generating CRCs etc)
and would LSet one to the other to copy data over.

Naturally we can't do that in .NET. I've done lots of reading in the
interoperability books and the closest thing I've been able to come up
with is:

Public Const dmq_s_user_data As Integer = 20 ' bytes

<StructLayout(LayoutKind.Explicit)> Structure dmq_user_struct
' <FieldOffset(0)> <VBFixedString(dmq_s_user_data)> Dim
dmq_b_user_data() As Byte
' <FieldOffset(0)> <MarshalAs(UnmanagedType.ByValArray,
sizeconst:=dmq_s_user_data, arraysubtype:=UnmanagedType.AsAny)> Dim
dmq_b_user_data() As Byte
' <FieldOffset(0)> <MarshalAs(UnmanagedType.ByValTStr,
sizeconst:=dmq_s_user_data)> Dim dmq_b_user_data() As Byte

<FieldOffset(0)> Dim dmq_l_user_1 As Integer
<FieldOffset(4)> Dim dmq_l_user_2 As Integer
<FieldOffset(8)> Dim dmq_l_user_3 As Integer
<FieldOffset(12)> Dim dmq_l_user_4 As Integer
<FieldOffset(16)> Dim dmq_l_user_5 As Integer

Public Sub Initialize()
' ReDim dmq_b_user_data(dmq_s_user_data)
End Sub
End Structure 'dmq_user_struct

You can see there's the data itself, plus commented out ways of
defining access to the data as a byte array. You can compile any of
these to try, but you'll get an exception when you run it because
arrays are a reference and you can't overlap references and values in
structs like this.

That's a huge problem for me. It turns out that in C# 2.0 you could
define an array of bytes in-place with a "fixed" keyword. So in a
structure you could have ...

fixed byte dmq_b_user_data(dmq_s_user_data)

And it would translate to something like ...

byte dmq_b_user_data_1
byte dmq_b_user_data_2
byte dmq_b_user_data_3
... dmq_s_user_data

Instead of being just a pointer to that array. I haven't been able to
find anything in VB.NET, or any similar language construct to
"fixed". I always get a pointer, and so I can't use that.

I've read other suggestions to specifically define each byte as a
reference type, but that's not going to work well for me seeing as
some of these structures are really large and it would be too
cumbersome.

The other suggestion was define one structure with the named fields,
and one structure of just the byte array, and use
Marshal.StructureToPtr and Marshal.Copy to copy the bytes from one to
the other. I can do that, but this is a performance application and I
don't want to unless I have to.

Any comments, suggestions, hints, gripes from people with the same
problem? I'd be particularly interested if there's been a "fixed" in-
place array type introduced in VB.NET somewhere that I'm not aware
of.

Thanks.

 

[Tom Shelton]

I'm converting a VB6 application over that called a 3rd party DLL with
specific data structures. The VB6 code defined custom types for those
data structures (ie: one with the specific data types, then one as a
word array, and one as a byte array, for use in generating CRCs etc)
and would LSet one to the other to copy data over.

Naturally we can't do that in .NET. I've done lots of reading in the
interoperability books and the closest thing I've been able to come up
with is:

Public Const dmq_s_user_data As Integer = 20 ' bytes

<StructLayout(LayoutKind.Explicit)> Structure dmq_user_struct
' <FieldOffset(0)> <VBFixedString(dmq_s_user_data)> Dim
dmq_b_user_data() As Byte

VBFixedString is not valid in interop scenarios. It is used by those
crappy VB file functions.

' <FieldOffset(0)> <MarshalAs(UnmanagedType.ByValArray,
sizeconst:=dmq_s_user_data, arraysubtype:=UnmanagedType.AsAny)> Dim
dmq_b_user_data() As Byte
' <FieldOffset(0)> <MarshalAs(UnmanagedType.ByValTStr,
sizeconst:=dmq_s_user_data)> Dim dmq_b_user_data() As Byte

<FieldOffset(0)> Dim dmq_l_user_1 As Integer
<FieldOffset(4)> Dim dmq_l_user_2 As Integer
<FieldOffset(8)> Dim dmq_l_user_3 As Integer
<FieldOffset(12)> Dim dmq_l_user_4 As Integer
<FieldOffset(16)> Dim dmq_l_user_5 As Integer

Public Sub Initialize()
' ReDim dmq_b_user_data(dmq_s_user_data)
End Sub
End Structure 'dmq_user_struct

You can see there's the data itself, plus commented out ways of
defining access to the data as a byte array. You can compile any of
these to try, but you'll get an exception when you run it because
arrays are a reference and you can't overlap references and values in
structs like this.

That's a huge problem for me. It turns out that in C# 2.0 you could
define an array of bytes in-place with a "fixed" keyword. So in a
structure you could have ...

fixed byte dmq_b_user_data(dmq_s_user_data)

Yes C# 2.0 added that, but it can only be used in unsafe code - and has
some other restrictions as well.

And it would translate to something like ...

byte dmq_b_user_data_1
byte dmq_b_user_data_2
byte dmq_b_user_data_3
... dmq_s_user_data

Instead of being just a pointer to that array. I haven't been able to
find anything in VB.NET, or any similar language construct to
"fixed". I always get a pointer, and so I can't use that.

You won't - because VB.NET doesn't allow unsafe code, and that's the
only place that fixed is legal in C#.

I've read other suggestions to specifically define each byte as a
reference type, but that's not going to work well for me seeing as
some of these structures are really large and it would be too
cumbersome.

The other suggestion was define one structure with the named fields,
and one structure of just the byte array, and use
Marshal.StructureToPtr and Marshal.Copy to copy the bytes from one to
the other. I can do that, but this is a performance application and I
don't want to unless I have to.

Any comments, suggestions, hints, gripes from people with the same
problem? I'd be particularly interested if there's been a "fixed" in-
place array type introduced in VB.NET somewhere that I'm not aware
of.

It might be helpful if you were to provide the C/C++ defintions of this
structure. It's really hard to know what your doing with out seeing the
original

 

[Bill McCarthy]

Try something like :


<StructLayout(LayoutKind.Explicit)> Structure dmq_user_struct
<FieldOffset(0)> <MarshalAs(UnmanagedType.ByValArray,
sizeconst:=dmq_s_user_data, arraysubtype:=UnmanagedType.U1)> Dim
dmq_b_user_data() As Byte

'<FieldOffset(0)> Dim dmq_l_user_1 As Integer
'<FieldOffset(4)> Dim dmq_l_user_2 As Integer
'<FieldOffset(8)> Dim dmq_l_user_3 As Integer
'<FieldOffset(12)> Dim dmq_l_user_4 As Integer
'<FieldOffset(16)> Dim dmq_l_user_5 As Integer

Public Sub Initialize()
ReDim dmq_b_user_data(0 To dmq_s_user_data - 1)
End Sub

Private Const dmq_s_user_data As Integer = 20 ' bytes
End Structure 'dmq_user_struct

 

[Lloyd Sheen]

VBFixedString is not valid in interop scenarios. It is used by those
crappy VB file functions.


Yes C# 2.0 added that, but it can only be used in unsafe code - and has
some other restrictions as well.


You won't - because VB.NET doesn't allow unsafe code, and that's the
only place that fixed is legal in C#.


It might be helpful if you were to provide the C/C++ defintions of this
structure. It's really hard to know what your doing with out seeing the
original

MS has a tool or a link to one that will translate the C/C++ structs
correctly for Dot.Net

http://msdn2.microsoft.com/en-us/magazine/cc164193.aspx

Hope this helps
Lloyd Sheen

 

[Tom Shelton]

MS has a tool or a link to one that will translate the C/C++ structs
correctly for Dot.Net

http://msdn2.microsoft.com/en-us/magazine/cc164193.aspx

Hope this helps
Lloyd Sheen

Interesting... I always just do the conversions by hand

 

[sexauthor]

Bill, I could do it like that but I then would not be able to access
that byte array easily as Integers.

It might be helpful if you were to provide the C/C++ defintions of this
structure. It's really hard to know what your doing with out seeing the
original

Ok, here is the VB6 code paraphrased by request. This is the simplest
structure, the code has more complex ones but they all work on the
same principle - lots of data values and then another type/struct that
splits it by bytes or words.

Public Const dmq_s_user_data As Long = 20

Public Type dmq_user_string
dmq_b_user_data(0 To dmq_s_user_data - 1) As Byte
End Type 'dmq_user_string

Public Type dmq_user_struct
dmq_l_user_1 As Long
dmq_l_user_2 As Long
dmq_l_user_3 As Long
dmq_l_user_4 As Long
dmq_l_user_5 As Long
End Type 'dmq_user_struct

Dim the_struct as dmq_user_struct
Dim the_string As dmq_user_string

' Fill the struct
the_struct.dmq_l_user_1 = 100
the_string.dmq_l_user_2 = 200
the_string.dmq_l_user_3 = 200
the_string.dmq_l_user_4 = 200
the_string.dmq_l_user_5 = 200
' Convert it to a string
the_string = LSet(the_struct)
' Pass it as an argument to a DLL
...

Because I was really hurting to get this phase of the conversion
project done last night, this is how I handled it. Clunky, but at
least the app runs (milestone!). Note: this is not the line for line
code, I actually managed to throw away the struct listed above and
used this code on a different one, but I've changed the code to
illustrate the technique.

Option Strict Off
Option Explicit On
Imports System.Runtime.InteropServices
Imports System.IO
Imports System.Runtime.Serialization.Formatters.Binary
Module Conversion

Public Function RawSerialize(ByRef Anything As Object) As Byte()
Call debug_message(80, "- Starting RawSerialize")
Dim Size As Integer = Marshal.SizeOf(Anything)

Dim RawDatas() As Byte
ReDim RawDatas(Size)

Dim Buffer As IntPtr
Buffer = Marshal.AllocHGlobal(Size)

Marshal.StructureToPtr(Anything, Buffer, False)
Marshal.Copy(Buffer, RawDatas, 0, Size)
Marshal.FreeHGlobal(Buffer)
RawSerialize = RawDatas

Call debug_message(80, "- Ending RawSerialize")
End Function

Public Function RawDeserialize(ByVal RawDatas As Byte(), ByVal
AnyType As Type) As Object
Call debug_message(80, "- Starting RawDeserialize")
Dim Size As Integer = Marshal.SizeOf(AnyType)

Dim Buffer As IntPtr
Buffer = Marshal.AllocHGlobal(size)

Marshal.Copy(RawDatas, 0, Buffer, Size)
RawDeserialize = Marshal.PtrToStructure(Buffer, AnyType)
Marshal.FreeHGlobal(Buffer)

Call debug_message(80, "- Ending RawDeserialize")
End Function
End Module

<StructLayout(LayoutKind.Sequential)> Public Structure dmq_user_struct
Dim dmq_l_user_1 As Integer
Dim dmq_l_user_2 As Integer
Dim dmq_l_user_3 As Integer
Dim dmq_l_user_4 As Integer
Dim dmq_l_user_5 As Integer
End Structure 'dmq_user_struct

<StructLayout(LayoutKind.Sequential)> Public Structure
dmq_user_struct_string
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=20)> Dim
dmq_user_string() As Byte

Public Sub Initialize()
ReDim dmq_user_string(20)
End Sub
End Structure 'dmq_header_as_words

...
Dim the_struct as dmq_user_struct
Dim the_string as dmq_user_struct_string

 

[sexauthor]

Bill, I could do it like that but I then would not be able to access
that byte array easily as Integers.


Ok, here is the VB6 code paraphrased by request. This is the simplest
structure, the code has more complex ones but they all work on the
same principle - lots of data values and then another type/struct that
splits it by bytes or words.

Public Const dmq_s_user_data As Long = 20

Public Type dmq_user_string
dmq_b_user_data(0 To dmq_s_user_data - 1) As Byte
End Type 'dmq_user_string

Public Type dmq_user_struct
dmq_l_user_1 As Long
dmq_l_user_2 As Long
dmq_l_user_3 As Long
dmq_l_user_4 As Long
dmq_l_user_5 As Long
End Type 'dmq_user_struct

Dim the_struct as dmq_user_struct
Dim the_string As dmq_user_string

' Fill the struct
the_struct.dmq_l_user_1 = 100
the_string.dmq_l_user_2 = 200
the_string.dmq_l_user_3 = 200
the_string.dmq_l_user_4 = 200
the_string.dmq_l_user_5 = 200
' Convert it to a string
the_string = LSet(the_struct)
' Pass it as an argument to a DLL
...

Because I was really hurting to get this phase of the conversion
project done last night, this is how I handled it. Clunky, but at
least the app runs (milestone!). Note: this is not the line for line
code, I actually managed to throw away the struct listed above and
used this code on a different one, but I've changed the code to
illustrate the technique.

Option Strict Off
Option Explicit On
Imports System.Runtime.InteropServices
Imports System.IO
Imports System.Runtime.Serialization.Formatters.Binary
Module Conversion

Public Function RawSerialize(ByRef Anything As Object) As Byte()
Call debug_message(80, "- Starting RawSerialize")
Dim Size As Integer = Marshal.SizeOf(Anything)

Dim RawDatas() As Byte
ReDim RawDatas(Size)

Dim Buffer As IntPtr
Buffer = Marshal.AllocHGlobal(Size)

Marshal.StructureToPtr(Anything, Buffer, False)
Marshal.Copy(Buffer, RawDatas, 0, Size)
Marshal.FreeHGlobal(Buffer)
RawSerialize = RawDatas

Call debug_message(80, "- Ending RawSerialize")
End Function

Public Function RawDeserialize(ByVal RawDatas As Byte(), ByVal
AnyType As Type) As Object
Call debug_message(80, "- Starting RawDeserialize")
Dim Size As Integer = Marshal.SizeOf(AnyType)

Dim Buffer As IntPtr
Buffer = Marshal.AllocHGlobal(size)

Marshal.Copy(RawDatas, 0, Buffer, Size)
RawDeserialize = Marshal.PtrToStructure(Buffer, AnyType)
Marshal.FreeHGlobal(Buffer)

Call debug_message(80, "- Ending RawDeserialize")
End Function
End Module

<StructLayout(LayoutKind.Sequential)> Public Structure dmq_user_struct
Dim dmq_l_user_1 As Integer
Dim dmq_l_user_2 As Integer
Dim dmq_l_user_3 As Integer
Dim dmq_l_user_4 As Integer
Dim dmq_l_user_5 As Integer
End Structure 'dmq_user_struct

<StructLayout(LayoutKind.Sequential)> Public Structure
dmq_user_struct_string
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=20)> Dim
dmq_user_string() As Byte

Public Sub Initialize()
ReDim dmq_user_string(20)
End Sub
End Structure 'dmq_header_as_words

...
Dim the_struct as dmq_user_struct
Dim the_string as dmq_user_struct_string

Oops the rest got cut off. Also noted above I mistyped something in
the example way above, it should look like this:

the_struct.dmq_l_user_1 = 100
the_struct.dmq_l_user_2 = 200
the_struct.dmq_l_user_3 = 200
the_struct.dmq_l_user_4 = 200
the_struct.dmq_l_user_5 = 200

' And resuming with the new code ...
the_string = RawDeserialize(RawSerialize(the_struct),
the_string.GetType())
' Now the_string has all the data from the_struct and can be
passed to the DLL

I'd seen code examples of this on the net but they didn't work for me
until I played around with the exact way I defined the structs. But
this works. I would have liked a more elegant solution though, and
I'm worried that the serialize/deserialize code might be buggy but I
couldn't find anything else.

 

[sexauthor]

Oops the rest got cut off. Also noted above I mistyped something in

the example way above, it should look like this:

the_struct.dmq_l_user_1 = 100
the_struct.dmq_l_user_2 = 200
the_struct.dmq_l_user_3 = 200
the_struct.dmq_l_user_4 = 200
the_struct.dmq_l_user_5 = 200

' And resuming with the new code ...
the_string = RawDeserialize(RawSerialize(the_struct),
the_string.GetType())
' Now the_string has all the data from the_struct and can be
passed to the DLL

I'd seen code examples of this on the net but they didn't work for me
until I played around with the exact way I defined the structs. But
this works. I would have liked a more elegant solution though, and
I'm worried that the serialize/deserialize code might be buggy but I
couldn't find anything else.

Another typo! Just to be complete, there is of course a call to
the_string.Initialize()
before you start using Serialize/Deserialize with it, otherwise it has
the tendency to crash.

 

[Tom Shelton]

Bill, I could do it like that but I then would not be able to access
that byte array easily as Integers.

... sure you could. You could use System.BitConverter to convert the
appropriate sections of the array.

Dim dmq_l_user_1 As Integer = BitConverter.ToInt32
(astruct.dmq_b_user_data, 0)
Dim dmq_l_user_2 As Integer =
BitConverter.ToInt32(astruct.dmq_b_user_data, 4)
....