Structure/Class to Byte Array and Back again

[Scott Townsend]

So I need to talk to a devices that expects all of the bits and bytes I sent
it to be in specific places (not yet 100% defined).

I wanted to create a structure/class with all of the data in it and then
convert that to a byte array, pass it to the device, then get a reply and
then convert that to a structure.

I'm having issues with making sure what I've coded is correct. Cant figure
out how to define an array in structure that is a Fixed length.
Could someone tell me if I'm way off in what I'm trying to do?


Public Sub initpacketData(ByRef myPacketData As PacketData)
myPacketData.MaxCmdLength = 254
myPacketData.CmdLength = 0
System.Array.Resize(CType(myPacketData.SQLCmd, Char()), 254)
System.Array.Resize(CType(myPacketData.padding, Char()), 3840)

'myPacketData.SQLCmd = New String(" ", 254)
'myPacketData.padding = New String(" ", 3840)
End Sub

<StructLayout(LayoutKind.Explicit, Size:=4096, CharSet:=CharSet.Ansi)> _
Structure PacketData
<FieldOffset(0)> Dim MaxCmdLength As Byte
<FieldOffset(1)> Dim CmdLength As Byte
<FieldOffset(2)> Dim SQLCmd As String
<FieldOffset(25)> Dim padding As String
End Structure


Dim DataToSend As PacketData
Dim DataToReceieve As PacketData
Dim DataReceived As String
Dim iSizeOfStruct As Integer

initpacketData(DataToSend)
initpacketData(DataToReceieve)
DataToSend.MaxCmdLength = 254
DataToSend.CmdLength = message.Length
DataToSend.SQLCmd = message

'Set Up Our Pointer to Some Memory
Dim ptrDataToSend As IntPtr = Marshal.AllocHGlobal(4096)
'Copy the structure to our memory location @ pointer
Marshal.StructureToPtr(DataToSend, ptrDataToSend, True)

iSizeOfStruct = Marshal.SizeOf(DataToSend)

DataReceived = Marshal.PtrToStringUni(ptrDataToSend, 4096)

Dim ptrDataToReceieve As IntPtr = Marshal.AllocHGlobal(4096)
ptrDataToSend = Marshal.StringToHGlobalUni(DataReceived)
DataToReceieve = CType(Marshal.PtrToStructure(ptrDataToSend,
GetType(PacketData)), PacketData)
iSizeOfStruct = Marshal.SizeOf(DataToReceieve)
------


DataReceived does not seem to be what I expected, though converting it back
to a struct seemed to work.
The Size is always 12 too?

Thanks,
Scott<-

 

[Scott Townsend]

Now I'm getting the Following Error:

System.ArgumentException was unhandled
Message="The parameter is incorrect. (Exception from HRESULT: 0x80070057
(E_INVALIDARG))"
Source="mscorlib"
StackTrace:
at System.Runtime.InteropServices.Marshal.StructureToPtr(Object
structure, IntPtr ptr, Boolean fDeleteOld) at
SQLQueryClient.Form1.btnCopyStruct_Click(Object sender, EventArgs e)


Here is some UPdated Code...

<StructLayout(LayoutKind.Explicit, CharSet:=CharSet.Ansi, Pack:=1,
Size:=54)> _
Structure SQL_PARMS
<FieldOffset(0)> Dim Data_Type As Byte
<FieldOffset(1)> Dim Unused As Byte
<FieldOffset(4), VBFixedArray(50)> Dim Data_Value As Byte()
End Structure

<StructLayout(LayoutKind.Explicit, CharSet:=CharSet.Ansi, Pack:=1,
Size:=2048)> _
Structure SQL_COMMAND
<FieldOffset(0)> Dim PLC_Type As Int16
<FieldOffset(4), VBFixedString(256)> Dim SQLCmd As String
<FieldOffset(260), VBFixedArray(20)> Dim PARMS As SQL_PARMS()
<FieldOffset(1340), VBFixedArray(708)> Dim UNUSED As Byte
End Structure

Dim DataToSend As New SQL_COMMAND
Dim DataToReceieve As New SQL_COMMAND
Dim DataReceived As String
Dim iSizeOfStruct As Integer
Dim BytesToSend() As Byte
Dim mySQLParms(19) As SQL_PARMS

' initpacketData(DataToSend)
' initpacketData(DataToReceieve)

'Set up packet to send
DataToSend.PLC_Type = 100
DataToSend.SQLCmd = txtMessage.Text
'Parm1
mySQLParms(0).Data_Type = PLCDataTypes.PLC_SINT16
mySQLParms(0).Data_Value = (New SQLParmData_FromValue(18)).Data
'Parm2
mySQLParms(1).Data_Type = PLCDataTypes.PLC_STRING
mySQLParms(1).Data_Value = (New SQLParmData_FromValue("Paul
Deas")).Data
DataToSend.PARMS = mySQLParms

iSizeOfStruct = Marshal.SizeOf(DataToSend)




'Set Up Our Pointer to Some Memory
Dim ptrDataToSend As IntPtr = Marshal.AllocHGlobal(2048)
'Copy the structure to our memory location @ pointer
-> Marshal.StructureToPtr(DataToSend, ptrDataToSend, True)
'Chokes here ^^^^^^^^^^^^^
iSizeOfStruct = Marshal.SizeOf(DataToSend)

DataReceived = Marshal.PtrToStringUni(ptrDataToSend, 2048)
BytesToSend = ConvertStringToByteArray(DataReceived)

 

[Göran Andersson]

So I need to talk to a devices that expects all of the bits and bytes I sent
it to be in specific places (not yet 100% defined).

I wanted to create a structure/class with all of the data in it and then
convert that to a byte array, pass it to the device, then get a reply and
then convert that to a structure.

I'm having issues with making sure what I've coded is correct. Cant figure
out how to define an array in structure that is a Fixed length.
Could someone tell me if I'm way off in what I'm trying to do?

8<


DataReceived does not seem to be what I expected, though converting it back
to a struct seemed to work.
The Size is always 12 too?

Thanks,
Scott<-

Trying to make data being stored in a specific way in memory is a bit
tricky, especially when you try to put objects like strings as inline
data in a structure.

I would just use a MemoryStream and a BinaryWriter to write the data the
way it needed to be.

 

[Scott Townsend]

Hmmm...

From my last attempt I think I'm m uch closer, though I think my size is a
bit off.

So with all my strings I think I need to convert them to Char() arrays as
technically they would be in the structure as just characters and no
'standard' way to tell how long it is or where it ends as it is dependant on
the device I'm talking to. So I need to have the Max Length of the
potential strings in there.

Is there a easy way to copy the contents of a structure or regular objects
with padding to the MemoryStream? What I was doing before was stuff like
this:

This is a sip from the code that used to parse the byte stream that came in
from TCP, _SQLCLientData is the Bytestream.

So to get access to the Data in the byte stream I had to index everything
and know exactly where it is. If I could just overlay the bytestream to a
structure I would not have to know that Parameter 3 is @ (256 + (2 * 52) +
3). Much rather use: DataToSend.PARMS(2).Data_value

For i As Integer = 0 To 19
PLCParms(i).PLCDataType = _SQLCLientData(256 + (i * 52))
Console.Write("Type: " & PLCParms(i).PLCDataType.ToString & ",
Value: ")
'setup SQL Parameters
Dim mySQLParm As SqlParameter = New SqlParameter
mySQLParm.ParameterName = "@parm" & (i + 1).ToString

Select Case PLCParms(i).PLCDataType
Case 0
Console.WriteLine("NIL")
Case 1
PLCParms(i).PLCParmBool = CType((_SQLCLientData(256 + (i
* 52) + 3) = 1), Boolean)
mySQLParm.SqlDbType = SqlDbType.Bit
mySQLParm.Value = PLCParms(i).PLCParmBool
Console.WriteLine(PLCParms(i).PLCParmBool.ToString)

 

[Göran Andersson]

Hmmm...

From my last attempt I think I'm m uch closer, though I think my size is a
bit off.

So with all my strings I think I need to convert them to Char() arrays as
technically they would be in the structure as just characters

Actually, an array is also a reference type, so it would not be stored
as inline data in the structure either.

and no
'standard' way to tell how long it is or where it ends as it is dependant on
the device I'm talking to. So I need to have the Max Length of the
potential strings in there.

Is there a easy way to copy the contents of a structure or regular objects
with padding to the MemoryStream?

As all the data is actually not inside the structure, you would need
something that could incorporate reference types as inline data. Perhaps
the marshalling can do that, but not without attributes to tell it what
to do with the data.

As far as I can tell, the VBFixedArrayAttribute is not recognised by the
marshalling, so you would have to use the MarshalAsAttribute instead.

 

[Scott Townsend]

So I think I found the MarshalAsAttribute and reconfiguted my Structs.
THough now when I convert them to a pointer I'm getting an error.

So on this line of code below:
Marshal.StructureToPtr(DataToSend, ptrDataToSend, True)

It gives me the Following Error:
Type could not be marshaled because the length of an embedded array
instance does not match the declared length in the layout.
If I A dont have the Embedded Struct Array it seems to work out fine.

Here is all of the code:


<StructLayout(LayoutKind.Explicit, CharSet:=CharSet.Ansi)> _
Structure SQL_PARMS
<FieldOffset(0)> Dim Data_Type As Byte
<FieldOffset(1)> Dim Unused As Byte
<FieldOffset(4), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=50)> Dim Data_Value As Byte()
End Structure

<StructLayout(LayoutKind.Explicit, CharSet:=CharSet.Ansi)> _
Structure SQL_COMMAND
<FieldOffset(0)> Dim PLC_Type As Int16
<FieldOffset(4), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=256)> Dim SQLCmd As Byte()
<FieldOffset(260), MarshalAsAttribute(UnmanagedType.ByValArray,
ArraySubType:=UnmanagedType.ByValArray, SizeConst:=20)> Dim PARMS As
SQL_PARMS()
<FieldOffset(1340), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=708)> Dim UNUSED As Byte()
End Structure


Dim DataToSend As New SQL_COMMAND
Dim DataToReceieve As New SQL_COMMAND
Dim DataReceived As String
Dim iSizeOfStruct As Integer
Dim iSizeOfParms As Integer
Dim BytesToSend() As Byte
Dim mySQLParms(19) As SQL_PARMS

' initpacketData(DataToSend)
' initpacketData(DataToReceieve)

'Set up packet to send
DataToSend.PLC_Type = 100
DataToSend.SQLCmd = ConvertStringToByteArray(txtMessage.Text)
'Parm1
mySQLParms(0).Data_Type = PLCDataTypes.PLC_SINT16
mySQLParms(0).Data_Value = (New SQLParmData_FromValue(18)).Data
'Parm2
mySQLParms(1).Data_Type = PLCDataTypes.PLC_STRING
mySQLParms(1).Data_Value = (New SQLParmData_FromValue("Paul
Deas")).Data

DataToSend.PARMS = mySQLParms
iSizeOfStruct = Marshal.SizeOf(DataToSend)


'Set Up Our Pointer to Some Memory
Dim ptrDataToSend As IntPtr = Marshal.AllocHGlobal(2048)
'Copy the structure to our memory location @ pointer
Marshal.StructureToPtr(DataToSend, ptrDataToSend, True)

iSizeOfStruct = Marshal.SizeOf(DataToSend)

DataReceived = Marshal.PtrToStringUni(ptrDataToSend, 2048)
BytesToSend = ConvertStringToByteArray(DataReceived)
Dim ptrDataToReceieve As IntPtr = Marshal.AllocHGlobal(2048)
ptrDataToSend = Marshal.StringToHGlobalUni(DataReceived)
DataToReceieve = CType(Marshal.PtrToStructure(ptrDataToSend,
GetType(SQL_COMMAND)), SQL_COMMAND)
iSizeOfStruct = Marshal.SizeOf(DataToReceieve)

 

[Göran Andersson]

Unless you are enjoying the hassles with MarshalAsAttribute, etc., I would go
about this the other way around.

Create a class which has the desired byte array as its internal storage:
Dim Bytes(2047) As Byte ' or 4095, as appropriate

Add properties that incorporate the specifications of where things are. You
have to do that somewhere, either in the structure or in the class properties;
doing it in properties would make life easier. Each Get would convert the
appropriate bytes of the array to the desired type, and each Set would convert
the passed value to bytes and store them in the array. For the parameters, you
can have an indexed property that does the same thing for each parameter.

So for instance, the SQLCmd property Get would get the bytes from 4 to 260 and
convert them to a trimmed string. The Set would clear bytes 4 to 260, then
convert the passed string to bytes and store them in the array, starting at byte
4.

Once you have done that, your code would create an instance of the class, myCmd,
and can manipulate it as myCmd.SQLCmd = "Select blah..", in a perfectly normal
way. All the conversion to byte offsets is done inside the class.

This way, the actual storage is a single byte array, so passing it in and out of
streams, or to and from a device, will be much simpler.

Even easier, use a BinaryWriter and a MemoryStream, as I suggested first
off.

 

[Göran Andersson]

So I think I found the MarshalAsAttribute and reconfiguted my Structs.
THough now when I convert them to a pointer I'm getting an error.

So on this line of code below:
Marshal.StructureToPtr(DataToSend, ptrDataToSend, True)

It gives me the Following Error:
Type could not be marshaled because the length of an embedded array
instance does not match the declared length in the layout.
If I A dont have the Embedded Struct Array it seems to work out fine.

Here is all of the code:


<StructLayout(LayoutKind.Explicit, CharSet:=CharSet.Ansi)> _
Structure SQL_PARMS
<FieldOffset(0)> Dim Data_Type As Byte
<FieldOffset(1)> Dim Unused As Byte
<FieldOffset(4), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=50)> Dim Data_Value As Byte()
End Structure

<StructLayout(LayoutKind.Explicit, CharSet:=CharSet.Ansi)> _
Structure SQL_COMMAND
<FieldOffset(0)> Dim PLC_Type As Int16
<FieldOffset(4), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=256)> Dim SQLCmd As Byte()
<FieldOffset(260), MarshalAsAttribute(UnmanagedType.ByValArray,
ArraySubType:=UnmanagedType.ByValArray, SizeConst:=20)> Dim PARMS As
SQL_PARMS()
<FieldOffset(1340), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=708)> Dim UNUSED As Byte()
End Structure


Dim DataToSend As New SQL_COMMAND
Dim DataToReceieve As New SQL_COMMAND
Dim DataReceived As String
Dim iSizeOfStruct As Integer
Dim iSizeOfParms As Integer
Dim BytesToSend() As Byte
Dim mySQLParms(19) As SQL_PARMS

' initpacketData(DataToSend)
' initpacketData(DataToReceieve)

'Set up packet to send
DataToSend.PLC_Type = 100
DataToSend.SQLCmd = ConvertStringToByteArray(txtMessage.Text)
'Parm1
mySQLParms(0).Data_Type = PLCDataTypes.PLC_SINT16
mySQLParms(0).Data_Value = (New SQLParmData_FromValue(18)).Data
'Parm2
mySQLParms(1).Data_Type = PLCDataTypes.PLC_STRING
mySQLParms(1).Data_Value = (New SQLParmData_FromValue("Paul
Deas")).Data

DataToSend.PARMS = mySQLParms
iSizeOfStruct = Marshal.SizeOf(DataToSend)


'Set Up Our Pointer to Some Memory
Dim ptrDataToSend As IntPtr = Marshal.AllocHGlobal(2048)
'Copy the structure to our memory location @ pointer
Marshal.StructureToPtr(DataToSend, ptrDataToSend, True)

iSizeOfStruct = Marshal.SizeOf(DataToSend)

DataReceived = Marshal.PtrToStringUni(ptrDataToSend, 2048)
BytesToSend = ConvertStringToByteArray(DataReceived)
Dim ptrDataToReceieve As IntPtr = Marshal.AllocHGlobal(2048)
ptrDataToSend = Marshal.StringToHGlobalUni(DataReceived)
DataToReceieve = CType(Marshal.PtrToStructure(ptrDataToSend,
GetType(SQL_COMMAND)), SQL_COMMAND)
iSizeOfStruct = Marshal.SizeOf(DataToReceieve)

You don't assign anything to the UNUSED property, so that will be undefined.

 

[Scott Townsend]

So Maybe I shouldn't own a Copy Of Visual Studio 2008... (-;

I've programmed years and years ago, so I am a bit rusty.

So with help and tips from you I think I have what I need.

The main reason I wanted a defined Structure to overlap this Byte array is
that what I'm writing is a prototype and I'm not 100% sure what the end
result datastream might look like. Its going to have the main elements,
though we might want to add somethere here or there and move some things
around. I didn't want to have to do a bunch of while loops in my byte array
copying out chunks here and there.

With the Structure, I jsut change the Structure and all 'should' be pretty
good. I've re-written most of it as a standard class with managed objects,
Then that class deals with receiving the byte array, converting it to Public
objects, or definition of the values and then converts to Byte array.

I found 2 routines (RawSerialize, RawDeSerialize, they were in C#, so I had
to convert to VB.Net) that do the conversion from struct to byte array and
back, so those help, but I guess my understanding that the structs I was
defining were just pointers to 'nothing' that I needed to define the memory
space, and or objects of the 'right' length and assign to them before I
converted to the byte array.

I've included some of the code below.

<StructLayout(LayoutKind.Explicit, CharSet:=CharSet.Ansi)> _
Structure SQL_PARMS
<FieldOffset(0)> Dim Data_Type As Byte
<FieldOffset(4), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=50)> Dim Data_Value As Byte()
End Structure

<StructLayout(LayoutKind.Explicit, CharSet:=CharSet.Ansi)> _
Structure SQL_COMMAND
<FieldOffset(0)> Dim PLC_Type As Int16
<FieldOffset(4), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=256)> Dim SQLCmd As Byte()
<FieldOffset(260), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=1080)> Dim PARMS_BA As Byte()
<FieldOffset(1340), MarshalAsAttribute(UnmanagedType.ByValArray,
SizeConst:=708)> Dim UNUSED As Byte()
End Structure


Public Function RawSerialize(ByVal anything As Object) As Byte()
Dim rawsize As Int16
Dim buffer As IntPtr

rawsize = Marshal.SizeOf(anything)
buffer = Marshal.AllocHGlobal(rawsize)
Marshal.StructureToPtr(anything, buffer, True)

Dim rawdatas(rawsize - 1) As Byte
Marshal.Copy(buffer, rawdatas, 0, rawsize)
Marshal.FreeHGlobal(buffer)
Return rawdatas
End Function

Public Function RawDeserialize(ByVal rawdatas As Byte(), ByVal anytype
As Type) As Object
Dim rawsize As Int16
Dim buffer As IntPtr
Dim retobj As Object

rawsize = Marshal.SizeOf(anytype)
If (rawsize > rawdatas.Length) Then
Return Nothing
End If
buffer = Marshal.AllocHGlobal(rawsize)
Marshal.Copy(rawdatas, 0, buffer, rawsize)
retobj = Marshal.PtrToStructure(buffer, anytype)
Marshal.FreeHGlobal(buffer)
Return retobj
End Function

Public Sub GenStructureFromByte(ByVal Data As Byte())
Dim FixedSSQLCommand As New SQL_COMMAND
Dim FixedSQLParms As New SQL_PARMS
Dim i As Integer = 0
FixedSSQLCommand = RawDeserialize(Data,
FixedSSQLCommand.GetType)
PLC_Type = FixedSSQLCommand.PLC_Type
_RawSQLCmd = FixedSSQLCommand.SQLCmd
While (i < PARMS.Length)
PARMS(i) = New InternalSQL_PARMS
Dim ParmData As Byte() = New Byte(SIZE_OF_PARM_STRUCT) {}
System.Array.Copy(FixedSSQLCommand.PARMS_BA, (i *
SIZE_OF_PARM_STRUCT), ParmData, 0, SIZE_OF_PARM_STRUCT)
FixedSQLParms = RawDeserialize(ParmData,
FixedSQLParms.GetType)
PARMS(i).Data_Type = FixedSQLParms.Data_Type
PARMS(i).Data_Value = FixedSQLParms.Data_Value
i = i + 1
End While

End Sub

Public Function ReturnBytedata() As Byte()
Dim FixedSSQLCommand As New SQL_COMMAND
Dim FixedSQLParms As New SQL_PARMS
Dim SQLParmBytes As Byte()
Dim i As Integer = 0
FixedSSQLCommand.PLC_Type = PLC_Type
FixedSSQLCommand.SQLCmd = _RawSQLCmd
FixedSSQLCommand.PARMS_BA = New
Byte(SIZE_OF_PARM_ARRAY_ARRAY_BOUNDS) {}
FixedSSQLCommand.UNUSED = New
Byte(SIZE_OF_UNUSED_PARM_DATA_ARRAY_BOUNDS) {}
While (i < NumParms)
FixedSQLParms.Data_Type = PARMS(i).Data_Type
FixedSQLParms.Data_Value = PARMS(i).Data_Value
SQLParmBytes = RawSerialize(FixedSQLParms)
SQLParmBytes.CopyTo(FixedSSQLCommand.PARMS_BA, i *
SIZE_OF_PARM_STRUCT)
i = i + 1
End While
Return RawSerialize(FixedSSQLCommand)
End Function