Array size cannot be specified in declaration?

[herobeat]

Hi all,

I'm having a hell of a time with declaring a struct to hold some
binary data I'm trying to read from some files on disk. What I would
like to do is something like this:

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[16] MD5Checksum;
}

In addition to the normal numeric stuff, there is a field I need to
read that's a 16-byte checksum. The problem is that when I try to do
this, I get CS0270, Array size cannot be specified in a variable
declaration (try initializing with a 'new' expression).

I'm a bit confused. I need a field that's exactly 16 bytes long
without actually instantiating 16 bytes of memory in the struct
declaration. Surely I don't have to do something silly like:

public byte MD5Checksum01;
public byte MD5Checksum02;
public byte MD5Checksum03;
...

Yes, I know, I could just use a couple of UInt64s, but what if the
field were 1024 bytes long? Would I be stuck declaring UInt64s
instead of bytes as shown above?

What is the best practice for declaring a struct with a byte array
like this?

Thanks for any help and/or advice.

P.S. Whether I use a struct or a class is irrelevant; I get the same
error.

 

[Nir]

Hi all,

I'm having a hell of a time with declaring a struct to hold some
binary data I'm trying to read from some files on disk. What I would
like to do is something like this:

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[16] MD5Checksum;

}

In addition to the normal numeric stuff, there is a field I need to
read that's a 16-byte checksum. The problem is that when I try to do
this, I get CS0270, Array size cannot be specified in a variable
declaration (try initializing with a 'new' expression).

I'm a bit confused. I need a field that's exactly 16 bytes long
without actually instantiating 16 bytes of memory in the struct
declaration. Surely I don't have to do something silly like:

public byte MD5Checksum01;
public byte MD5Checksum02;
public byte MD5Checksum03;
...

Yes, I know, I could just use a couple of UInt64s, but what if the
field were 1024 bytes long? Would I be stuck declaring UInt64s
instead of bytes as shown above?

What is the best practice for declaring a struct with a byte array
like this?

Thanks for any help and/or advice.

P.S. Whether I use a struct or a class is irrelevant; I get the same
error.

Hello,

If I understand you problem correctly, than you may use one the the
following two solutions:
1)

Declare the struct like this:

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[] MD5Checksum;
}

later in you code, you may want to set the size of your array:
binHeader a;
a.MD5Checksum = new byte[16];


2)

Alternatively you can use a class:

public class binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[] MD5Checksum = new byte[16];
}


Hope this helps,
Nir Levy

 

[G.Doten]

Hi all,

I'm having a hell of a time with declaring a struct to hold some
binary data I'm trying to read from some files on disk. What I would
like to do is something like this:

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[16] MD5Checksum;
}

In addition to the normal numeric stuff, there is a field I need to
read that's a 16-byte checksum. The problem is that when I try to do
this, I get CS0270, Array size cannot be specified in a variable
declaration (try initializing with a 'new' expression).

I'm a bit confused. I need a field that's exactly 16 bytes long
without actually instantiating 16 bytes of memory in the struct
declaration. Surely I don't have to do something silly like:

public byte MD5Checksum01;
public byte MD5Checksum02;
public byte MD5Checksum03;
...

Yes, I know, I could just use a couple of UInt64s, but what if the
field were 1024 bytes long? Would I be stuck declaring UInt64s
instead of bytes as shown above?

What is the best practice for declaring a struct with a byte array
like this?

Thanks for any help and/or advice.

P.S. Whether I use a struct or a class is irrelevant; I get the same
error.

Try this:

public byte[] MD5Checksum = new MD5Checksum[16];

"byte[]" is a type (an array of bytes).

To load up your struct from the binary file you'd need a read method
that reads the appropriate number of bytes from the file and fills in
the struct appropriately.

 

[G.Doten]

Hi all,

I'm having a hell of a time with declaring a struct to hold some
binary data I'm trying to read from some files on disk. What I would
like to do is something like this:

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[16] MD5Checksum;

}

In addition to the normal numeric stuff, there is a field I need to
read that's a 16-byte checksum. The problem is that when I try to do
this, I get CS0270, Array size cannot be specified in a variable
declaration (try initializing with a 'new' expression).

I'm a bit confused. I need a field that's exactly 16 bytes long
without actually instantiating 16 bytes of memory in the struct
declaration. Surely I don't have to do something silly like:

public byte MD5Checksum01;
public byte MD5Checksum02;
public byte MD5Checksum03;
...

Yes, I know, I could just use a couple of UInt64s, but what if the
field were 1024 bytes long? Would I be stuck declaring UInt64s
instead of bytes as shown above?

What is the best practice for declaring a struct with a byte array
like this?

Thanks for any help and/or advice.

P.S. Whether I use a struct or a class is irrelevant; I get the same
error.

Hello,

If I understand you problem correctly, than you may use one the the
following two solutions:
1)

Declare the struct like this:

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[] MD5Checksum;
}

later in you code, you may want to set the size of your array:
binHeader a;
a.MD5Checksum = new byte[16];


2)

Alternatively you can use a class:

public class binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[] MD5Checksum = new byte[16];
}


Hope this helps,
Nir Levy

Brain cramp! Nir's right. My previous reply only works if it is a class
and not a struct. Another way to keep it as a struct is like this:

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[] MD5Checksum;

public binHeader(uint id, uint offset, byte[] checksum)
{
Id = id;
Offset = offset;
MD5Checksum = checksum;
}
}

 

[Jon Skeet [C# MVP]]

I'm having a hell of a time with declaring a struct to hold some
binary data I'm trying to read from some files on disk. What I would
like to do is something like this:

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[16] MD5Checksum;
}

If you really need the byte array to be "inline" with the rest of the
struct, you'll have to use unsafe code and the "fixed" modifier. This
is only available in C# 2 (and higher) by the way - hopefully that
won't be an issue for you.

How many instances of this struct will you have though? If it's not a
*huge* amount, I'd stick with the more straightforward practice of
creating the byte array separately and storing a reference in the
struct. Aside from anything else, that way you don't need to deal with
unsafe code, which can be messier to work with *and* is less well
understood in the community. (I certainly don't know much about unsafe
code, as I very, *very* rarely use it.)

 

[herobeat]

Maybe I'm asking the wrong question.

As per an earlier post, I am trying to read binary data from a file in
the most efficient manner possible. Ideally, something like this
would work. (Please keep in mind this is pseudo-ish code, meant
mainly to convey the idea.)

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[16] MD5Checksum;
}

Now, to read it from the file, I would do something like this:

FileStream fs = File.OpenRead("myfile.bin");
BinaryReader reader = new BinaryReader(fs);
binHeader header = new binHeader;
header = reader.ReadBytes(sizeof header);

Or, if I wanted to read a thousand of them, something like this:

FileStream fs = File.OpenRead("myfile.bin");
BinaryReader reader = new BinaryReader(fs);
binHeader[] header = new binHeader[1000];
header = reader.ReadBytes(sizeof header * 1000);

I know, I know, that's more C-ish than C Sharp-ish, but such is my
background. I'm trying to learn the New Way™; really, I am. So
instead of asking about a specific implementation, I'll just ask for
more along the lines of advice.

I need to read a large number of fixed-size data structures, some of
which contain byte array fields, not just primitive types, quickly and
efficiently from a formatted binary file, some of which are over a
gigabyte. Right now, I've got some classes that read the data
piecemeal (get me a 32-bit integer... now get me another 32-bit
integer... now get me a 16-byte checksum...), and they work, but the
performance is dreadful.

Isn't there some good way of pulling, for example, 2,016 bytes of data
out of a file and ending up with, say, an array of 84 C# data
structures as shown above (24 bytes each), each one addressable as
myObject.Id, myObject.Offset, and myObject.MD5Checksum, in one
operation? You know, without having to read a record, copy it into a
structure, read another record, copy it into a structure, and so on?
And definitely without having to read a UInt32 and copy it over, read
another UInt32 and copy it over, then read a byte array and copy it
over? That's what I'm doing now, and like I said, the performance is
awful. There's just got to be a better way.

 

[G.Doten]

Maybe I'm asking the wrong question.

As per an earlier post, I am trying to read binary data from a file in
the most efficient manner possible. Ideally, something like this
would work. (Please keep in mind this is pseudo-ish code, meant
mainly to convey the idea.)

public struct binHeader
{
public UInt32 Id;
public UInt32 Offset;
public byte[16] MD5Checksum;
}

Now, to read it from the file, I would do something like this:

FileStream fs = File.OpenRead("myfile.bin");
BinaryReader reader = new BinaryReader(fs);
binHeader header = new binHeader;
header = reader.ReadBytes(sizeof header);

Or, if I wanted to read a thousand of them, something like this:

FileStream fs = File.OpenRead("myfile.bin");
BinaryReader reader = new BinaryReader(fs);
binHeader[] header = new binHeader[1000];
header = reader.ReadBytes(sizeof header * 1000);

I know, I know, that's more C-ish than C Sharp-ish, but such is my
background. I'm trying to learn the New Way™; really, I am. So
instead of asking about a specific implementation, I'll just ask for
more along the lines of advice.

I need to read a large number of fixed-size data structures, some of
which contain byte array fields, not just primitive types, quickly and
efficiently from a formatted binary file, some of which are over a
gigabyte. Right now, I've got some classes that read the data
piecemeal (get me a 32-bit integer... now get me another 32-bit
integer... now get me a 16-byte checksum...), and they work, but the
performance is dreadful.

Isn't there some good way of pulling, for example, 2,016 bytes of data
out of a file and ending up with, say, an array of 84 C# data
structures as shown above (24 bytes each), each one addressable as
myObject.Id, myObject.Offset, and myObject.MD5Checksum, in one
operation? You know, without having to read a record, copy it into a
structure, read another record, copy it into a structure, and so on?
And definitely without having to read a UInt32 and copy it over, read
another UInt32 and copy it over, then read a byte array and copy it
over? That's what I'm doing now, and like I said, the performance is
awful. There's just got to be a better way.

In that case, you have to fill in your struct with the appropriate
methods from the binary reader. Like:

myObject.MD5Checksum = new byte[16];
reader.ReadBytes(myObject.MD5Checksum, 16);

(Or something like that, I'm writing this from memory.)

In other words, the binary reader will let you get at the information in
the file in binary format--you have to map those binary numbers to the
appropriate high-level structure or whatever. It sounds like you are
actually doing this already which is the way it is done.

 

[G.Doten]

gigabyte. Right now, I've got some classes that read the data
piecemeal (get me a 32-bit integer... now get me another 32-bit
integer... now get me a 16-byte checksum...), and they work, but the
performance is dreadful.

What makes you think the performance is dreadful? Do you have numbers
showing how long it takes? And if so, what are you comparing those
numbers to?

 

[Jon Skeet [C# MVP]]

Maybe I'm asking the wrong question.

As per an earlier post, I am trying to read binary data from a file in
the most efficient manner possible.

It would be nice to get a bit more information on this before we go too
much further.

1) What kind of efficiency are you after? Memory or speed?
2) How many of these will you be reading in real life?
3) How many do you want to have in memory at any one time?

*Often* performance comes at the cost of elegance or readability. It's
always worth having a clear idea of just how tight you need something
to be before you start.

I'm sure that with the use of unsafe code and P/Invoke we can get it
all fiendishly fast - at the cost of readability, flexibility,
maintainability etc. Alternatively, we can chip away at things to keep
as much of the "goodness" as possible until we hit the target
performance, only optimising the points which are really significant.

If I/O is the problem, then using BinaryReader's FillBuffer method may
well make a huge difference - fill the buffer with as much data as you
need, then convert it appropriately. That means temporarily having two
copies of the data in memory of course, but you can strike a balance
between the amount to buffer and the speed involved.

You might also look at Buffer.BlockCopy, but as I said before you'd
need to use unsafe code to get the fixed sized buffer "inline".

 

[Mike D Sutton]

Maybe I'm asking the wrong question.

As per an earlier post, I am trying to read binary data from a file in
the most efficient manner possible. Ideally, something like this
would work. (Please keep in mind this is pseudo-ish code, meant
mainly to convey the idea.)

Now, to read it from the file, I would do something like this:

FileStream fs = File.OpenRead("myfile.bin");
BinaryReader reader = new BinaryReader(fs);
binHeader header = new binHeader;
header = reader.ReadBytes(sizeof header);

<snip>

Try something like this, perhaps:

***
public struct binHeader {
public UInt32 Id;
public UInt32 Offset;
public byte[] MD5Checksum;

// Checksum size
public const int MD5ChecksumSize = 16;

// Initialise struct from stream
public binHeader(BinaryReader inStream) {
if (inStream != null) {
Id = inStream.ReadUInt32();
Offset = inStream.ReadUInt32();
MD5Checksum = inStream.ReadBytes(MD5ChecksumSize);
} else { // Populate with default data
Id = 0;
Offset = 0;
MD5Checksum = new byte[MD5ChecksumSize];
}
}

// Read single binHeader structure from stream
public static binHeader FromStream(BinaryReader inStream) {
return new binHeader(inStream);
}

// Read multiple binHeader structures from stream
public static binHeader[] FromStream(BinaryReader inStream, int inCount) {
if ((inCount > 0) && (inStream != null)) {
binHeader[] RetArr = new binHeader[inCount];

for (int i = 0; i < inCount; i++)
RetArr = new binHeader(inStream);

return RetArr;
} else
return null;
}
}
***

Then you can use it like:

***
FileStream fs = File.OpenRead("myfile.bin");
BinaryReader reader = new BinaryReader(fs);
binHeader header = new binHeader(reader);
***

Or to read multiple structures:

***
FileStream fs = File.OpenRead("myfile.bin");
BinaryReader reader = new BinaryReader(fs);
binHeader[] headers = binHeader.FromStream(reader, 1000);
***

Hope this helps,

Mike


- Microsoft Visual Basic MVP -
E-Mail: (e-mail address removed)
WWW: Http://EDais.mvps.org/