Fastest way to copy from a one dimension byte array to two dimension int array

[johannblake]

I need to copy all the data from a one dimensional byte array to a
specific column in a two dimensional int array. For example:

byte[] byteArray = {4, 5, 6};
int[,] intArray = new int[2, 3];

After copying to the first column in intArray, I want the intArray to
have the following data:

intArray[0, 0] has the value 4
intArray[0, 1] has the value 5
intArray[0, 2] has the value 6

I actually need to copy thousands of bytes, so what is the fastest
solution without using a loop?

Thanks,
Johann

 

[Stefan Simek]

Hi Johann,

How many bytes do you really need to copy? If we're talking about
thousands (maybe hundreds of thousands), then performance shouldn't be
an issue. Even if it is, you certainly won't get out of it without using
loops, because the source array element size is different than the
destination. There is no function I'm aware of that would stretch a byte
array to an int array. You might get a little performance increase
through using unsafe code, though. But I would resort to this only in
case the simplest approach would be a terrible performance killer.

HTH,
Stefan

 

[johannblake]

I need to copy about 30000 bytes within a single thread but there are
about 9 threads that need to run simultaneously, so the net effect is
270000 bytes. This must all happen well under a second. You're probably
right about having to use unsafe code but that isn't an issue. Anything
that accomplishes the job. Iterating 270000 times sounds kind
horrifying for performance issues.

 

[Stefan Simek]

I need to copy about 30000 bytes within a single thread but there are
about 9 threads that need to run simultaneously, so the net effect is
270000 bytes. This must all happen well under a second. You're probably
right about having to use unsafe code but that isn't an issue. Anything
that accomplishes the job. Iterating 270000 times sounds kind
horrifying for performance issues.

You have to measure.

I've just done a few benchmark, copying bytes from a byte[270000] to an
int[270, 1000] array, using three methods. In all cases, the source and
destination arrays were preallocated in order to avoid garbage
collections during the benchmark:

[Benchmark]
public static void TestSimple()
{
for (int i = 0; i < src.Length; i++)
dest[i / 1000, i % 1000] = src;
}

[Benchmark]
public static void TestDouble()
{
int psrc = 0;

for (int y = 0; y < 270; y++)
{
for (int x = 0; x < 1000; x++)
{
dest[y, x] = src[psrc++];
}
}
}

[Benchmark]
public unsafe static void TestUnsafe()
{
fixed (int* fd = dest)
{
int* pd = fd;

fixed (byte* fs = src)
{
byte* ps = fs;

for (int i = 0; i < 270000; i++)
*pd++ = *ps++;
}
}
}

And the results were as follows:

-------------------------------------------------
Benchmarking 'BenchmarkFX.ArrayTest - Void TestSimple()'
Generating code...
Running benchmark...
Result: Performed 1000 iterations in 8,33078142615884 sec
Single call took: 8,331 ms
-------------------------------------------------
Benchmarking 'BenchmarkFX.ArrayTest - Void TestDouble()'
Generating code...
Running benchmark...
Result: Performed 1000 iterations in 3,05974032866275 sec
Single call took: 3,060 ms
-------------------------------------------------
Benchmarking 'BenchmarkFX.ArrayTest - Void TestUnsafe()'
Generating code...
Running benchmark...
Result: Performed 3000 iterations in 2,56928532893266 sec
Single call took: 856,428 us
Done


As you can see, the biggest performance killer appears to be the div /
mod in the simple case. I think the performance gained through
optimizing the code is negligible (even though it's an order of
magnitude from the simplest case to unsafe code), as long as it's
required to run "well under a second".

All the tests were made using a P4C, running 2.6GHz

HTH,
Stefan

 

[Marcus Andrén]

I need to copy about 30000 bytes within a single thread but there are
about 9 threads that need to run simultaneously, so the net effect is
270000 bytes. This must all happen well under a second. You're probably
right about having to use unsafe code but that isn't an issue. Anything
that accomplishes the job. Iterating 270000 times sounds kind
horrifying for performance issues.

You are seriously overestimating the time the task takes. The
following code is with managed code. (And I doubt unmanaged code will
make it much faster). If you run the following code you will notice
that it runs quite fast, probably 10 ms at most.

public static void CopyByteToInt(byte[] source, int[,] target, int
column)
{
System.Diagnostics.Trace.Assert(source.Length ==
target.GetLength(1));

for (int i = 0; i < source.Length; i++)
target[column, i] = source;
}

public static void Main()
{
byte[] byteArray = new byte[30000];
int[,] intArray = new int[2, 30000];

DateTime start = DateTime.Now;
for (int i = 0; i < 10; i++)
CopyByteToInt(byteArray, intArray, 0);
Console.WriteLine(DateTime.Now - start);
}

 

[Willy Denoyette [MVP]]

I need to copy about 30000 bytes within a single thread but there are
about 9 threads that need to run simultaneously, so the net effect is
270000 bytes. This must all happen well under a second. You're probably
right about having to use unsafe code but that isn't an issue. Anything
that accomplishes the job. Iterating 270000 times sounds kind
horrifying for performance issues.

What do you mean exactly with ... 9 threads that need to run
simultaneously...?
Unless you have a 9 way SMP box, these threads won't run simultaneously,
such number of threads that are CPU bound are useless on a mono CPU box, and
make the whole process to take more time to execute than a single thread.
So make sure that when you realy need them that they are not CPU bound.

Willy.