Array/ArrayList Performance

[Guest]

Hey Everyone -

I'm trying to determine the fastest way of moving array information around
and could use some help.

Here's the setup:
Class A stores a DateTime and an amount (a payment).
Class B is functionally an array of A objects (representing a cashflow).
Class C is functionally an array of B objects (representing an organization
of cashflows).
Class D contains a C object.
Class E contains a static method that takes a collection of D objects, does
some computations, creates a bunch of B objects, one at a time, and basically
calls D.C.Add(B) for each.

I'm trying to determine the fastest way of moving the information created in
E into the appropriate D objects. I've tried various situations including
having B and C extend CollectionBase, changing the ArrayList of B objects in
C to an array, using AddRange() vs. CopyTo(), etc...

1. class E
2. {
3. static Method(D[] pD)
4. {
5. C mC = new C();
6. foreach (D d in pD)
7. {
8. B mB = Compute(d);
9. mC.Add(mB);
10. }
11. for (int i = 0; i < mC.Length; i++)
12. {
13. pD.c = new C(1);
14. pD.c[0].b = new B();
15. pD.c[0].b.AddRange(mC);
16. }
17. }
18. }

Basically, I want to optimize lines 13 - 15. Here are some setups I used
for lines 13-15 and the resulting time the method took to run those lines
when pD contains a total of ~20million A objects:

Case 1:
14. pD.c[0] = new B(mC.Length); // sets the capacity for b's list
15. pD.c[0].b.AddRange(mC);
Time: 5.23 seconds

Case 2:
14. pD.c[0] = new B(mC.Length); // sets the capacity for b's list
15. pD.c[0].b = mC; // direct access to b
Time: 4.16 seconds

Case 3:
13-14. pD.c.Add(new B(mC.Length)); // sets the capacity for b's list
and adds the new b to c
15. pD.c[0].b = mC;
Time: 5.76 seconds

Case 4:
13-14. pD.c.Add(new B(mC.Length)); // sets the capacity for b's list
and adds the new b to c
15. mC.CopyTo(mD.c[0].b, 0);
Time: 5.18 seconds

Case 5:
Then I tried the following just for testing purposes and found that it ran
much more quickly.
13. ArrayList mT = new ArrayList();
14. B[] mB = new B[mC.Length];
14.5. mT.Add(mB);
15. mC.CopyTo((B[])mT[0], 0);
Time: 0.15 seconds!!!
It creates dummy objects limited to the scope of E.Method() and moves the
computed mC[] information to those dummy objects just to see how fast the
operation could take.

Any thoughts on why it takes at least 4 more seconds setting the computed
information to pD versus the setup used in Case 5? Or any suggestions on how
to really streamline this operation? Hopefully at least some of this makes
sense...

Thanks in advance.

Craig

 

[Bruce Wood]

A little bit of it makes sense, but it sounds to me as though you've
answered your own question.

For almost all business applications the difference between arrays and
ArrayList is irrelevant. However, your application is one of those few
cases in which the volume of data is such that it matters.

If you know how many entries you need in advance (as you appear to for
the size of your B array), arrays are always faster than ArrayList,
because there's simply less machinery on top of them. Not much less,
but enough, as you've discovered, to make a difference when the number
of operations bumps up into the tens of millions.

ArrayList is a more flexible structure, but that flexibility comes with
a cost. Not a very high cost, but a cost nonetheless.

Other than that, all I can say is that you're going about this in
exactly the right way: write the code, use a profiler to find out where
the bottleneck is, and optimize just enough to remove the bottleneck.

I say this because some programmers reading this post might think, "Oh,
then I'll use arrays everywhere, because they're 'more efficient'." You
have, correctly, not optimized what is not a problem, and found the
problem before looking to optimize. That's the correct way to write
fast code.