StringBuilder

[Alvin Bruney]

On the advice of a user, I've timed stringbuilder v string. Here are the
results.

Here are the numbers:
Total # queries 3747
Time in Milliseconds

StringBuilder: String
460.6624 320.4608
350.504 220.3168
240.3456 230.3312
----------- -----------
Ave. 350.504 Ave. 257.0362

Sample query:
"Update ds_active_rate SET (rate, surcharge, min_sec, increment_sec,
begin_date, new_rate, new_rate_date, location_desc, category_id,
country_code, customer_number, user_name)=('0','0','0','0','2150-01-01
12:00:00','','','Afghanistan',2,93,'1STAMERICAN','') where rate_deck =
'COPYRATE' and location_id = 1001 and world_id = 1 and rate_level = 1"

Specs on my machine
1Gig SDRAM
P-IV 2Gig Intel
20Gig Hard Drive
Dell C-640

What I did: I let the routine run till it hit a breakpoint after a timespan
object. Noted the time, and pointed execution back to the first time object
and let it run to the break point. This gave me my three times. Obviously,
some sort of optimization was going on in the background for successive runs
which accounts for increasingly lower times. Then I converted the code to
string and repeated the procedure. I don't claim that these numbers are
scientific. But I did my best to control the process.

My conclusion is: I think these microsoft guys have been feeding us a bunch
of bullshit with this stringbuilder efficiency crap.

 

[Richard A. Lowe]

Strings are expensive to concatenate as a (partial)
function of their size... While I can't be sure of what
kind of test you were doing, if you test concatenation of
increasingly large strings to one another, the
StringBuilder can be dozens, if not hundreds of times
faster than the + operator.

If you are just concatenating two small strings, then yes,
the + operator will be faster.

Richard

 

[Jay B. Harlow [MVP - Outlook]]

Alvin,

My conclusion is: I think these microsoft guys have been feeding us a bunch
of bullshit with this stringbuilder efficiency crap.

If you are simply building a single update statement then yes the
StringBuilder maybe like using a shot gun to kill the fly on your foot.

However if you were to build 1000 update statements as a single batch, then
the StringBuilder will really shine!

In other words the first will not perform well (as I believe you
demonstrated), the second will fly!

for (I = 0; I < 1000; I++)
{
StringBuilder sb = new StringBuilder("insert(");
sb.Append("x");
sb.Append(")");
String sql = sb.ToString();
}

StringBuilder sb = new StringBuilder();
for (i = 0; i < 1000; i++)
{
sb.Append("insert(");
sb.Append("x");
sb.Append(")");
sb.Append(';'); // end of one statement
}
String sql = sb.ToString();

As in the first I am building 1000 individual commands, presumable to
execute 1000 individual times. Where as in the second I am building a single
batch with 1000 commands.

Forgive the super simple insert statement, I'm demonstrating how to use the
StringBuilder, not SQL syntax ;-)

Also irregardless of the timing, you also have temporary string objects to
consider, in the first case there may be fewer temporary string objects
created, so although its slightly 'slower' the GC does not need to work as
hard.

Don't forget when you allocate the StringBuilder, that you should give the
capacity value if you know roughly how large the string may get, as this
improves performance also. (the StringBuilder does not need to reallocate
its buffer, reallocating the buffer takes time!)

This gave me my three times. Obviously,
some sort of optimization was going on in the background for successive runs
which accounts for increasingly lower times.

Generally its the JIT compiler the first time through things get 'compiled'
the second time things are already compiled.

One last comment, as was discussed earlier (in a different thread), if you
are using string concatenation for the parameters to StringBuilder.Append,
then you may be negating the benefit of the StringBuilder, as you are
creating temporary string objects.

Depending on intended use of the class, I will actually allocate a single
StringBuilder in the constructor then continue to reuse the single
StringBuilder. Setting StringBuilder.Length = 0 each time I started. This
causes the contents of the buffer to be 'cleared' without allocating a new
StringBuilder buffer.

Hope this helps
Jay

 

[Alvin Bruney]

I apologize for not providing the code. The batch command explanation passed
over my head. I'd like you to explain this a little more. I believe the code
follows your quidelines. This is a raw cut and paste by the way.

for(int index = GlobalRateMan.INITVALUE; index < splitField2.Length;
index++)

{

splitField = GlobalRateMan.RegexComma.Split(splitField2[index]);

if(ForceCreate.Checked)

strTempString.Append(GlobalRateMan.sqlUpdateActiveRate2);

else

//insert extra fields

strTempString.Append(GlobalRateMan.sqlUpdateActiveRate1);

//check for valid record. replace some excel formatting

if(splitField.Length == 37)

{

newrate = splitField[NEW_RATE];

newdate = GlobalRateMan.FormatDate(splitField[NEW_RATE_DATE]);

//both fields must be entered or neither in the case of updates to field
like surcharges

if(newrate.Length > 0 && newdate.Length > 0)

{

strTempString.Append("'").Append(splitField[RATE]).Append("',");

strTempString.Append("'").Append(splitField[SURCHARGE]).Append("',");

strTempString.Append("'").Append(splitField[MIN_SEC]).Append("',");

strTempString.Append("'").Append(splitField[INCR_SEC]).Append("',");

strTempString.Append("'").Append(GlobalRateMan.FormatDate(splitField[BEGIN_D
ATE])).Append("',");

strTempString.Append("'").Append(newrate.Replace("$",GlobalRateMan.EMPTY)).A
ppend("',");

strTempString.Append("'").Append(newdate).Append("',");

strTempString.Append("'").Append(InsertUserName().TrimEnd()).Append("')");


//option which is checked for the time test

if(ForceCreate.Checked)

{

strTempString.Append("'").Append(splitField[8] + "',");

strTempString.Append(splitField[12]).Append(",");

strTempString.Append(splitField[14]).Append(",");

strTempString.Append("'").Append(splitField[2]).Append("',");

}

}


strTempString = strTempString.Replace("',)","')");

if(newrate.Length == 0 && newdate.Length == 0)

strTempString = strTempString.Replace("new_rate,new_rate_date,","");

//usually 3 guys are in this listbox

for(int i = 0; i < RateDeckConfirm.Items.Count; i++)

{

strTempString2.Length = 0;

strTempString2.Append(" where rate_deck =
'").Append(RateDeckConfirm.Items.Text.TrimEnd()).Append("'");

strTempString2.Append(" and location_id = ").Append(splitField[16]);

strTempString2.Append(" and world_id = ").Append(splitField[10]);

strTempString2.Append(" and rate_level = ").Append(splitField[34]);

//caught a live one

sql.Add(strTempString + strTempString2.ToString());

}

strTempString.Length = 0;

newrate = newdate = "";

}

}

 

[Jon Skeet]

On the advice of a user, I've timed stringbuilder v string.

Doing what with them? You haven't actually said what operations you're
performing on them in the first place.

Note that for doing simple benchmarks like this you might like to use
my benchmarking framework:

http://www.pobox.com/~skeet/csharp/benchmark.html

That's much easier than using breakpoints, and it means you can run
without the debugger attached (which is going to be much more like
real-world performance).

My conclusion is: I think these microsoft guys have been feeding us a bunch
of bullshit with this stringbuilder efficiency crap.

StringBuilder *is* more efficient for building strings - but as you
haven't said what you did in your tests, it's rather hard to tell what
you've actually measured.

If you really doubt that using StringBuilder is a good idea for
building up strings, I suggest you try these two programs:

using System;
using System.Text;

public class TestStringBuilder
{
public static void Main(string[] args)
{
DateTime start = DateTime.Now;
StringBuilder sb = new StringBuilder();
for (int i=0; i < 100000; i++)
{
sb.Append ('x');
}
DateTime end = DateTime.Now;
Console.WriteLine (sb.ToString().Length);
Console.WriteLine (end-start);
}
}



using System;
using System.Text;

public class TestString
{
public static void Main(string[] args)
{
DateTime start = DateTime.Now;
string s = "";
for (int i=0; i < 100000; i++)
{
s = s+'x';
}
DateTime end = DateTime.Now;
Console.WriteLine (s.Length);
Console.WriteLine (end-start);
}
}

 

[Hasani]

You didn't track the memory usage. which is one of the points of using the
stringbuilder over string

 

[Jon Skeet]

Depending on intended use of the class, I will actually allocate a single
StringBuilder in the constructor then continue to reuse the single
StringBuilder. Setting StringBuilder.Length = 0 each time I started. This
causes the contents of the buffer to be 'cleared' without allocating a new
StringBuilder buffer.

I don't believe this is a good idea, personally. Creating a single
empty StringBuilder is cheap - and the code is more natural. When you
want to build a string, you create a StringBuilder and use it. When you
want to build the next string, you create a new StringBuilder. No
threading worries, no extra member which has no logical reason for
existing, etc.

Also, things are likely to stay in generation 0 - why get stuff pushed
up to higher generations (where write barriers might become an issue)
unnecessarily?

Unless I had some reason to think that the StringBuilder code was the
bottleneck *and* I had profiled both ways of working, I'd use the more
natural code.

 

[Jon Skeet]

I apologize for not providing the code. The batch command explanation passed
over my head. I'd like you to explain this a little more. I believe the code
follows your quidelines. This is a raw cut and paste by the way.

Yes - if you could change the formatting so it came out looking
slightly more readable in future, that would be appreciated

You've shown the StringBuilder code, but not the String code.

I strongly suggest you refactor this code in a way that lets you do a
straight, repeatable benchmark (i.e. it's a complete program which
doesn't depend on any external factors, it has all the data constructed
before the loop is entered, etc) and post both cases of that. Certainly
based on that code, I would expect it to be significantly faster at
constructing the queries than straight string concatenations.

<snip>

 

[Jay B. Harlow [MVP - Outlook]]

Jon,
Doh! ;-)

That's why the statement is prefixed with 'depending on intended use of the
class'. Read as 'The StringBuilder is an integral part of the implementation
of the class'. In other words a number of methods of the class delegate to a
StringBuilder.

Similar to how the System.IO.StringWriter class wraps a StringBuilder class.

Otherwise I agree for most things its not a good idea. I suppose I should
have emphasized that more.

Thanks for bringing up this caution!

Jay

 

[Jon Skeet]

That's why the statement is prefixed with 'depending on intended use of the
class'. Read as 'The StringBuilder is an integral part of the implementation
of the class'. In other words a number of methods of the class delegate to a
StringBuilder.

Ah, to a single StringBuilder. I'd thought you'd meant it would be
effectively started afresh each time, and had nothing to do with the
logical state of the object.

Similar to how the System.IO.StringWriter class wraps a StringBuilder class.

That makes a lot of sense, yes.

Otherwise I agree for most things its not a good idea. I suppose I should
have emphasized that more.

Nah - don't blame yourself for my lack of comprehension skills

 

[Alvin Bruney]

Now hold on. I said they weren't scientific. I prefaced the results by that.
I don't have a lot of time on my hands to go about making sure this will
stand up to the hounds of hell. I leave it up to MS for that.

What difference does it make what i do with them. The question I was after
was which way is faster to build strings. FYI, i take these queries and
write to the database. I don't see what is gained by that tidbit.

This is a real world program by the way. Not an arbitrary couple lines of
code make up. If I can't see results there, and I should, then either I am
doing something wrong. In this case point it out so I can fix the code or MS
is on a marketing hype with this stringbuilder/.NET crap.

On the advice of a user, I've timed stringbuilder v string.

Doing what with them? You haven't actually said what operations you're
performing on them in the first place.

Note that for doing simple benchmarks like this you might like to use
my benchmarking framework:

http://www.pobox.com/~skeet/csharp/benchmark.html

That's much easier than using breakpoints, and it means you can run
without the debugger attached (which is going to be much more like
real-world performance).

My conclusion is: I think these microsoft guys have been feeding us a bunch
of bullshit with this stringbuilder efficiency crap.

StringBuilder *is* more efficient for building strings - but as you
haven't said what you did in your tests, it's rather hard to tell what
you've actually measured.

If you really doubt that using StringBuilder is a good idea for
building up strings, I suggest you try these two programs:

using System;
using System.Text;

public class TestStringBuilder
{
public static void Main(string[] args)
{
DateTime start = DateTime.Now;
StringBuilder sb = new StringBuilder();
for (int i=0; i < 100000; i++)
{
sb.Append ('x');
}
DateTime end = DateTime.Now;
Console.WriteLine (sb.ToString().Length);
Console.WriteLine (end-start);
}
}



using System;
using System.Text;

public class TestString
{
public static void Main(string[] args)
{
DateTime start = DateTime.Now;
string s = "";
for (int i=0; i < 100000; i++)
{
s = s+'x';
}
DateTime end = DateTime.Now;
Console.WriteLine (s.Length);
Console.WriteLine (end-start);
}
}

 

[Alvin Bruney]

True. I haven't tested for that. This was beyond the scope of the exercise.

 

[Alvin Bruney]

I used the editor to replace .append( with +
compiled and fixed the compiler issues. nothing more. nothing less. I have
projects due, i can't spend 10 hours preparing a bench mark.

I strongly suggest you refactor this code in a way that lets you do a

I think you are grasping at straws here. This is the same code, one running
stringbuilder, the other running string (formatted as described above). I'm
not seeing the results that was hyped about stringbuilder straight out of
the box. I wouldn't want to COOK the results (refactor or what ever the
latest terminology is) to get these results either. If it is as good as they
say, I shouldn't be jumping thru hoops to get the results. This isn't C++.

My analogy. If car A is faster than car B then car A should be faster on an
old dirt road or the autobahn. They shouldn't have to be refactored to race
downhill to show these results.

 

[Jon Skeet]

I used the editor to replace .append( with +
compiled and fixed the compiler issues. nothing more. nothing less. I have
projects due, i can't spend 10 hours preparing a bench mark.


I think you are grasping at straws here.

I'm not. I'm suggesting that posting only *one* of the pieces of code,
and not doing scientific tests in the first place, isn't a good way of
coming up with the kind of conclusions you're doing.

This is the same code, one running
stringbuilder, the other running string (formatted as described above).

But not as actually shown. I'd like to *see* the code. For one thing
then I could come up with some sample tests myself.

I'm
not seeing the results that was hyped about stringbuilder straight out of
the box. I wouldn't want to COOK the results (refactor or what ever the
latest terminology is) to get these results either.

I wasn't suggesting "cooking" anything. I was suggesting that before
you start saying that people are lying, you should have some good
grounds.

If it is as good as they
say, I shouldn't be jumping thru hoops to get the results. This isn't C++.

My analogy. If car A is faster than car B then car A should be faster on an
old dirt road or the autobahn. They shouldn't have to be refactored to race
downhill to show these results.

Your analogy is horribly flawed. Car A can be faster than car B, but
without timing them properly you won't know that.

 

[Jon Skeet]

Now hold on. I said they weren't scientific. I prefaced the results by that.

Yes, and then accused MS of lying, in quite strong terms. Doing that
without good evidence is a really bad idea, IMO.

I don't have a lot of time on my hands to go about making sure this will
stand up to the hounds of hell. I leave it up to MS for that.

What difference does it make what i do with them. The question I was after
was which way is faster to build strings. FYI, i take these queries and
write to the database. I don't see what is gained by that tidbit.

Did your tests include the time taken executing the queries? If so, it
could easily be the database which was being slower, or something
similar. In order to test string vs StringBuilder performance, you
should *only* include that.

This is a real world program by the way. Not an arbitrary couple lines of
code make up. If I can't see results there, and I should, then either I am
doing something wrong.

If the bulk of the time is spent doing the database query, then you
*wouldn't* see much difference.

In this case point it out so I can fix the code or MS
is on a marketing hype with this stringbuilder/.NET crap.

Or maybe you could accept that your tests were quite possibly entirely
bogus.

 

[Jon Skeet]

On the advice of a user, I've timed stringbuilder v string. Here are the
results.

Here are the numbers:
Total # queries 3747
Time in Milliseconds

StringBuilder: String
460.6624 320.4608
350.504 220.3168
240.3456 230.3312
----------- -----------
Ave. 350.504 Ave. 257.0362

Just looking at these results again, I'm *sure* the bulk of the time is
spent in executing the query. I certainly wouldn't expect it to take
that long just building up the strings.

Just for interest's sake, I ran a test where I built up 100000 strings,
each of the form (hello)(hello)(hello).... (x50) in a similar way to
most of your code. On my (admittedly faster) laptop, that took about
half a second. Here's the code:

using System;
using System.Text;

public class Test
{
public static void Main(string[] args)
{
DateTime start = DateTime.Now;

for (int i=0; i < 100000; i++)
{
StringBuilder sb = new StringBuilder();
for (int j=0; j < 50; j++)
{
sb.Append ('(').Append("hello").Append(')');
}
}

DateTime end = DateTime.Now;
Console.WriteLine (end-start);
}
}

Given those results, I really suspect that very little of your time is
spent actually building the strings.

One thing you *could* do to improve your code is change lines such as:

strTempString.Append("'").Append(splitField[RATE]).Append("',");
strTempString.Append("'").Append(splitField[SURCHARGE]).Append("',");
strTempString.Append("'").Append(splitField[MIN_SEC]).Append("',");
strTempString.Append("'").Append(splitField[INCR_SEC]).Append("',");

to:

strTempString.Append("'");
strTempString.Append(splitField[RATE]).Append("','");
strTempString.Append(splitField[SURCHARGE]).Append("','");
strTempString.Append(splitField[MIN_SEC]).Append("','");
strTempString.Append(splitField[INCR_SEC]).Append("'.");

etc which would reduce the number of appends by a third. I suspect you
won't see much difference though, for the reasons given above.

Out of interest, is there any reason for not using parameterised
queries and then passing in the parameters? I would expect that could
be faster at both the query-building end (with fewer variables
involved) and at the database end.

 

[Alvin Bruney]

ok (long okayyyyyyyyyyyyy).

I am attaching the complete routine with a short explanation.
This routine constructs queries and returns them on an arraylist. That is
all that was timed. Another threaded routine somewhere else takes care of
the database write. That routine has no knowledge of the construction
process and was not timed. For the test, the listbox had 3 items. (1249
queries repeated 3 times = 3747).
With this supplied code it would be trivial to reproduce the results. I
already said how I changed the appends to strings.

The times are in milliseconds by the way not seconds, which is a good time
IMO all things considered.

I purposely stay away from contrived examples because usually these examples
in no way mimic the demands of real world applications. Therefore, and for
the most part, the results can only be used as a rough guideline, certainly
not an absolute.

I didn't say that microsoft is lying. Do not misquote me. I alluded to the
fact that they may have overstated the facts (hype), reported results on the
extremely optimistic side (long history of doing so) etc etc. Can you find a
quote of me calling them liars?

On the other hand, I remain suspicious of MS' conclusions because lately
questions have arisen which seem to not support MS' claims. Case in point,
the garbage collector DOES NOT actually return recycled memory to the
operating system. MS now claims this is a bug somewhere deep in their
framework, but it bellies the point that their claim of garbage collection
was not correct in the first place.

I will continue to be vigilant. I still think .net is the greatest thing
since sliced bread. I am just more cautious about repeating stuff from the
microsoft marketing machine. If you think my results are completely bogus,
you are welcomed to post your times.














yada yada yada they lied

On the advice of a user, I've timed stringbuilder v string. Here are the
results.

Here are the numbers:
Total # queries 3747
Time in Milliseconds

StringBuilder: String
460.6624 320.4608
350.504 220.3168
240.3456 230.3312
----------- -----------
Ave. 350.504 Ave. 257.0362

Just looking at these results again, I'm *sure* the bulk of the time is
spent in executing the query. I certainly wouldn't expect it to take
that long just building up the strings.

Just for interest's sake, I ran a test where I built up 100000 strings,
each of the form (hello)(hello)(hello).... (x50) in a similar way to
most of your code. On my (admittedly faster) laptop, that took about
half a second. Here's the code:

using System;
using System.Text;

public class Test
{
public static void Main(string[] args)
{
DateTime start = DateTime.Now;

for (int i=0; i < 100000; i++)
{
StringBuilder sb = new StringBuilder();
for (int j=0; j < 50; j++)
{
sb.Append ('(').Append("hello").Append(')');
}
}

DateTime end = DateTime.Now;
Console.WriteLine (end-start);
}
}

Given those results, I really suspect that very little of your time is
spent actually building the strings.

One thing you *could* do to improve your code is change lines such as:

strTempString.Append("'").Append(splitField[RATE]).Append("',");
strTempString.Append("'").Append(splitField[SURCHARGE]).Append("',");
strTempString.Append("'").Append(splitField[MIN_SEC]).Append("',");
strTempString.Append("'").Append(splitField[INCR_SEC]).Append("',");

to:

strTempString.Append("'");
strTempString.Append(splitField[RATE]).Append("','");
strTempString.Append(splitField[SURCHARGE]).Append("','");
strTempString.Append(splitField[MIN_SEC]).Append("','");
strTempString.Append(splitField[INCR_SEC]).Append("'.");

etc which would reduce the number of appends by a third. I suspect you
won't see much difference though, for the reasons given above.

Out of interest, is there any reason for not using parameterised
queries and then passing in the parameters? I would expect that could
be faster at both the query-building end (with fewer variables
involved) and at the database end.

 

[Jon Skeet]

[Apologies for the code at the end not wrapping nicely. Reformatting all
of those lines to get them to under 72 columns would have taken forever.]

ok (long okayyyyyyyyyyyyy).

I am attaching the complete routine with a short explanation.
This routine constructs queries and returns them on an arraylist. That is
all that was timed. Another threaded routine somewhere else takes care of
the database write.

Hang on - you've got other threads running at the same time? If so, that
*completely* stuffs up the tests. You should run the tests on an otherwise
idle machine.

That routine has no knowledge of the construction
process and was not timed. For the test, the listbox had 3 items. (1249
queries repeated 3 times = 3747).
With this supplied code it would be trivial to reproduce the results. I
already said how I changed the appends to strings.

Except no, it's *not* trivial to reproduce the results, because it refers
to all kinds of variables for which we haven't the faintest idea of real
world values. However, I've done it, in a fashion - see later.

The times are in milliseconds by the way not seconds, which is a good time
IMO all things considered.

Except it's not, given the times I posted earlier, where 100,000 sets of appends
(with 50 iterations in each set, appending 3 times per iteration - in other
words, more work than it looks like is happening in your routine) took only
half a second.

I purposely stay away from contrived examples because usually these examples
in no way mimic the demands of real world applications. Therefore, and for
the most part, the results can only be used as a rough guideline, certainly
not an absolute.

So give real world examples of the variables used in your test, and we can
try to reproduce things, and improve the times.

I didn't say that microsoft is lying.

You said:

<quote>
My conclusion is: I think these microsoft guys have been feeding us a bunch
of bullshit with this stringbuilder efficiency crap.

Do not misquote me.

The above is an exact quote. If someone is "feeding us a bunch of bullshit"
that says to me that they're lying.

I alluded to the fact that they may have overstated the facts (hype)

Overstating is lying.

reported results on the extremely optimistic side (long history of doing so)
etc etc. Can you find a quote of me calling them liars?

I don't think there are many people who wouldn't view "feeding us a bunch of
bullshit" as lying.

On the other hand, I remain suspicious of MS' conclusions because lately
questions have arisen which seem to not support MS' claims. Case in point,
the garbage collector DOES NOT actually return recycled memory to the
operating system. MS now claims this is a bug somewhere deep in their
framework, but it bellies the point that their claim of garbage collection
was not correct in the first place.

I don't think there's anything sinister going on - they just have a bug,
which is understandable. That's not the same situation at all as with
StringBuilder, which has been shown time and time again to be faster in
situations like yours.

I will continue to be vigilant. I still think .net is the greatest thing
since sliced bread. I am just more cautious about repeating stuff from the
microsoft marketing machine. If you think my results are completely bogus,
you are welcomed to post your times.

Kinda difficult without any sample values, isn't it? However, I've done my best -
see below.

yada yada yada they lied

Pardon?


Anyway, I've taken your code and bodged it around so that it'll compile, giving it
some dummy values etc.

It creates a total of 300,000 queries in 100,000 iterations (3 per iteration, just
like yours). My results are:

10000 iterations took 00:00:01.6406250
10000 iterations took 00:00:01.6250000
10000 iterations took 00:00:01.6250000

In other words, it's doing nearly 100 times as much work as your tests, in less than 10
times the time. I don't believe my computer is 10 times faster than yours - I believe
that your tests were just broken, basically. Why don't you run the code below for
yourself, and see what happens. If you get much faster times than you did before, the
natural conclusion (IMO) is that most of the time spent in your previous tests wasn't
spent running the query-building code - and thus your tests are invalidated.



Here's the code:


using System;
using System.Text;
using System.Collections;

class Test
{
// Dummy variables
string[] items = {"COPYRATE1", "COPYRATE2", "COPYRATE3"};
bool ForceCreateChecked = true;

int NEW_RATE=0;
int NEW_RATE_DATE=1;
// 2 is used directly elsewhere
int RATE=7;
int SURCHARGE=3;
int MIN_SEC=4;
int INCR_SEC=5;
int BEGIN_DATE=6;

static void Main()
{
// Repeat multiple times to allow for JITting.
for (int j=0; j < 3; j++)
{
DateTime start = DateTime.Now;
for (int i=0; i < 100000; i++)
{
new Test().SuckTheJuiceOut("newrate,newdate,1STAMERICAN,0,0,0,2150-01-01 12:00:00"+
",0,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,0,1,2");
}
DateTime end = DateTime.Now;
Console.WriteLine ("100000 iterations took {0}", end-start);
}
}

private ArrayList SuckTheJuiceOut(string databaseBound)
{
/*
if(RateDeckConfirm.Items.Count == 0)
{
Page.Controls.Add(new LiteralControl("<script>alert('An error has occurred. "+
"The target rate deck is empty. Please refill this to continue.')</script>"));
return null;
}
*/

// Not sure what this line was doing, nor why splitField2 isn't a local variable
// splitField2 = GlobalRateMan.RegexNewline.Split(databaseBound);

string[] splitField2 = databaseBound.Split ('\n');

ArrayList sql= new ArrayList();

//we build the query here, take the records out the back end, it's more efficient that way
//stuff a dummyfield to help us remove the first hidden record
StringBuilder strTempString = new StringBuilder();

// Changed GlobalRateMan.INIT_VALUE to 0 here and elsewhere; if it's not 0, I don't know what it is.
for(int index = 0; index < splitField2.Length; index++)
{
// Split by commas using just string.Split for the moment
// splitField = GlobalRateMan.RegexComma.Split(splitField2[index]);

// Again, why isn't splitField a local variable?
string[] splitField = splitField2[index].Split(',');
// Removed the "." here so I can just have a single variable
if(ForceCreateChecked)
// I don't know what GlobalRateMan.sqlUpdateActiveRate1/2 are, so I've just got literals here
strTempString.Append("Update ds_active_rate SET (rate, surcharge, "+
"min_sec, increment_sec, "+
"begin_date, new_rate, new_rate_date, location_desc, "+
"category_id, country_code, customer_number, user_name)=(");
else
strTempString.Append("Update ds_active_rate1 SET (rate, surcharge, "+
"min_sec, increment_sec, "+
"begin_date, new_rate, new_rate_date, location_desc, "+
"category_id, country_code, customer_number, user_name)=(");
//check for valid record. replace some excel formatting
if(splitField.Length == 37)
{
// Changed the location of the declarations of these variables, and put
// the assignment of data alongside the declarations. I don't know what
// FormatDate does, so I'm just going to assume we've passed in the
// right format for the moment.
string newrate = splitField[NEW_RATE];
string newdate = splitField[NEW_RATE_DATE];

//both fields must be entered or neither in the case of updates to field like surcharges
if(newrate.Length > 0 && newdate.Length > 0)
{
//rate, surcharge, min_sec, increment_sec, begin_date, new_rate, new_rate_date, user_name
//temporarily insert rate_id so we can set the where clause. remove it later
strTempString.Append("'").Append(splitField[RATE]).Append("',");
strTempString.Append("'").Append(splitField[SURCHARGE]).Append("',");
strTempString.Append("'").Append(splitField[MIN_SEC]).Append("',");
strTempString.Append("'").Append(splitField[INCR_SEC]).Append("',");
// Again, ignore FormatDate
strTempString.Append("'").Append(splitField[BEGIN_DATE]).Append("',");
// Assume GlobalRateMan.EMPTY==""
strTempString.Append("'").Append(newrate.Replace("$","")).Append("',");
strTempString.Append("'").Append(newdate).Append("',");
strTempString.Append("'").Append(InsertUserName().TrimEnd()).Append("')");

// Removed the "." here so I can just have a single variable
if(ForceCreateChecked)
{
strTempString.Append("'").Append(splitField[8] + "',");
strTempString.Append(splitField[12]).Append(",");
strTempString.Append(splitField[14]).Append(",");
strTempString.Append("'").Append(splitField[2]).Append("',");
}
}

strTempString = strTempString.Replace("',)","')");
if(newrate.Length == 0 && newdate.Length == 0)
strTempString = strTempString.Replace("new_rate,new_rate_date,","");

for(int i = 0; i < items.Length; i++)
{
// Create a new StringBuilder each time - cleaner code
StringBuilder strTempString2 = new StringBuilder();
strTempString2.Append(" where rate_deck = '").Append(items.TrimEnd()).Append("'");
strTempString2.Append(" and location_id = ").Append(splitField[16]);
strTempString2.Append(" and world_id = ").Append(splitField[10]);
strTempString2.Append(" and rate_level = ").Append(splitField[34]);

//caught a live one
sql.Add(strTempString + strTempString2.ToString());

}

// I don't think any of this is needed
// strTempString.Length = GlobalRateMan.INITVALUE;
// newrate = newdate = "";
}
}

return sql;
}

string InsertUserName()
{
return "jon";
}
}

 

[Jon Skeet]

<snip>

Just for kicks, I had a go at using string concatenation instead. It
took longer, but not *that* much longer. I looked a bit closer, and
found that a significant amount of the time was being taken up by
String.Split in my implementation - if you're using a Regex to split
the data, that could account for quite a lot of your time difference.
(I discovered this by only splitting the string once at the start of
the program, and using the split form. Obviously you can't do this in
your code, but it does point to the bottleneck not being in the
appending.)

Further, I found that the replacement:
strTempString = strTempString.Replace("',)","')");
took a fair amount of the time.

Getting rid of the appending entirely by giving a list of 38 fields
instead of 37 only cut the time down by a factor of 2 or so - which
means that the difference in time spent actually appending is much more
significant than it first appeared - I reckon that the actual guts of
the appends are about 50% slower using string concatenation than using
StringBuilder. The time can further be reduced by giving the
StringBuilder constructors realistic capacities to start with - I gave
400 and 200, and it made a significant difference.

You might also be able to make things faster (and more readable) using
String.Format, or StringBuilder.AppendFormat. It would at least be
worth a try - but I really don't think the appends are what's taking
the time here.

As well as the split, the final append for each item is also going to
take a significant length of time whatever you do, by the way.

 

[Alvin Bruney]

I'll get back to this in a couple hours. You have raised some valid concerns
and I need to spend some time examining these concerns with the 'bullshit
detector' light turned on. (that was light humor by the way. it really was).
I think i'll also attach the contents of the splitstring as well as the
other variables. I'm willing to consider the fact that breaking into the
debugger could have thrown the times off.