Writing flat file to network is slow

[Barry Flynn]

Hi

I am working with a VB 2005 program which has been converted from VB6.
It writes data out to a flat file, with code like the following line

WriteLine(riFileNo, "Hist", lsAssetID, lsRecordType, lsXNbr, lsFiscYr,
"Beg", CStr(H.BegBalAccDepn), CStr(H.BegBalCost), CStr(H.BegBalCostReval),
CStr(H.BegBalDepCost), CStr(H.BegBalDepnReval))

The program is running from within a Virtual PC
If it is writing to a file on drive c:, the job takes about 30 seconds.
If it is writing to a file on a network drive, it takes about 20 minutes.

If it writes to drive C:, then I copy the file to the network drive, the
copy operation takes just a couple of seconds.
There is very little traffic on the network.

What are possible reasons for it running so slowly when my code writes to
the network drive?

Thanks

Barry

 

[Andrew Morton]

I am working with a VB 2005 program which has been converted from VB6.
It writes data out to a flat file, with code like the following line

WriteLine(riFileNo, "Hist", lsAssetID, lsRecordType, lsXNbr, lsFiscYr,
"Beg", CStr(H.BegBalAccDepn), CStr(H.BegBalCost),
CStr(H.BegBalCostReval), CStr(H.BegBalDepCost),
CStr(H.BegBalDepnReval))
The program is running from within a Virtual PC
If it is writing to a file on drive c:, the job takes about 30
seconds. If it is writing to a file on a network drive, it takes
about 20 minutes.
If it writes to drive C:, then I copy the file to the network drive,
the copy operation takes just a couple of seconds.
There is very little traffic on the network.

What are possible reasons for it running so slowly when my code
writes to the network drive?

For a start, you could try running it on a non-virtual PC to see if that's
creating a bottleneck.

Also, is the file so large that you can't put all the data in a
stringbuilder and then write that in one go to the [network] drive?

Andrew

 

[Barry Flynn]

I am working with a VB 2005 program which has been converted from VB6.
It writes data out to a flat file, with code like the following line

WriteLine(riFileNo, "Hist", lsAssetID, lsRecordType, lsXNbr, lsFiscYr,
"Beg", CStr(H.BegBalAccDepn), CStr(H.BegBalCost),
CStr(H.BegBalCostReval), CStr(H.BegBalDepCost),
CStr(H.BegBalDepnReval))
The program is running from within a Virtual PC
If it is writing to a file on drive c:, the job takes about 30
seconds. If it is writing to a file on a network drive, it takes
about 20 minutes.
If it writes to drive C:, then I copy the file to the network drive,
the copy operation takes just a couple of seconds.
There is very little traffic on the network.

What are possible reasons for it running so slowly when my code
writes to the network drive?

For a start, you could try running it on a non-virtual PC to see if that's
creating a bottleneck.

Also, is the file so large that you can't put all the data in a
stringbuilder and then write that in one go to the [network] drive?

Andrew

Andrew - thanks for the suggesrions.

For a start, you could try running it on a non-virtual PC to see if that's
creating a bottleneck.

Unfortunately, that's a little difficult.
It runs in the Microsoft Dynamics SL environment, and the "real PC" has a
conflicting (VB6) version of that environment.
However, I thought that (a) the fact that it runs in a few seconds when
writng to drive c:, coupled with (b) the fact that I can copy the file to
the network in a few seconds, sort-of suggested that the virtual pc wasn't
the guilty party.
I recognze that a virtual pc is likely to be slower than a "real one", but
this job takes about 40 times as long when writing to the network.

Also, is the file so large that you can't put all the data in a
stringbuilder and then write that in one go to the [network] drive?

Yes I agree that it would be better to use string builders, and probably
streams.
If I was writimg the program from scratch, I probably would do that.
This prog was converted from a VB6 version, and contains a lot of these
WriteLine statements as it writes different data from several different
tables.
So there's a bit of work involved in changing it.

I do intend to do some testing to see if different ways of writing the data
make a difference.
However, I thought that the fact that it runs in a few seconds when writing
to a local drive indicated that use of stringbuilders etc. might not make a
huge difference.

Thanks

Barry

 

[Andrew Morton]

I do intend to do some testing to see if different ways of writing
the data make a difference.
However, I thought that the fact that it runs in a few seconds when
writing to a local drive indicated that use of stringbuilders etc.
might not make a huge difference.

Ah, but then you could write the whole file in one go so buffers might have
more efficient use made of them [sorry about that grammar]. The local file
system will use 32KB buffers (I think) whereas sending it out from a virtual
PC through real hardware and over a network, well, I can only make guesses.

Andrew