File writes in a loop

[AdemusPrime]

I need to write data to a file in a loop. I get a "file already in
use" error on the second loop. The problem is, I don't know the name
of the file until I'm in the loop. How can I keep a file open for
writing in a loop?

while ((data = log.ReadLine()) != null)
{
string date = data.Substring(0, 10);
FileStream fileOut = new FileStream(dir + date + name,
FileMode.OpenOrCreate, FileAccess.Write);

fileOut.Write(Encoding.UTF8.GetBytes(data), 0, data.Length);
fileOut.Flush();
}

 

[Hilton]

while ((data = log.ReadLine()) != null)
{
string date = data.Substring(0, 10);
using (FileStream fileOut = new FileStream(dir + date + name,
FileMode.OpenOrCreate, FileAccess.Write))
{
byte[] bytes = Encoding.UTF8.GetBytes(data);
fileOut.Write(bytes, 0, bytes.Length);
}

 

[parez]

I need to write data to a file in a loop. I get a "file already in
use" error on the second loop. The problem is, I don't know the name
of the file until I'm in the loop. How can I keep a file open for
writing in a loop?

while ((data = log.ReadLine()) != null)
{
string date = data.Substring(0, 10);
FileStream fileOut = new FileStream(dir + date + name,
FileMode.OpenOrCreate, FileAccess.Write);

fileOut.Write(Encoding.UTF8.GetBytes(data), 0, data.Length);
fileOut.Flush();

}

are you going have multiple files open in the loop?if yes and you
dont want to close them then you could
use a a hashtable and use dir + date + name as the key and the
filestream as the object.

 

[zacks]

I need to write data to a file in a loop. I get a "file already in
use" error on the second loop. The problem is, I don't know the name
of the file until I'm in the loop. How can I keep a file open for
writing in a loop?

while ((data = log.ReadLine()) != null)
{
string date = data.Substring(0, 10);
FileStream fileOut = new FileStream(dir + date + name,
FileMode.OpenOrCreate, FileAccess.Write);

fileOut.Write(Encoding.UTF8.GetBytes(data), 0, data.Length);
fileOut.Flush();



}

Your question is cryptic, so I am just guessing here but maybe you
need a:

fileOut.Close();

after the flush?

And maybe even a:

fileOut.Dispose();

 

[AdemusPrime]

Thank you for the reply.

This code will close, dispose and reopen the file on each loop.
Therefore, after each loop there is only one record in the output file
which is a last write.

 

[AdemusPrime]

In order to create the hashtable I would have to know the name of all
the files. I won't know the name until after I read the record in the
logfile.

 

[AdemusPrime]

Not sure which part is cryptic. I need to open the file for writing
and keep it open in a loop.

I can close and/or dispose the object but I want to keep the file open
for writing instead of making roundtrip opens/closes each time. Plus,
i would have to advance the cursor to the end of the file each time I
open it, which could be big.

 

[Joe Cool]

Not sure which part is cryptic. I need to open the file for writing
and keep it open in a loop.

I can close and/or dispose the object but I want to keep the file open
for writing instead of making roundtrip opens/closes each time. Plus,
i would have to advance the cursor to the end of the file each time I
open it, which could be big.

You need to close the file after flushing it. If you need to add to
the file in a subsequent iteration, I would first check to see if the
file exists yet or not with the File.Exists method.

If if does not exist, open the file for create or open. If it does,
open it for append.

 

[Cor Ligthert[MVP]]

Joe,

In my idea do you give the best solution

(This in fact to the OP to try this first).

Cor

 

[AdemusPrime]

Here the un-vague version of the question.

I have a 112 GB log file. I need to parse out this very large file to
smaller files. We've decided for business reasons to parse it by
date. I won't know the date, which is the name of the output file,
until after I read the record in the log. The records are supposed to
be in date order but this is NOT guaranteed. When I read a record I
then open the output file for writing. If I already have the output
file open I don't want to close it and then re-open it. The preceived
performance hit of closing and opening the same file thousands of
times doesn't make sense.

My original design was to do the following...

Open Log
Loop
Read Record
Open/Create Ouput
Write Record
Next
Close Log

All this works fine until it does the Open/Create Output on the second
interation. I was hoping to learn of a way to check if the file is
already open and wrap the Open/Create in a conditional statement.

The dictionary suggestions seem like a good solution for mulitple open
files. I will give this a try.

Thanks for the help.

 

[parez]

Here the un-vague version of the question.

I have a 112 GB log file. I need to parse out this very large file to
smaller files. We've decided for business reasons to parse it by
date. I won't know the date, which is the name of the output file,
until after I read the record in the log. The records are supposed to
be in date order but this is NOT guaranteed. When I read a record I
then open the output file for writing. If I already have the output
file open I don't want to close it and then re-open it. The preceived
performance hit of closing and opening the same file thousands of
times doesn't make sense.

My original design was to do the following...

Open Log
Loop
Read Record
Open/Create Ouput
Write Record
Next
Close Log

All this works fine until it does the Open/Create Output on the second
interation. I was hoping to learn of a way to check if the file is
already open and wrap the Open/Create in a conditional statement.

The dictionary suggestions seem like a good solution for mulitple open
files. I will give this a try.

Thanks for the help.

you mite want to look at this..

http://weblogs.asp.net/fbouma/archive/2006/05/02/444779.aspx

 

[AdemusPrime]

This is what I ended up with.

using (StreamReader log = File.OpenText(myArgs["i"]))
{
FileStream outFile = null;
string old = string.Empty;
while ((data = log.ReadLine()) != null)
{
string date = data.Substring(0, 10);
if (old != date)
{
if (outFile != null)
{
outFile.Close();
}
outFile = new FileStream(dir + date +
name, FileMode.Append, FileAccess.Write);
old = date;
}
outFile.Write(Encoding.UTF8.GetBytes(data), 0,
data.Length);
outFile.Write(Encoding.UTF8.GetBytes("\r\n"),
0, 2);
outFile.Flush();
}
outFile.Close();
}