close and open file again or keep open -- for log file?

[Ryan Liu]

Hi,

Both way works, I'd just ask some experts which way is better?

My application creates a log file daily.

Now each time when I write a log, I will open the file and append to the
end. Ocz, if the file is not exist(e.g. another day), it will creates the
file first.

Now I am thinking keep the file strem open, I think this should can gain
better performance. Ocz I will close the file strem when applicatoin exist
and/or in de-constructor, and when switch to another log file(at mid-night,
another day begins).

Either way, I will open the log file in ReadWrite share mode.

Which way is common practise?

Thanks a lot!
Ryan

 

[Laurent Bugnion]

Hi,

Hi,

Both way works, I'd just ask some experts which way is better?

My application creates a log file daily.

Now each time when I write a log, I will open the file and append to the
end. Ocz, if the file is not exist(e.g. another day), it will creates the
file first.

Now I am thinking keep the file strem open, I think this should can gain
better performance. Ocz I will close the file strem when applicatoin exist
and/or in de-constructor, and when switch to another log file(at mid-night,
another day begins).

Either way, I will open the log file in ReadWrite share mode.

Which way is common practise?

Thanks a lot!
Ryan

I think it's not a good idea. If your application crashes, the stream
won't be closed. Keeping the file open complicates access through
another application (typically, you need to open the log file in a
viewer or editor when you need to debug the application on a production
machine, and typically you have only notepad available in this
circumstance).

Maybe you should consider using log4net, which has excellent
performances AFAICS and which handles opening and closing the log files
itself. Additionally, it's fairly easy to add listeners to log to a
remote machine, for example.

HTH,
Laurent

 

[Peter Duniho]

I think it's not a good idea. If your application crashes, the stream
won't be closed.

Clarification: the stream may not be *flushed*. It definitely will be
closed, if the application crashes. (Of course, one hopes that the
application crashing is a rare circumstance). Data may be lost, but any
data that has actually been written out to the file will be safely stored
there, even if the application crashes.

Keeping the file open complicates access through another application
(typically, you need to open the log file in a viewer or editor when you
need to debug the application on a production machine, and typically you
have only notepad available in this circumstance).

Very true. If the desire is to access the log file while the application is
running, via some independent program, keeping the file open for write
access may prevent that access.

Of course, similarly one would want to make sure to use some means to view
the logfile that doesn't also keep it open (notepad, as you suggest, is
appropriate for this). Otherwise, the application itself may find itself
unable to write to the file.

This could happen in any case, since the file needs to be kept open at least
long enough to read it to make a new copy (in memory as notepad does, or to
a different file). So it may be worthwhile to include logic in the logging
code to either queue and retry logging attempts, or start a new file if the
current one can't be opened.

Another thing to keep in mind is how often the log file will be written to.
If the application is likely to be generating log entries on a near-constant
basis, keeping the file open may be important for performance reasons. It's
not VERY expensive to open and close a file, but it's expensive enough if
you're doing it many times a second.

Of course, one could take the Windows event manager approach, and use an
independent process to handle logging as well as viewing the logs. That
would, of course, greatly complicate the implementation and it may be that
something like "log4net" (as mentioned already) would be more appropriate in
that situation (I wouldn't know, since this is the first I've heard of it
).

Pete

 

[Laurent Bugnion]

Hi,

Clarification: the stream may not be *flushed*. It definitely will be
closed, if the application crashes. (Of course, one hopes that the
application crashing is a rare circumstance). Data may be lost, but any
data that has actually been written out to the file will be safely stored
there, even if the application crashes.

Yes, sorry.

Of course, similarly one would want to make sure to use some means to view
the logfile that doesn't also keep it open (notepad, as you suggest, is
appropriate for this). Otherwise, the application itself may find itself
unable to write to the file.

I typically use a web browser to view log files. It makes it easy to
refresh the view when the application write more entries to the file,
and it doesn't block the access.

Of course, one could take the Windows event manager approach, and use an
independent process to handle logging as well as viewing the logs. That
would, of course, greatly complicate the implementation and it may be that
something like "log4net" (as mentioned already) would be more appropriate in
that situation (I wouldn't know, since this is the first I've heard of it
).

Pete

log4net is a quite well known logging component. It's a port from log4j
(for Java) to .NET. It's free and easy to customize, and generally more
friendly than the built-in Trace and Debug classes. That said, it's 3rd
party, so one may want to use the built-in classes anyway...
http://logging.apache.org/log4net/

HTH,
Laurent

 

[Ryan Liu]

Thanks , Laurent and Peter!

So that's why I want to close the stream in the finally of main() and also
in static de-constructor, and hope that will help when the application
crashes. (So when it crashes, even finally might not be executed?)

And for the same reason I open file stream in shared read/write mode, so
even the stream is not properly closed, it can still be read and write next
time.

Will those two tricks help?

And is that really costly to open a file and append at the end? I am afraid
it will read whole file (could be big) then write. If it can locate and
right "jump" to the end of file without reading file, it should not that
costly. I just don't know the underneath implementation.

Thanks a lot!

 

[Peter Duniho]

Thanks , Laurent and Peter!

So that's why I want to close the stream in the finally of main() and also
in static de-constructor, and hope that will help when the application
crashes. (So when it crashes, even finally might not be executed?)

It depends on why it crashes. If the error is in your own code, or in .NET
code, you have an excellent chance of having the opportunity to clean up in
the finally clause of your main() function. If the entire computer faults
(video driver problem, for example) or the process is terminated abnormally
by external means ("End Process", for example) then no.

However, one presumes that on a system where you want a continuously running
process for which a log file is very important, you are minimizing your
exposure to buggy third-party code. Hopefully the scenarios in which your
application doesn't get a chance to clean up are even more rare than the
scenarios in which your application itself might crash, and presumably those
latter scenarios are themselves exceedingly rare.

If not, your time would be better spent figuring out how to ensure that they
*are* exceedinly rare, rather than worrying about the log file.

And for the same reason I open file stream in shared read/write mode, so
even the stream is not properly closed, it can still be read and write
next
time.

I have seen Windows fail to clean up an opened file correctly. However,
never after a simple application crash, and infrequently in any case (I've
seen it happen maybe three times since I first started using Windows NT,
almost 15 years ago). How an application opens the file doesn't have
anything to do with access to the file after that application crashes.
Normally, Windows will clean things up for you and the file will be returned
to a normal, closed state ready for access by other processes.

So, no...using a shared file access mode has no bearing on access to the
file after the application crashes.

And is that really costly to open a file and append at the end? I am
afraid
it will read whole file (could be big) then write. If it can locate and
right "jump" to the end of file without reading file, it should not that
costly. I just don't know the underneath implementation.

Seeking to a particular position in the file is relatively fast. The size
of the file shouldn't affect performance much, if at all. It's just that
those operations (open, seek, and close) aren't free and relative to
whatever work you could be doing on the computer they may even be fairly
expensive. If you do them every few seconds or even less frequently, I
would guess that they would not be a noticeable problem. But doing them
several times a second or more often could easily take i/o and CPU time away
from more important work.

If it's something that really matters in your situation, the best thing to
do is to measure it. Measure your performance (throughput, whatever is
relevant in your situation) without logging, and measure logging with and
without keeping the file open. This will give you some real data as to what
the best compromise will be for you.

Keeping in mind, of course, that over the years as hardware performance
changes -- CPUs get faster or more numerous, disk i/o gets faster, etc. --
the exact ratios you've measured will change, possibly affecting the
analysis as well. This measurement is most useful only with the computer
configuration on which you've done the measurement, but it should be
instructive regardless.

If it's not important enough to do some basic measurements, it's probably
not important enough to worry much at all about the performance of different
implementations.

Pete