Threading in Order?

[Rico]

I have a background task (thread) of logging messages, so that I do
not take execution time away from the main web application.

Unfortunately, the messages are not stored in order of occurrence.

How can I set up a queue, pool, single pipe (whatever is necessary),
so that threads are executed in the order, or more specifically, the
messages are stored in the order, in which they are started?

Thanks.

 

[Jeroen Mostert]

I have a background task (thread) of logging messages, so that I do
not take execution time away from the main web application.

Unfortunately, the messages are not stored in order of occurrence.

How can I set up a queue, pool, single pipe (whatever is necessary),
so that threads are executed in the order, or more specifically, the
messages are stored in the order, in which they are started?

Not an answer to your question, but possibly helpful nonetheless: use
log4net (http://logging.apache.org/log4net/index.html). It's thread-safe and
probably a whole lot more powerful than any solution you can cook up yourself.

 

[Peter Duniho]

I have a background task (thread) of logging messages, so that I do
not take execution time away from the main web application.

Unfortunately, the messages are not stored in order of occurrence.

How can I set up a queue, pool, single pipe (whatever is necessary),
so that threads are executed in the order, or more specifically, the
messages are stored in the order, in which they are started?

That really depends on what your requirements are. The primary answer
is: "you can't". That is, you don't have enough control over the
ordering of thread execution to accomplish exactly what you're asking
about in a feasible way. You'd need to address the issue at a different
level (i.e. not manipulating the threads themeselves).

I agree with Jeroen that the first choice would be to simply use some
existing logging framework that already handles your needs.

If you have to implement it yourself, and it's important for events to
be processed in order of occurrence, you'll have to timestamp the events
being logged, and order them yourself.

This can be more complicated than one might think, because Windows isn't
a real-time OS and so it's theoretically impossible to know for sure
when you see a given event with a given timestamp, that there will not
later be an event with an earlier timestamp.

You can get reasonably close by setting some delay for each logged event
(e.g. ten seconds), maintaining the logged events in-memory in sorted
order according to timestamp, and writing the events out as their delay
expires (i.e. an event is logged only when it's been in the in-memory
event log for the delay period, ten seconds in this example).

But a) this introduces a lag in the logging, which may or may not be
desirable, and b) no matter what delay you choose, there is the
possibility of some thread or threads getting held up long enough that
you still wind up with some events logged out of order (you can increase
the delay to a large enough value that the odds of this are practically
nil, but of course as the delay period gets longer, the lag in the
logging becomes more of an issue).

If you can keep all the events in-memory until some point in time when
you know for sure no more events will occur, then you can simply always
use an insertion sort, or even sort them after the fact, and just emit
the logged events at that appropriate time when you know for sure no
more events will occur.

I admit, I haven't actually used log4net, or its Java equivalent, so I
don't know exactly how it deals with the issue. But, if you want to log
directly to a file, I think the best solution is simply to include the
timestamp in the record in-file, and then sort it as necessary after the
fact (i.e. once logging is done).

Pete

 

[Adam Benson]

Unfortunately, the messages are not stored in order of occurrence.
What makes you say that? The time stamps are out of order? Or something
else?

If you want to ensure that if thread A calls the logging method just before
thread B then thread A's message goes in the log first do something like
this (pseudo code) :

AutoResetEvent evtLogging = new AutoResetEvent(false);
object lockQ = new object();
Queue<string> logmsgs = new Queue<string>();

void LogAMsg(string s)
{
lock (lockQ)
{
logmsgs.Enqueue(s);
}
evtLogging.Set();
}


void MyLoggingThread()
{
for (;
{
evtLogging.WaitOne();

string msg;
lock (lockQ)
{
while (lockQ.Count > 0)
{
msg = lockQ.Dequeue();
// Write msg to your log
}
}
}
}

HTH,

Adam.

 

[Peter Duniho]

What makes you say that? The time stamps are out of order? Or something
else?

If you want to ensure that if thread A calls the logging method just before
thread B then thread A's message goes in the log first do something like
this (pseudo code) : [code snipped]

Nope. You simply cannot do that. Each thread can get interrupted by
the OS at arbitrary points. Just because thread A calls the LogAMsg()
method before thread B does, that doesn't mean that thread A will
acquire the lock before thread B; it could happen to get pre-empted just
before acquiring the lock, with thread B running immediately after and
acquiring the lock before thread A.

The best one can do is not generate a timestamp until the lock has
actually been acquired, but then the timestamp winds up being not the
time of the event being logged, but the time that the event _is_ logged.
A subtle, but VERY important difference.

If you want the timestamp to reflect the time that the event actually
_happened_, then it is not possible to guarantee with 100% certainty
that the event is added to the event queue in the same order in which
the event happened. You can reorder the events in the log once logging
is done and you know no more events will be added, but while the logging
is happening, it's just not possible. The thread scheduler is
controlling things at a lower level than the managed application can affect.

Pete

 

[Adam Benson]

Understood. But I would guess the OP was after a best fit. He might try it
and find it suits his needs.

- Adam.
=======