Threads accessing a dictionary

[Peter]

Hi, I have a Dictionary<key, value> which is accessed by three threads.

One thread puts my value objects in the dictionary (occasionally), and
also updates the contents of existing value objects - by finding the
appropriate object via a key and updating the value (this updating
happens several times a second).

Another thread removes "dead" value objects from the dictionary (very
occasionally)

The third thread needs to loop through all the value objects in the
dictionary performing calculations based on their contents. This thread
needs to run maybe once every 10 seconds, do the calculations, and then
loop/wait again. I am not sure exactly how long these calculations will
take - but it will depend very much on the size of the dictionary
(maybe 1 or 2 seconds).

I foresee a problem with these threads accessing the same dictionary.
Especially the need for the third thread to loop over all the values
while it is possible these values could be being updated, or new values
could be being added or old values deleted.

Do I need to "lock" the entire dictionary while the third thread is
looping over the values? Is there a better way to avoid threading
problems?


Thanks for any comments,
Peter


--

 

[Jon Skeet [C# MVP]]

Hi, I have a Dictionary<key, value> which is accessed by three threads.

One thread puts my value objects in the dictionary (occasionally), and
also updates the contents of existing value objects - by finding the
appropriate object via a key and updating the value (this updating
happens several times a second).

Another thread removes "dead" value objects from the dictionary (very
occasionally)

The third thread needs to loop through all the value objects in the
dictionary performing calculations based on their contents. This thread
needs to run maybe once every 10 seconds, do the calculations, and then
loop/wait again. I am not sure exactly how long these calculations will
take - but it will depend very much on the size of the dictionary
(maybe 1 or 2 seconds).

I foresee a problem with these threads accessing the same dictionary.
Especially the need for the third thread to loop over all the values
while it is possible these values could be being updated, or new values
could be being added or old values deleted.

Do I need to "lock" the entire dictionary while the third thread is
looping over the values? Is there a better way to avoid threading
problems?

Yes, you should make sure you only access the dictionary from one
thread at a time. However, you could consider fetching all the values
from the dictionary in the third thread *then* doing the calculations,
to avoid locking the dictionary for a long time.

 

[Peter Morris]

In short, yes you need to lock.

This is what I would do

01: Don't give access directly to Dictionary, write a class that your
threads interact with.
02: Declare a private object in your class like so

private object SyncRoot = new object();

03: When you want to update a value do this

lock (SyncRoot)
MyPrivateDictionary[key] = value;


04: When you want to process items do this (using the correct types for the
KeyValuePair)

public IList<KeyValuePair<string, object>> GetValues()
{
List<KeyValuePair<string, object>> result = new
List<KeyValuePair<string, object>>();
lock (SyncRoot)
foreach (KeyValuePair<string, object> kvp in MyPrivateDictionary)
result.Add( new KeyValuePair<kvp.Key, kvp.Value>() );
return result;
}

Now your worker thread can get a snapshot of the keys as they were at the
time it needed to process them. So other threads can continue to add /
clean the list without blocking your worker. Like so

(in your worker thread class)
foreach (KeyValuePair<string, object> kvp in JobList.GetValues())
DoWork(kvp.Key, kvp.Value);



05: To kill dead items you would again use GetValues() and then go through
the list determining if the item is dead or not

(in your cleanup thread class)
List<string> deadKeys = new List<string>();
foreach (KeyValuePair<string, object> kvp in JobList.GetValues())
if (IsDead(kvp.Key))
deadKeys.Add(kvp.Key);
MyJobList.CleanUp(deadKeys);

and then finally tell your class to clean them all up in one go, to avoid
multiple lock/unlock.


(In your job list class)
public void CleanUp(List<string> keys)
{
lock (SyncRoot)
foreach (string currentKey in keys)
MyPrivateDictionary.Remove(currentKey);
}

This assumes that IsDead takes a while. If it is quick then you may as well
just do this

(in your cleanup thread class)
MyJobList.CleanUp();

(in your job list class)
public void CleanUp()
{
lock (SyncRoot)
foreach (KeyValuePair<string, object> kvp in MyPrivateDictionary)
if (IsDead(kvp.Key))
MyPrivateDictionary.Remove(kvp.Key);
}



Pete

 

[Marc Gravell]

Do I need to "lock" the entire dictionary while the third thread is

looping over the values?

Just to clarify: you would need to lock from all 3 threads, not just
this one; locking only works if all the code plays by the same rules.

You might be able to do some things with reader/writer locks, but to
be honest I'd keep it simple and start with a simple exclusive lock
(aka Monitor, i.e. "lock(foo) {...}"). As Jon stated - if your
calculations are relatively slow, you can reduce contention by cloning
the data first then releasing the lock and doing the calculations on
the cloned data.

Of course; if your first thread is updating *properties* of the values
(rather than swapping the value entirely), then life gets even more
complex; in this case even a clone can't guarantee much, and you'd
need (to keep things simple) a lock covering the entire calculation
step.

Marc

 

[Peter]

Hi, I have a Dictionary<key, value> which is accessed by three
threads.

Thanks for any comments,
Peter

Thanks heaps for all your comments. After contemplating a little I can
see I actually have more problems than I first thought. The first
(update) thread does indeed update properties on the value objects -
and further to this, the third (calculation) thread will also actually
be updating other properties.

So in the light of your comments I'll need to reconsider my design a
little...

Thanks,
Peter

--

 

[Marc Gravell]

The first

(update) thread does indeed update properties on the value objects -
and further to this, the third (calculation) thread will also actually
be updating other properties.

So in the light of your comments I'll need to reconsider my design a
little...

Note that this is still possible - just that you need to think about
two threads tweaking the same object at the same time and/or getting
intermediate states - i.e. with properties "Foo" and "Bar", with one
thread updating both, and the first thread reading the new "Foo" and
the old "Bar".

In this scenario you either need object-level locking, or (to keep it
simple) just treat the entire dictionary/collection as a unit, and
only let one thread in at a time. Given the different frequencies of
call, I suspect that this would work reasonably... worth a try, at
least...

Marc