Volatile & Interlocked

[Pierre]

Using the "volatile" keyword, creates a problem if I intend to use any of
the interlocked APIs. The compiler generates an error if I use the following
line, for example:

Interlocked.Increment(ref count);

The error says that a volatile field cannot be used as ref or out, but if I
don't use the volatile field, the value may be cached away in some method
that is just reading the field.

Consider, for example, an integer field being used by three threads. Two
threads intend to increment the value of the field while the third one just
wishes to check the value. Consider, for example, the initial value of the
field is 5.

If both the writer threads increment the value using the interlocked API,
the final value of the field is guaranteed to be 7. Without the use of the
interlocked API, the final value could be 6. Therefore, using the
interlocked API is a must in this case.

Without the field being marked as volatile, the reader thread may always see
the value as 5.

As can be seen, we need both the mechanisms under some cases.

What might be the solution for such situations?

 

[John Puopolo]

Hi...

Are you marking the field as "volatile"?

Just so we are clear, when you make a variable as volatile, it means that
there is a chance that the item could be changed asynchronously. In
general, if you control the threads, etc., you can simply use the "lock"
keyword:

lock(obj) {
// do work
}

If you need inter-process control, consider using a Mutex.
If you need multple readers with one writer, all banging away on the same
variable, consider the ReaderWriterLock class provided in the .NET framework
class library.

Hope this helps,
John Puopolo


If you have reader/writer threads are are looking to have multiple readers
with one writer, you can use the

 

[Brian Gideon]

Pierre,

The lock keyword (Monitor.Enter and Monitor.Exit) has volatile
read/write semantics. You could use that instead of the Interlocked
class. It would be significantly easier to understand, but would be
slower to execute. The slower execution will not matter in most cases.

If you still wish to use the Interlocked class you'll be foreced to
remove the volatile modifier. I *think* the Interlocked class
implicitly creates a memory barrier. Can someone confirm? If so then
you can safely read the value using the Interlocked.CompareExchange
method.

int safeRead = Interlocked.CompareExchange(ref count, 0, 0);

Topics involving volatile read/writes, memory barriers, and the memory
model are very confusing. It's easy to make a mistake.

Brian

 

[Willy Denoyette [MVP]]

Pierre,

The lock keyword (Monitor.Enter and Monitor.Exit) has volatile
read/write semantics. You could use that instead of the Interlocked
class. It would be significantly easier to understand, but would be
slower to execute. The slower execution will not matter in most cases.

If you still wish to use the Interlocked class you'll be foreced to
remove the volatile modifier. I *think* the Interlocked class
implicitly creates a memory barrier. Can someone confirm? If so then
you can safely read the value using the Interlocked.CompareExchange
method.

int safeRead = Interlocked.CompareExchange(ref count, 0, 0);

Topics involving volatile read/writes, memory barriers, and the memory
model are very confusing. It's easy to make a mistake.

Brian

You are right, the Interlocked functions do create memory barriers with
aquire & release semantics, so there is no need for the volatile modifier.

Willy.

 

[Willy Denoyette [MVP]]

Using the "volatile" keyword, creates a problem if I intend to use any of
the interlocked APIs. The compiler generates an error if I use the
following line, for example:

Interlocked.Increment(ref count);

The error says that a volatile field cannot be used as ref or out, but if
I don't use the volatile field, the value may be cached away in some
method that is just reading the field.

Consider, for example, an integer field being used by three threads. Two
threads intend to increment the value of the field while the third one
just wishes to check the value. Consider, for example, the initial value
of the field is 5.

If both the writer threads increment the value using the interlocked API,
the final value of the field is guaranteed to be 7. Without the use of
the interlocked API, the final value could be 6. Therefore, using the
interlocked API is a must in this case.

Without the field being marked as volatile, the reader thread may always
see the value as 5.

As can be seen, we need both the mechanisms under some cases.

What might be the solution for such situations?

There's no need for the volatile modifier, the Win32 InterlockedXXX
functions called by Interlocked.XXXX(..) do insert memory barriers with
aquire/release semantics on X86.

Willy.

 

[Jon Skeet [C# MVP]]

There's no need for the volatile modifier, the Win32 InterlockedXXX
functions called by Interlocked.XXXX(..) do insert memory barriers with
aquire/release semantics on X86.

Although you can then never just use the field directly - you need to
use the InterlockedXXX methods for *every* access to it. Otherwise,
there could be a memory barrier in the incrementing thread, but no
memory barrier in the reading thread.

 

[Willy Denoyette [MVP]]

Although you can then never just use the field directly - you need to
use the InterlockedXXX methods for *every* access to it. Otherwise,
there could be a memory barrier in the incrementing thread, but no
memory barrier in the reading thread.

Sorry, the Win32 InterlockedXXX should read the CLR InterlockedXXX...

The InterlockedXXX operations (as implemented by the CLR) are atomic, there
is no problem for the reading thread.
The operations use the atomic asm instructions xadd, xchg, cmpxchg on single
CPU with lock prefix on SMP or multicores (X86)
Spinlocks are used to protect simultanious Interlocked operations on long's.
Note that Interlocked operations only serve correctly aligned pointer sized
types (32 & 64 bits).

Willy.

 

[Alvin Bruney [Microsoft MVP]]

that info coming off the top of your head? or you are reading it from
somewhere? pray tell.

--
Regards
Alvin Bruney
[Shameless Author Plug]
The Microsoft Office Web Components Black Book with .NET
available at www.lulu.com/owc

 

[Jon Skeet [C# MVP]]

Sorry, the Win32 InterlockedXXX should read the CLR InterlockedXXX...

The InterlockedXXX operations (as implemented by the CLR) are atomic, there
is no problem for the reading thread.

You're missing something. You need a memory barrier in both the reading
thread *and* the thread which is changing the value.

Suppose I have the code:

int x = nonVolatileVariable;
....
int y = nonVolatileVariable;

where "..." consists of operations that the JIT knows don't change the
value of nonVolatileVariable. There's nothing to stop it from caching
the value, and making y the same as the initial value of x, even if
another thread has used InterlockedXXX to change the value during
"...".

InterlockedXXX makes sure that a value is read from main memory,
changed, and rewritten to main memory, all atomically. It *doesn't*
(and can't) make sure that all others threads *use* main memory to see
the value it's written.

 

[Willy Denoyette [MVP]]

You're missing something. You need a memory barrier in both the reading
thread *and* the thread which is changing the value.

Suppose I have the code:

int x = nonVolatileVariable;
...
int y = nonVolatileVariable;

where "..." consists of operations that the JIT knows don't change the
value of nonVolatileVariable. There's nothing to stop it from caching
the value, and making y the same as the initial value of x, even if
another thread has used InterlockedXXX to change the value during
"...".

*** Sure, but here you assume that the JIT can cache the non-volatile member
field value in a CPU register, and as far as I can see (but I could be
wrong) this is not done in V1.x.
From what I see in the v1.1 JIT/codegen source code, registers are
exclusively used to cache locals not member fields. I will try to spend some
time to investigate the exact semantics of volatile in the CLR.

InterlockedXXX makes sure that a value is read from main memory,
changed, and rewritten to main memory, all atomically. It *doesn't*
(and can't) make sure that all others threads *use* main memory to see
the value it's written.

*** Note that v2.0 doesn't generate an error when using a volatile field
ref. in Interlocked.XXXX, it just generates following:

warning CS0420: xxxxxxx: a reference to a volatile field will not be treated
as volatile

it's clear the "reference" will not be treated as volatile but the field is
treated as volatile.

Willy.

 

[Brian Gideon]

*** Sure, but here you assume that the JIT can cache the non-volatile
member field value in a CPU register, and as far as I can see (but I
could be wrong) this is not done in V1.x.
From what I see in the v1.1 JIT/codegen source code, registers are
exclusively used to cache locals not member fields. I will try to
spend some time to investigate the exact semantics of volatile in the
CLR.

That might explain why it is nearly impossible for the problem in the
following code to manifest itself.

class Test
{
// To fix the problem this member should be marked as volatile.
private bool _Stop = false;

public void DoThreadStuff()
{
Thread t = new Thread(new ThreadStart(ThreadMethod));
t.Start();

Thread.Sleep(10000);

_Stop = true;
}

public void ThreadMethod()
{
int i = 0;
while (!_Stop)
{
i++;
}
}
}

It seems like a good idea to me to assume that any non-volatile member
could be cached. That way you are protected when you start running
your code on a new version of the framework that has a more aggressive
JIT optimizer.

*** Note that v2.0 doesn't generate an error when using a volatile
field ref. in Interlocked.XXXX, it just generates following:

warning CS0420: xxxxxxx: a reference to a volatile field will not be
treated as volatile

it's clear the "reference" will not be treated as volatile but the
field is treated as volatile.

Interesting. I'm still trying to determine if this a good a idea or
not.

Brian

 

[Jon Skeet [C# MVP]]

*** Sure, but here you assume that the JIT can cache the non-volatile member
field value in a CPU register, and as far as I can see (but I could be
wrong) this is not done in V1.x.
From what I see in the v1.1 JIT/codegen source code, registers are
exclusively used to cache locals not member fields. I will try to spend some
time to investigate the exact semantics of volatile in the CLR.

If you're going to go by what the current JIT does rather than what the
specification says, there are any number of things which could work - I
suspect double-checked locking works in the current implementation, for
instance.

*** Note that v2.0 doesn't generate an error when using a volatile field
ref. in Interlocked.XXXX, it just generates following:

warning CS0420: xxxxxxx: a reference to a volatile field will not be treated
as volatile

it's clear the "reference" will not be treated as volatile but the field is
treated as volatile.

Right. Interesting.

 

[Willy Denoyette [MVP]]

If you're going to go by what the current JIT does rather than what the
specification says, there are any number of things which could work - I
suspect double-checked locking works in the current implementation, for
instance.

Agreed, but the specification is a little confusing at best.
I guess you are talking about the C# Language (and/or the C# programmers
reference?) spec's. (IMO this doesn't even belong to the C# spec's at all).

From these, it's clear that volatile prevents reordering of the volatile
fields by the compiler, but it doesn't clearly state that it :
- prevents reads and writes to be cached in registers, the C# programmers
reference is a bit more explicit though (As I said before, IMO the v1.x JIT
doesn't cache fields in registers on X86).
- prevents combining read or write operations (I assume it does).
- prevents optimizations of non volatile variables relative to the volatiles
(here I assume it doesn't). For example, a write to a non volatile that
precedes a read from a volatile might be moved to execute after the read.
- it doesn't prevent reordering by the CPU hardware (Here I'm sure it
doesn't)
- guarantees that all CPU's see memory accesses in the same order (IMO not
possible)

Add to (my) confusion that:
VB.NET and J# do not support volatile - I wonder why?,
The C++ volatile specification is simply a description of the C compilers
implementation of volatile.
Different behavior, in both compiler and framework versions, when using
volatile fields in Interlocked class methods.
For me enough reasons not to rely on 'volatile' and prefer to use the
synchronization primitives offered by the framework, and that's your point I
agree with, use Interlocked methods to 'read' the an interlocked variable.

Willy.

 

[Willy Denoyette [MVP]]

synchronization primitives offered by the framework, and that's your point
I agree with, use Interlocked methods to 'read' the an interlocked
variable.

Or simply the System.Threading.Thread.VolatileRead methods.

Willy.

 

[Jon Skeet [C# MVP]]

Agreed, but the specification is a little confusing at best.
I guess you are talking about the C# Language (and/or the C# programmers
reference?) spec's. (IMO this doesn't even belong to the C# spec's at all).

No - I'm talking about the CLR spec along with the C# spec.

From these, it's clear that volatile prevents reordering of the volatile
fields by the compiler, but it doesn't clearly state that it :
- prevents reads and writes to be cached in registers, the C# programmers
reference is a bit more explicit though (As I said before, IMO the v1.x JIT
doesn't cache fields in registers on X86).

I think the C# spec says all that's needed, actually. Section 10.10 of
the ECMA spec:

<quote>
Execution shall proceed such that the side effects of each
executing thread are preserved at critical execution points. A side
effect is defined as a read or write of a volatile field, a write to a
non-volatile variable, a write to an external resource, and the
throwing of an exception. The critical execution points at which the
order of these side effects must be preserved are references to
volatile fields (§17.4.3), lock statements (§15.12), and thread
creation and termination.
</quote>

When the spec later talks about reordering, it is talking about
reordering the writes and reads in the memory model - this isn't
necessarily the order that the processor issues the instructions in,
due to caches etc.

Add to (my) confusion that:
VB.NET and J# do not support volatile - I wonder why?,

I suspect it's because volatility is a really tricky way of achieving
thread-safety. Locks are normally much easier to sort out.

The C++ volatile specification is simply a description of the C compilers
implementation of volatile.
Different behavior, in both compiler and framework versions, when using
volatile fields in Interlocked class methods.
For me enough reasons not to rely on 'volatile' and prefer to use the
synchronization primitives offered by the framework, and that's your point I
agree with, use Interlocked methods to 'read' the an interlocked variable..

I'd prefer to use straight locks unless I absolutely *knew* that they
would be a performance problem, and I had a lot of time to really
convince myself that declaring a variable as being volatile would be
good enough.