IEnumerator Usage

[Leslie Sanford]

I've been kicking around an idea mostly as a thought experiment at this
point. The idea is to create a thread safe collection that implements
the IEnumerable interface. The enumerators it creates via the
GetEnumerator method would be synchronized with the collction; if the
collection changes, the existing enumerators are notified via an event.
The accompanying EventArgs derived class object carries information
about the change to the collection so that the enumerators can update
themselves to keep the enumerations in sync.

This type of approach allows enumeration over a collection while it is
being changed without an exception being thrown. This type of approach
may be useful to me at some point in the future.

My question is this: the IEnumerator interface specification for the
MoveNext method states that an InvalidOperationException will be thrown
if the collection is modified after the enumerator was created. Using
the approach I described above, this would no longer be the case. So
would it be considered bad design to implement the IEnumerator interface
but not enforce this precondition? Should I abandon implementing the
IEnumerator interface?

The advantage of using the IEnumerator interface is it allows
enumerators to be used seamlessly with the foreach construct, so I would
really like to use it but have my doubts about weakening the IEnumerator
specification. Thoughts?

 

[Lucian Wischik]

My question is this: the IEnumerator interface specification for the
MoveNext method states that an InvalidOperationException will be thrown
if the collection is modified after the enumerator was created. Using
the approach I described above, this would no longer be the case. So
would it be considered bad design to implement the IEnumerator interface
but not enforce this precondition?

No!

That's my take on it. Also think of "yield return" and iterator
blocks, which offer the IEnumerator interface even though they don't
even have a collection at all.

I've even used IEnumerator to enumerate over values that must be
removed from a collection, where I *require* the value to be removed
from the collection before calling MoveNext. (because that was how the
"must-be-removed" algorithm worked). (nb. with values rather than
references.)

 

[Tasos Vogiatzoglou]

It is a bad design. When you iterate over a collection you are
interested in the collection's state at the moment you invoked the
iteration (these are the semantics).

Imagine having to take a decision a particular moment about an event
while iterating a collection that you cannot be sure if it has been
changed or not.

If you want to create a producer/consumer pattern you should consider
an alternative design.

As the MSDN documentation states "Enumerating through a collection is
intrinsically not a thread-safe procedure".

Regards,
Tasos

 

[Leslie Sanford]

It is a bad design. When you iterate over a collection you are
interested in the collection's state at the moment you invoked the
iteration (these are the semantics).

Imagine having to take a decision a particular moment about an event
while iterating a collection that you cannot be sure if it has been
changed or not.

Consider this example: Suppose you have a program for playing/editing
MIDI files (my area of expertise). MIDI files are made up of tracks
which are themselves made up of MIDI events. In other words, each track
is a collection of MIDI events.

Say that you would like to edit a track as it is being played. For
example, you may want to move an event forward in time or delete it.

Playing a MIDI file essentially involves iterating over the collection
of MIDI events in each track (with the speed of the iteration being
driven by timing events). If you want to edit these tracks as they are
being played, you are modifying a collection as it is being iterated
over.

So with the above scenario, I was thinking of creating iterators that
are capable of keeping themselves in sync with the track collections as
they are being modified.

IEnumerator aside, would this be considered bad design?

If you want to create a producer/consumer pattern you should consider
an alternative design.

Hmm, I will study the producer/consumer approach. Given the above
description, can you suggest a way of applying producer/consumer to my
problem?

As the MSDN documentation states "Enumerating through a collection is
intrinsically not a thread-safe procedure".

This is true of true of the collection classes provided by the .NET
framework, but does it have to hold true for your own custom collection
classes?

 

[Tasos Vogiatzoglou]

A "rough" approach (if I understood the problem correctly):

You have collection A that holds MIDI events. You have a thread PLAYER
that iterates through the collection at specified intervals. If you
modify collection A at position POS you can notify the PLAYER to stop
the iteration and begin again at the position that stopped. That way
the PLAYER will get the latest change (if it was at the position of
editing).

This could be one approach without having to resort to a IEnumerable
that ignores collection changes.

I don't think that producer/consumer applies well to this problem.

The rules are meant to be broken of course and because .NET
documentation says so this doesn't mean that you can't create such a
collection. But I personally believe that this may introduce bugs that
are not immediately visible ( or addressable) .

Regards,
Tasos

 

[Christof Nordiek]

I've been kicking around an idea mostly as a thought experiment at this
point. The idea is to create a thread safe collection that implements the
IEnumerable interface. The enumerators it creates via the GetEnumerator
method would be synchronized with the collction; if the collection
changes, the existing enumerators are notified via an event. The
accompanying EventArgs derived class object carries information about the
change to the collection so that the enumerators can update themselves to
keep the enumerations in sync.

This type of approach allows enumeration over a collection while it is
being changed without an exception being thrown. This type of approach may
be useful to me at some point in the future.

<snip>

Hi Leslie,

i think the greatest problem with this is, clearly defining the semantic.
I mean, how should the enumerator react if certain changes occur in the
underlying list. If that's clearly defined than your approach could work.
But maybe Tasos is right, that Enumarator is not the design for your
solution

HTH
Christof

 

[Nicholas Paldino [.NET/C# MVP]]

Leslie,

I agree with the majority of the opinions here that it is bad design.
The interface implies a contract, and you are violating that. It's not
something to be taken lightly, IMO. There are semantics that users of the
iterface you implement assume, and if you violate that, then their claims
about what they can do can't be enforced, and so on and so on.

To make your collection thread safe, I would offer one of the following:

- Throw an exception when the enumeration is changed
- Use locks to block changes on the collection while it is being enumerated
- Take a snapshot of the collection while enumerating and have the
enumerator use that instead of the store that is being modified.

Hope this helps.

 

[Leslie Sanford]

A "rough" approach (if I understood the problem correctly):

You have collection A that holds MIDI events. You have a thread PLAYER
that iterates through the collection at specified intervals. If you
modify collection A at position POS you can notify the PLAYER to stop
the iteration and begin again at the position that stopped. That way
the PLAYER will get the latest change (if it was at the position of
editing).

This could be one approach without having to resort to a IEnumerable
that ignores collection changes.

This is a good suggestion. Instead of trying to shoehorn
IEnumerable/IEnumerator into my design, I could instead have a class
dedicated to playback and keeping track of changes to the collection so
that the playback is synchronized. Thanks for your response.

 

[Lucian Wischik]

I agree with the majority of the opinions here that it is bad design.
The interface implies a contract, and you are violating that.

It's an odd sort of contract, though. This contract that you're
violating is a contract about what ISN'T supported (modifying the
collection in flight), rather than one about what IS supported. So the
only clients who would be affected by the proposal are those who
*rely* upon the "CollectionChanged" exception. And I'd call those
clients broken.

 

[Tasos Vogiatzoglou]

Lucian,
At first it may seem like an odd contract, but you are not violating a
contract about what is not supported, you violate the intented usage
pattern.

If you bring new developers to your project, when they will see a
collection they will assume that they can iterate the collection just
keeping in mind the case of concurrent modifications. But by violating
that contract, you give to your co-developers an IEnumerable with
broken semantics.

Noone excpects an infite loop for example for a collection enumeration,
something that can happen if you violate the (implied at least)
pattern.

Regards,
Tasos

 

[Lucian Wischik]

Noone excpects an infite loop for example for a collection enumeration,
something that can happen if you violate the (implied at least)
pattern.

I don't see it. I mean, the only way the code would get an infinite
loop is if another thread was modifying the collection. So in the
original semantics this would have caused a fault (exception). And in
the new semantics it also causes a fault (infinite loop). All clients
who expected the original semantics will have avoided the exception,
and so their code will work exactly in the same way, unchanged, with
the new semantics. So this is a "conservative" extension to the
semantics -- it doesn't break anything, it merely adds new
functionality.

 

[Tasos Vogiatzoglou]

I agree. But in the "original semantics" there is a fail-fast
assurance. If there is a concurrent modification then you have an
exception. The "new semantics" are more prone to race conditions
(comparing to the original ones).

If I were to design such a system, I would first check the alternatives
but if nothing was a good solution of course I would go with your way
....

What I am saying is that, it's a risky choice.

Regards,
Tasos