memory leak

[Guest]

Patience - when I get more info, I promise I will let you know. I see
different behavior in different situations, which is _not_ a good thing, so
the test team is taking a look at it. I assume they have other
responsibilities too, so sometimes it takes a few days for them to be able
to get to it.

-Chris

 

[Functional Illiterate]

Hey all,

Just wanted to chime in on this one. I'm glad that ctacke is getting
some .NET folk to take a look at this little bitty. I've seen this
failed bitmap "artifact" problem in image intensive applications that I
have worked on. It was much more apparent in CF 1.0, but since the GC
in 2.0 is so much better, the bugger shows it s head much less
frequently.

I am pretty diligent when it comes to disposing unmanaged resources.
Bitmaps, however, manage to thwart all my attempts to be concientious.
I have seen the following.

If an OOM occurs when attempting to create a bitmap from a binary
stream or whatever, the GC does run and all unused objects will be
cleaned up, making it appear that the device has more available memory.
However, due to some strange heap corruption or something, the mem
allocated for the bitmap that failed to be created does not get
released. Since these "artifacts" stay in memory through multiple
instances of an application, hitting one's head against this ceiling
too many times will eventually lead to the device being useless until
you soft reset.

I suppose this is the same behaviour pointed out by Hilton's code. I
hope some resolution is found for this problem; a fix for this will
make me a hero in my village.

f(illiterate)

 

[Guest]

And here's what I found out:

http://blog.opennetcf.org/ctacke/PermaLink,guid,987041fc-2e13-4bab-930a-f79021225b74.aspx

 

[Hilton]

Chris,

Thanks for posting this info - I'm glad I found the problem, I definitely
learned from the info the CF team came back with. Having said that:

1. Is it just my natural "Defensive Coding" nature or does anyone else find
this really really scarey? For example, it seems as though, since MSFT have
already said that they will fix this problem, that we're going to have to
check for and code for specific CF versions. Secondly, performance is
dependant on which constructor gets used?!? Thirdly, exception handling and
GC behavior is dependant on which constructor gets used. So now we're left
to do our own memory management and we have no idea how our performance
will/may take a hit when users change to a new CF.

2. Chris, do you know if the information you posted is specific for CF 1,
CF 2, or both? (I have seen the same (incorrect) behavior on both)

3. When do they plan on fixing it? How do they plan on fixing it? How
will their fix impact an application's performance?

4. What other classes in CF (or Full) exhibit such odd (and incorrect)
behavior? Do the CF Team know of any that might jump up and bite us like
Bitmap did to me?

5. Why did they do it like this?

6. When will I run out of video RAM? i.e. I'm assuming that I could still
have lots of 'free memory', yet a "new Bitmap (w, h)" call will fail when
video RAM is exhausted. This will throw an OOM exception (perhaps
ArgumentException on Full Framework) when I have a ton of virtual memory
left because it may be allocating memory in the 'dedicated video ram'.

This is more than an implementation bug, this is a fundamental design flaw
in the CF (and Full Framework which is fixed in its latest version) and one
which goes against the fundamental nature of VM/GC notion. Again, I love
what the CF Team have given us, but IMHO this is a serious failure of the
design, implementation, and testing of .NET at Microsoft.

As always, comments please and correct me if I'm wrong.

Again for the record, I love C#, CF, etc etc etc, but I call it as I see it.

Hilton

 

[Guest]

Inline...

1. Is it just my natural "Defensive Coding" nature or does anyone else
find this really really scarey? For example, it seems as though, since
MSFT have already said that they will fix this problem, that we're going
to have to check for and code for specific CF versions. Secondly,
performance is dependant on which constructor gets used?!? Thirdly,
exception handling and GC behavior is dependant on which constructor gets
used. So now we're left to do our own memory management and we have no
idea how our performance will/may take a hit when users change to a new
CF.

If you code defensively, you'll never hit this. If you use the
WaitForPendingFinalizers approach, then if it is fixed in a future version,
the exception won't get thrown and the wait will never occur, so in theory
perf will improve with no code modifications.

2. Chris, do you know if the information you posted is specific for CF 1,
CF 2, or both? (I have seen the same (incorrect) behavior on both)

I only tested with CF 2.0 SP1, so I can't say for certain. If I were to
guess I imagine it's been there all along.

3. When do they plan on fixing it? How do they plan on fixing it? How
will their fix impact an application's performance?

I don't work for Microsoft so I have no idea if or when this behavior may
get changed. I can't imagine it would impact perf in any way except the GC
would collect properly without you having to call Dispose. It might
actually be a little slower because it may add an additional check.

4. What other classes in CF (or Full) exhibit such odd (and incorrect)
behavior? Do the CF Team know of any that might jump up and bite us like
Bitmap did to me?

That's like asking what bugs are out there that are unknown. I don't
specifically know of any other behaviors like this, but then there's no
guarantee. The CF is heavily tested, but only through actual use and
feedback will more difficult to find problems get addressed.

5. Why did they do it like this?

I didn't architect it or have any part in coding it, so I have no idea. I
do know that the CF team is full of very good developers and architects, so
I'm sure there was some fundamental reason behind it. Whether is was "good"
or "bad" I can't say. I just may have done things slightly different if the
world were perfect and I got to run things.

6. When will I run out of video RAM? i.e. I'm assuming that I could
still have lots of 'free memory', yet a "new Bitmap (w, h)" call will fail
when video RAM is exhausted. This will throw an OOM exception (perhaps
ArgumentException on Full Framework) when I have a ton of virtual memory
left because it may be allocating memory in the 'dedicated video ram'.

What is your definition of "video RAM"? Many CE devices don't have a
separate graphics controller with its own RAM. Some that do still don't use
it for bus-architecture reasons. And if such hardware is present, the CF
has no idea that its there, so it's purely up to the driver if/when it
should allocate against it. Typically this is used for the display driver's
framebuffer(s)

You could try to allocate and have it throw if you're out of virtual memory.

This is more than an implementation bug, this is a fundamental design flaw
in the CF (and Full Framework which is fixed in its latest version) and
one which goes against the fundamental nature of VM/GC notion. Again, I
love what the CF Team have given us, but IMHO this is a serious failure of
the design, implementation, and testing of .NET at Microsoft.

Again, I see it as a simple implementation bug. When it OOMs on creation,
the runtime should collect, wait for finalizers and allocate again. Plain
and simple. It would then behave like a GC system is supposed to.


--
Chris Tacke
OpenNETCF Consulting
Managed Code in the Embedded World
www.opennetcf.com
--

 

[Hilton]

I don't work for Microsoft so I have no idea if or when this behavior may
get changed. I can't imagine it would impact perf in any way except the
GC would collect properly without you having to call Dispose. It might
actually be a little slower because it may add an additional check.

See your blog, in it the CF Team mention degraded performance when doing
LockBits.

That's like asking what bugs are out there that are unknown.

Nope, I asked if they *knew* of any similar cases.

What is your definition of "video RAM"?

See your blog, in it the CF Team mention "video RAM".

Hilton

 

[Guest]

I don't work for Microsoft so I have no idea if or when this behavior may

See your blog, in it the CF Team mention degraded performance when doing
LockBits.

Perf depends on whether you're using a DDB or DIB under the hood. For the
same code, a change to prevent this OOM will have no effect on whether a DIB
or DDB is used and therefore isn't going to change perf.

Nope, I asked if they *knew* of any similar cases.

Again, I don't work for MS and I have no idea what they know. Just like
this one - Scott blogged about it some months ago, so they're not "hiding"
anything if that's what you're asking. They're quite forward with what they
know.

See your blog, in it the CF Team mention "video RAM".

I know, but that doesn't affect my statement. The statement was "...the
driver could allocate these in dedicated video ram." which again has nothing
to do with the CF. In fact Microsoft has zero control over this. The OEM
has to write the display driver, and they can choose to do it in a near
infinite number of ways. The CF has no way of knowing anything about how it
works, nor should it. It simply calls to GWES and says "I need a DDB of x
size". GWES calls the driver and gets back a handle. It has no idea where
it came from or how it was allocated.

-Chris

 

[Functional Illiterate]

Hey all,

First off, thank you. I no longer consider this a bug, rather, I
consider it a caveat that any .NET CF developer using Bitmaps must be
aware of.

Chris, as you suggested in your blog, I spent a bit of time massaging
the code listed to see how the memory responds. I used the .NET
Compact Framework RPM of CF 2.0 SP1 & perfmon to display a running
account of GC statistics. I also used the Win CE RPM v5.0 that comes
with EVC 4.0, the tool the .Net CF RPM seems to have replaced.

For the WinCeRPM(evc4.0) I displayed the following CE Memory
Statistics:
- Available Physical (scale = 0.000001)
- Memory Load (scale = 1.0)
- Total Physical (scale = 0.000001)

The main difference I witnessed was with the second example where the
Bitmap (int, int) constructor was used. Using the WinCE RPM(evc 4.0),
you'll quickly see the Available Physical Memory drop 27 - 32MB,
roughly equivalent to the max mem allowable for the process. I'd
expect the CLR to clean that up when the application exits, however,
that memory is "leaked" until the device is hard reset. This is what I
had been seeing occasionally with one app I have been working on.

While reading the response from the CF team concerning DIBs and DDBs, I
started to wonder what happens with an ImageList, more specifically,
what happens under the hood with a ImageList? When adding a bitmap to
an ImageList, is a reference to the original bitmap used or does a new
bitmap get created??? If a new bitmap is created, which constructor is
used???

After a bit of experimenting it seems that the ImageList creates a copy
of the original Image. Fortunately, it does not seem to use the
Bitmap(int, int) constructor. I was fearing for the worse given the
ImageSize property. Exactly what happens under the hood, only MS knows
for certain? What is certain is that all members of the ImageList
collection must also be disposed of since it appears new images are
created.

I took the liberty to modify some of the code from the blog to
determine if a reference or a copy of the Bitmap was used in the
ImageList. The following code snippet runs in a ThreadPool thread,
hence the raising events & etc.:

while (loop_run) {
try {
iterations++;
using (Bitmap b = new Bitmap(GetImageStream())) {

il_5.Images.Add(b);
}

RaiseUpdateListViewEvent(obj_since_failure++);

if (iterations % 100 == 0) {
Debug.WriteLine(string.Format("{0} objects
Created", iterations));
break;
}
} catch (OutOfMemoryException) {
Debug.WriteLine("Waiting for finalizers to
run...");
Debug.WriteLine(string.Format("{0} objects since
last failure", obj_since_failure));
obj_since_failure = 0;
ClearImageList(il_5);
GC.WaitForPendingFinalizers();
} catch (Exception) {
Debugger.Break();
loop_run = false;
}
}


In the example above, the ImageList il_5 is bound to the SmallImageList
of a ListView object. The RaiseUpdateListView method raises an event
to create a new ListViewItem and update the Listview. I figured, if
the ListView displays the Bitmap even after Bitmap b is disposed, then
the Image must have been copied somewhere. This is indeed what
happens.

In addition to this example, I also added the bitmaps created in each
example of ctacke's blog to separate ImageLists. With example 1 from
ctacke's blog, adding the Bitmap to a ImageList made the "ideal"
performance dissappear pretty quickly. Fortunately, all resources were
released when the application exits. All of the other examples behave
roughly in the same manner. Example number 4 will occasionally throw
an regular Exception when attempting to add the bitmap to the ImageList
after memory usage gets past a certain point.(Weird)

Anyhow, this ballyhoo has shed some light on what one I must do to
clean up my app which uses a fair amount of Bitmaps, ImageLists, etc.

Thank you all for the insight.

f(illiterate)

 

[Guest]

Thanks for the in-depth follow up. Could you post the same into a comment
on my Blog so others looking for this info find the results of your hard
work as well?