Bogus memory usage

[Nebulon]

Noticing my computer slowing down here, and task manager says 960MB in
use. (It really shouldn't start to swap and get slow for another 64 or
so MB, then, but go figure.)

Oddly enough, the task manager doesn't seem to explain the usage. As
administrator and with "Show processes from all users" checked, the VM
Size column sums to maybe 700-750MB. The three biggest items are
159,604 (That one's something that uses Java, of course), 138,940
(Browser with many open windows and yep, a Java applet loaded in one of
'em), and 101,852 (Exploder). Another 11 items are in the five digits;
all close to 20M except for three that average more like 40M, so call
that 14*20 or another 280 meg. The rest might make up another 20-50M.

So we have two big items adding to around 300M, many small items adding
to maybe another 300, and another big item weighing in at around 100.

All of this is somewhat approximate (particularly as I'm estimating
megs as 1 million bytes instead of 1048576) but I very much doubt that
the sum exceeds 750.

That means about 200MB are allocated by something that isn't any
running process, as far as I can tell.

Of course, the first guess is that maybe malware is hiding itself from
task manager, but nothing shows up (including with RootkitRevealer --
hiding 200MB from a system administrator probably requires deploying a
rootkit, but nothing crops up). AV, spyware detectors, etc. show clean.

This leads me to suspect that if we aren't dealing with a "rootkit"
we're dealing with a "severe Windows bug" -- what a shocker -- and it's
either causing task manager to misreport memory usage, misreport
process sizes, and/or miss processes entirely, and/or it's actually
gobbling up memory somewhere that task manager doesn't look, maybe
device drivers, DMA buffers, and/or the kernel.

Anyone have a clue, or any idea how to free some of it up short of
rebooting?

 

[Richard Urban]

Process Explorer. Find that malware hiding on your system using up
resources.

http://www.microsoft.com/technet/sysinternals/utilities/ProcessExplorer.mspx

--

Regards,

Richard Urban
Microsoft MVP Windows Shell/User
(For email, remove the obvious from my address)

Quote from George Ankner:
If you knew as much as you think you know,
You would realize that you don't know what you thought you knew!

 

[Nebulon]

Process Explorer. Find that malware hiding on your system using up
resources.

Eh. Strange. It shows VM sizes for everything vastly huger than task
manager does -- in fact, the numbers add up to way too *much*. Some
things are doubled in size. Total usage is around 2GB if Process
Explorer is the accurate one.

My guess is that neither of them are. Task Manager is underestimating
by a moderate amount and Process Explorer overestimating by a huge
amount.

Why either of them are *estimating* at all I haven't a clue.

I see that Explorer and Yahoo Instant Messenger are the culprits
responsible for leaking so many GDI handles.

On the other hand, it's Firefox and Java that take the memory cake.
Process Explorer claims they are each half a gig(!) in size -- surely
not. The Java task should be almost exactly 256MB given the -Xmx256M on
its command line. And no Web browser should ever bloat up to even
256MB. Task manager shows me a Firefox process 1/3 the size of the one
Process Explorer shows. (And yes, there's only the one.)

Private Bytes looked like it might be more accurate -- problem is
instead of counting shared material multiple times it doesn't even
count it once, for a grand total of about 500M, a worse underestimate
than Task Manager's.

Peak Working Set Size gives the numbers closest to Task Manager's VM
Size.

*Nothing* seems to be able to enlighten me as to where 200M just
disappeared to.

No suspicious processes are showing up. (In particular, nothing that
isn't in Task Manager, aside from DPCs and Interrupts, which seems to
be normal.)

 

[Nebulon]

Nebulon wrote:
[snip]

Well, some of the bloat was hidden in Yahoo Instant Messenger. I noted
the commit charge in Task Manager (940M at that moment) and the YIM
process size (40M), then nuked it. (Actually, I right clicked its icon
and hit "exit", but "nuked it" sounds more impressive.) Once it had
died (no YIM entry in Task Manager) the commit charge showed as 880M.

It doesn't take a genius to note that 940 - 40 = 900, not 880.

So 1/10 of the "dark matter" was clumped in an invisible halo around
YIM.

No doubt, the other 9/10 is clumped around Exploder, where it can't be
freed without having to either reboot or lose all my folder position
and current-directory state.

Yuck.

 

[Mak]

Eh. Strange. It shows VM sizes for everything vastly huger than task
manager does -- in fact, the numbers add up to way too *much*. Some
things are doubled in size. Total usage is around 2GB if Process
Explorer is the accurate one.

My guess is that neither of them are. Task Manager is underestimating
by a moderate amount and Process Explorer overestimating by a huge
amount.

<snip>
Both TM and PE are right / accurate.
The reason for the difference you are seeing is quite simple: TM columns
labels are not the same as PE labels, i.e. they show different things.
(and IMO names MS chose for TM columns are way off)

Let's label everything as it should be:

Perfmon counter TaskManager column ProcessExplorer column
Explanation
(Process object) (processes tab)
private bytes VM size Private Bytes
Processe's contribution to commit charge (stuff that get's backup by paging
file)
virtual bytes (can't show) Virtual Size
Total virtual memory in use by process
working set mem usage Working Set
Part of the processe's virtual memory currently in RAM (some of it is
sharable)

 

[Nebulon]

Mak wrote:
[snip]

Interesting, but it doesn't solve my problem, to wit, plugging the
holes in whatever's leaking.

 

[Mak]

I wasn't trying to solve the problem you are having...

....reading your original post now.... how do you know the slow down is
because of.. memory? Maybe some of the threads you are running are I/O
bound?( or CPU bound?)
I mean, what makes you think your computer paging to disk excessively due to
high page fault rate in one or more of your applications? It can be, I just
can't see it in your OP post.
BTW what your paging file size is set to? How much of that 940MB of commit
charge is actiually in paging file?

 

[Nebulon]

I wasn't trying to solve the problem you are having...

...reading your original post now.... how do you know the slow down is
because of.. memory?

When the mem use tops 1GB and the system starts to swap, it really
isn't very hard to guess.

 

[Mak]

The smart young man like you - what was I thinking?
Good luck with your Windows 98.

 

[Nebulon]

The smart young man like you - what was I thinking?
Good luck with your Windows 98.

I'm using XP actually. This *is* the XP-specific newsgroup...

 

[Mak]

xp doesn't swap

I'm using XP actually. This *is* the XP-specific newsgroup...

 

[Nebulon]

xp doesn't swap

What have you been smoking, and can I have some too?

No, seriously -- what modern OS doesn't have VM and the ability to have
gigs more VM than physical RAM? And on what planet is it to be found?
Sheesh.

Now, could someone who possesses something vaguely resembling what's
known colloquially as a "clue" please respond?

 

[Mak]

in plain English - Win98 (and alike) has memory reclamation protocol known
as swapping, XP (and alike) use paging. The difference isn't just a word
used. Since the OS you're running is "swapping" I conclude you have Windows
98.

quoting your post: "When the mem use tops 1GB and the system starts to swap,
it really
isn't very hard to guess. "

Now, are you going to answer the questions I asked you, or you are going to
*guess* what's happening all by yourself?

 

[Nebulon]

in plain English - Win98 (and alike) has memory reclamation protocol known
as swapping, XP (and alike) use paging.

Call it what you like. And quit splitting hairs. Memory use exceeds
physical RAM and the system slows down blocking on disk I/O all the
time because of that. Now can we get back to the topic at hand?

 

[Mak]

You've managed to not answer my questions again.
Your attitude hasn't changed.

This news group has plenty of knowledgeable people willing to help, but... I
don't see them falling all over each other trying to do that for you.
Good luck to you.

 

[Nebulon]

You've managed to not answer my questions again.

I'm the one asking the questions here.

Your attitude hasn't changed.

Said attitude being that when I ask a question, I prefer answers to
more questions, and I don't especially like semantic word-games?

Explorer and other processes are bloating up. This causes the system to
spend a lot of time blocking on disk I/O, once the VM in use exceeds
the amount of physical RAM. Whatever you choose to call that process,
it makes things slow. What is causing the bloat/leakage? Is there a
known fix?