Measurement of CPU usage

[Josef W. Segur]

Josef W. Segur escreveu:


Hi Joe
Thanks for your comments.
That's an interesting observation about a difference in CPU activity
between Win9x/Me and NT OSs. However, I cannot as yet see how this
would lead to two different numeric assessments of CPU activity on
machines with Win9x/Me.

I agree that it doesn't indicate _what_ differences would be needed
for calculating usage on Win9X vs. WinNT based systems, just that
there would need to be separate routines.

Since I initiated this thread, I have learnt that FreeMeter and Process
Explorer give essentially identical CPU Usage readings on machines
running WinXP (based on tests with two different computers).
Nevertheless, my own computer on WinMe continues to show a very marked
difference (FM number is normally 10-20 times that of PE) between the
indicated CPU Usage for these two utilities.
I still don't know if this is due to some peculiarity of my computer or
whether it is a characteristic of machines running on Win9x/Me.
Is there anybody out there running any of these non-NT OSs who would be
prepared to try a small, free, risk-free test on CPU Usage?

So I _did_ a small, free, risk-free test. That consisted of shutting
down BOINC/setiathome on my Win98 system and looking at a few things.
TClockEx reported 32% CPU usage, as did System Monitor. WinTop and
Process Explorer showed over 97% idle.

Both TClockEx and System Monitor are simply getting info from the
Performance section of the registry, specifically in the key
HKEY_DYN_DATA\PerfStats\StatData\KERNEL\CPUUsage
and I expect if you'd check the other utilities which report the
bogus high values they'd be using the same.

WinTop has its own vxd to get timing information for each thread, so
doesn't rely on the registry. Process Explorer also reports CPU for
each process, so must use a similar method.

This obviously doesn't answer the original question, but may reduce
it to "Why does the OS report more usage than it should?".

To add to the confusion I'll note that for the same test on my Win95
system, System Monitor and TClockEx showed only about 3% CPU usage.

 

[PaulFXH]

Josef W. Segur escreveu:

I agree that it doesn't indicate _what_ differences would be needed
for calculating usage on Win9X vs. WinNT based systems, just that
there would need to be separate routines.


So I _did_ a small, free, risk-free test. That consisted of shutting
down BOINC/setiathome on my Win98 system and looking at a few things.
TClockEx reported 32% CPU usage, as did System Monitor. WinTop and
Process Explorer showed over 97% idle.

Hi Josef
Thanks a lot for this extremely interesting reply. For the first time,
somebody else has reported the same major discrepancy between CPU usage
as reported by various different utilities for a computer running on
Win9x.
At last I can conclude tht I'm not going totally crazy.

Both TClockEx and System Monitor are simply getting info from the

Are you sure you mean TClockEx? This is a clock that, as far as I know,
does not at all measure CPU usage.

Performance section of the registry, specifically in the key
HKEY_DYN_DATA\PerfStats\StatData\KERNEL\CPUUsage
and I expect if you'd check the other utilities which report the
bogus high values they'd be using the same.

I was unaware of the existence of this Key. Do you know where it gets
its information from?
I compared the figures that this Key was showing with the CPU usage
indicated by FreeMeter and they are certainly similar.
Do you know how I can set up a graph of the figures displayed by this
key? (as they jump around so much it's hard to get an impression of the
average when you have to close the key and open it again to see a new
value).

WinTop has its own vxd to get timing information for each thread, so
doesn't rely on the registry. Process Explorer also reports CPU for
each process, so must use a similar method.

This obviously doesn't answer the original question, but may reduce
it to "Why does the OS report more usage than it should?".

As far as I can see, this goes a very long way towards explaining the
phenomenon I had observed.
However, are you saying that the result shown in the
HKEY_DYN_DATA\PerfStats\StatData\KERNEL\CPUUsage key is just nonsense
(as consequently are the results of all utilities that use this for CPU
usage reporting)?

To add to the confusion I'll note that for the same test on my Win95
system, System Monitor and TClockEx showed only about 3% CPU usage.

Nevertheless, what you have shown is that what I had observed appears
to be a real phenomenon and may be (at least partially) explained.
If you have any further thoughts on this subject, I hope you will share
them as I've waited quite some time for this breakthrough.
Many thanks Josef
Paul

 

[Terry Russell]

I was unaware of the existence of this Key. Do you know where it gets
its information from?
system

I compared the figures that this Key was showing with the CPU usage
indicated by FreeMeter and they are certainly similar.
Do you know how I can set up a graph of the figures displayed by this
key? (as they jump around so much it's hard to get an impression of the
average when you have to close the key and open it again to see a new
value).
dynamic


As far as I can see, this goes a very long way towards explaining the
phenomenon I had observed.
However, are you saying that the result shown in the
HKEY_DYN_DATA\PerfStats\StatData\KERNEL\CPUUsage key is just nonsense
(as consequently are the results of all utilities that use this for CPU
usage reporting)?


Nevertheless, what you have shown is that what I had observed appears
to be a real phenomenon and may be (at least partially) explained.
If you have any further thoughts on this subject, I hope you will share
them as I've waited quite some time for this breakthrough.
Many thanks Josef
Paul

There have been several methods to get performance data in windows,
each seems to break with new generation processors.

e.g. win98 builtin cpu monitor
on P400 unloaded 3% , on P2000 50%, run a second and third copy
and sometimes they tend to be mirror images of each other, bad results
similar things can happen for most simple cpu measures

just find one that seems okay, or go for a more intrusive app

Some official means to get cpu, if 'taint broke why does it need fixin?

http://msdn.microsoft.com/library/d...-us/perfmon/base/createtoolhelp32snapshot.asp
http://msdn.microsoft.com/library/d...e/taking_a_snapshot_and_viewing_processes.asp
http://msdn.microsoft.com/library/d...egistry_functions_to_consume_counter_data.asp

then , on later model OS you have available
http://msdn.microsoft.com/library/d...dk/wmi/win32_perfrawdata_perfos_processor.asp

the best a non-intrusive app can do is guess, or rely on nor very reliable
windows logging
accuracy also varies with OS and almost everything that gets clocks under
the OS
it hides real cpu use , anything could be consuming slabs of clocks
within the OS and still show as 0% use
that 0..100 is just whats left over for you

The apps are probably reading off each other, see if they change with start
order
and if 4 copies of each app give the same results
the most stable and consistent is probably the most accurate

nonetheless most of the guesses can still be useful for judging relative use
...the change is what you are measuring rather than absolute use.

There is little point in trying to come to a deep understanding of OS and
hardware operation,
the vast bulk of anything you learn will be redundant next year.

 

[PaulFXH]

Terry Russell escreveu:

There have been several methods to get performance data in windows,
each seems to break with new generation processors.

e.g. win98 builtin cpu monitor
on P400 unloaded 3% , on P2000 50%, run a second and third copy
and sometimes they tend to be mirror images of each other, bad results
similar things can happen for most simple cpu measures

Hi Terry
Thanks for your comments.
Are you saying that this particular cpu monitor can produce results
that are nonsense?

just find one that seems okay, or go for a more intrusive app

But how do I know what's OK? On my computer, cpu usage seems to fall
into one of two categories; very high (FreeMeter, ACPU, PCWizard and
many others) or low (WinTop, Process Explorer)
Are you saying this results from some being non-intusive (FreeMeter
etc) while WinTop/Process Explorer are intrusive gathers of CPU usage
information?
Do you know of a link from which I can learn some more regarding this
concept?

Some official means to get cpu, if 'taint broke why does it need fixin?

http://msdn.microsoft.com/library/d...-us/perfmon/base/createtoolhelp32snapshot.asp
http://msdn.microsoft.com/library/d...e/taking_a_snapshot_and_viewing_processes.asp
http://msdn.microsoft.com/library/d...egistry_functions_to_consume_counter_data.asp

then , on later model OS you have available
http://msdn.microsoft.com/library/d...dk/wmi/win32_perfrawdata_perfos_processor.asp

the best a non-intrusive app can do is guess, or rely on nor very reliable
windows logging
accuracy also varies with OS and almost everything that gets clocks under
the OS
it hides real cpu use , anything could be consuming slabs of clocks
within the OS and still show as 0% use
that 0..100 is just whats left over for you

Once again, you seem to be saying that non-intrusive measurement of
cpou usage produces nonsense results?

The apps are probably reading off each other, see if they change with start
order
and if 4 copies of each app give the same results
the most stable and consistent is probably the most accurate

I ran 4 copies of FreeMeter on my computer and compared the results
with the registry key that Josef pointed to in his last post
(HKDynData).
Initially all four copies gave very similar results. However, after
about 2-3 minutes, three of them rose to 100% and more or less stayed
there over a 15 minute period.
The other copy of FreeMeter showed a consistent CPU usage of around 75%
throughout this time.
During this time, the values on the HKDynDate registry key appeared to
fluctuate between 64(hex=100dec) and 4b (hex=75dec).
At the same time, Process Explorer showed that CPU usage (other than
System Idle Processes) was 5-6%.

If I assume that FreeMeter is a non-intrusive indicator of CPU usage,
how can we explain that on machines running WinXP it appears to provide
the same results as does the intrusive Process Explorer?
If it can measure (I presume) intrusively on WinXP boxes, why can it
not do so on machines running WinMe?

nonetheless most of the guesses can still be useful for judging relative use
..the change is what you are measuring rather than absolute use.

There is little point in trying to come to a deep understanding of OS and
hardware operation,

I'd actually settle for a limited understanding which I'm just starting
to get.

Many thanks for providing these insights.
Paul

 

[Josef W. Segur]

Josef W. Segur escreveu:

Hi Josef
Thanks a lot for this extremely interesting reply. For the first time,
somebody else has reported the same major discrepancy between CPU usage
as reported by various different utilities for a computer running on
Win9x.
At last I can conclude tht I'm not going totally crazy.

Are you sure you mean TClockEx? This is a clock that, as far as I know,
does not at all measure CPU usage.

Quote from the TClockEx help file:

Although the main purpose of TClockEx is to provide time and date
functions, the ability to monitor CPU, memory and resource usage
is often invaluable. TClockEx has special display elements that
expand to report CPU usage, memory load, and system resources.

I was unaware of the existence of this Key. Do you know where it gets
its information from?

There are vxd files installed to support System Monitor which provide
the performance data.

I compared the figures that this Key was showing with the CPU usage
indicated by FreeMeter and they are certainly similar.
Do you know how I can set up a graph of the figures displayed by this
key? (as they jump around so much it's hard to get an impression of the
average when you have to close the key and open it again to see a new
value).

Use System Monitor. Although it's awkward to configure, that's what
Microsoft has provided.

As far as I can see, this goes a very long way towards explaining the
phenomenon I had observed.
However, are you saying that the result shown in the
HKEY_DYN_DATA\PerfStats\StatData\KERNEL\CPUUsage key is just nonsense
(as consequently are the results of all utilities that use this for CPU
usage reporting)?

I wouldn't call it nonsense, just a flawed measurement. The numbers
do go up for heavy usage. Terry's comment that it's related to the
CPU type explains the inaccuracy well enough, I think.

 

[PaulFXH]

Josef W. Segur escreveu:

Quote from the TClockEx help file:

Although the main purpose of TClockEx is to provide time and date
functions, the ability to monitor CPU, memory and resource usage
is often invaluable. TClockEx has special display elements that
expand to report CPU usage, memory load, and system resources.

Hi Josef
I stabd corrected. Never knew that CPU usage was there in TClockEx. If
nothing else, it does add a little color to the gray taskbar.

There are vxd files installed to support System Monitor which provide
the performance data.

From what I can see, the information that both you and Terry have

provided refer only to the (apparently) flawed and (perhaps)
questionable CPU usage information such as TClockEx provides.
But how does Process Explorer make its assessment of CPU usage and why
is this apparently more difficult and more complicated to access?

Use System Monitor. Although it's awkward to configure, that's what
Microsoft has provided.

Josef, I cannot seem to find any System Monitor on my WinMe OS. Should
it be there?
Bear in mind that my computer is configured in the Portuguese language
so it may be called something else although I have searched for all
reasonable translations.

I wouldn't call it nonsense, just a flawed measurement. The numbers
do go up for heavy usage. Terry's comment that it's related to the
CPU type explains the inaccuracy well enough, I think.

So, basically, it's useful on a relative basis but is only roughly
comparable to the more accurate information derived from Process
Explorer or WinTop.

Many thanks
Paul

 

[Josef W. Segur]

Josef W. Segur escreveu:


Josef, I cannot seem to find any System Monitor on my WinMe OS. Should
it be there?
Bear in mind that my computer is configured in the Portuguese language
so it may be called something else although I have searched for all
reasonable translations.

It may not be installed. For English you'd double-click My Computer,
Control Panel, and Add/Remove Programs. On the Windows Setup tab,
double-click System Tools and check System Monitor.

 

[PaulFXH]

Josef W. Segur escreveu:

It may not be installed. For English you'd double-click My Computer,
Control Panel, and Add/Remove Programs. On the Windows Setup tab,
double-click System Tools and check System Monitor.

Hi Josef
Thanks for your reply.
Yeah, you're right. System Monitor wasn't installed on my computer.
However, now it is and provides the graph I was looking for. This gives
more or less the same numbers as FreeMeter.
Interestingly, my WinMe OS provides me with two Windows measurements of
CPU Usage which are very, very different. Hasn't anybody noticed this
before?
The posts I saw from both you and Terry over the last week or so have
been extremely informative.
However, I really would like to learn a little more and I really cannot
find anything on Google.
Are you aware of a link that can explain why there are two methods of
measuring CPU Usage, which provide wildly different results; why there
is a difference in measurement method and what is the
significance/reliability of each category of result?

TIA
Paul

 

[Terry Russell]

Josef W. Segur escreveu:


Hi Josef
Thanks for your reply.
Yeah, you're right. System Monitor wasn't installed on my computer.
However, now it is and provides the graph I was looking for. This gives
more or less the same numbers as FreeMeter.
Interestingly, my WinMe OS provides me with two Windows measurements of
CPU Usage which are very, very different. Hasn't anybody noticed this
before?
yes

The posts I saw from both you and Terry over the last week or so have
been extremely informative.
However, I really would like to learn a little more and I really cannot
find anything on Google.
Are you aware of a link that can explain why there are two methods of
measuring CPU Usage, which provide wildly different results; why there
is a difference in measurement method and what is the
significance/reliability of each category of result?

a quick google shows
http://www.jsifaq.com/SUBQ/tip8400/rh8404.htm
http://www.microsoft.com/technet/archive/win95/w9p96c2.mspx?mfr=true

which means it has been mystifying people for a long time
ME has a faulty cpu use algorithm ,they say, but I have seen the same on all
version of windows.
I don't have a clue _why_ I just know enough to recognise it.

You haven't gotten an answer because no-one feels like giving a course in
multitasking operating system
scheduling or can be sure what mirror they are looking at through the smoke.



Any task that has the cpu runs at full speed until it is done or uses up its
timeslice and time is given to another process, or a higher priority
interrupt
preempts it for a short time.
If nothing is waiting the cpu goes to the idle process, which usually does
very little. This is a good place for things to get done that are not
urgent.

The measure of time uses can be as reported by the system, or by scanning
all processes and taking the difference, which should be what the system
reports
but isn't always, or subtracting it from the proportion used in idle
process.

Now systems may have multiple processor cores , different generations
have different lookahead and cache which may fool the counter about
how much work has been done in a period, the measure of cpu time
vs things done in that time isn't linear. Now the count of clocks should be
accurate, but
it appears the system doesn't always count that, and isn't the same
on older processors anyway.
Some systems may be running power management, which can halt or
slow down the processor when it would otherwise be in-idle,
which may affect whatever method the logger app uses..
or any other number of ways I don't know or care to think too much about

Don't think of CPU use in terms of car tacho and fuel consumption, it
isn't measuring anything static anyway
think of 5% as 'currently wasting $950 of your $1000 investment'.

Unless you use a laptop..in which case power management should be builtin,
there isn't much point worrying too much about cpu use , just get a tool
you think is reliable, get a benchmark of resting and turning sluggish, call
that
0% and 100% and work with that.
Differences are what count.

 

[PaulFXH]

Terry Russell escreveu:

a quick google shows
http://www.jsifaq.com/SUBQ/tip8400/rh8404.htm
http://www.microsoft.com/technet/archive/win95/w9p96c2.mspx?mfr=true

Hi Terry
Thanks for your reply.
These are two interesting links. However, neither directly addresses
the anomaly that started this thread 50 posts ago which can, perhaps,
be more simply stated as the lack of information/explanation WRT the
presence of TWO DISTINCT AND VERY DIFFERENT measures of CPU usage in
WinMe (at least).

I may post this exact question on the WinMe NG as there are some really
good people there who might want to comment.

which means it has been mystifying people for a long time
ME has a faulty cpu use algorithm ,they say, but I have seen the same on all
version of windows.
I don't have a clue _why_ I just know enough to recognise it.

You haven't gotten an answer because no-one feels like giving a course in
multitasking operating system
scheduling or can be sure what mirror they are looking at through the smoke.



Any task that has the cpu runs at full speed until it is done or uses up its
timeslice and time is given to another process, or a higher priority
interrupt
preempts it for a short time.
If nothing is waiting the cpu goes to the idle process, which usually does
very little. This is a good place for things to get done that are not
urgent.

The measure of time uses can be as reported by the system, or by scanning
all processes and taking the difference, which should be what the system
reports
but isn't always, or subtracting it from the proportion used in idle
process.

Now systems may have multiple processor cores , different generations
have different lookahead and cache which may fool the counter about
how much work has been done in a period, the measure of cpu time
vs things done in that time isn't linear. Now the count of clocks should be
accurate, but
it appears the system doesn't always count that, and isn't the same
on older processors anyway.
Some systems may be running power management, which can halt or
slow down the processor when it would otherwise be in-idle,
which may affect whatever method the logger app uses..
or any other number of ways I don't know or care to think too much about

Don't think of CPU use in terms of car tacho and fuel consumption, it
isn't measuring anything static anyway
think of 5% as 'currently wasting $950 of your $1000 investment'.

That seems reasonable as indeed the user doesn't require pin-point
accuracy in whatever indication of CPU usage he/she is using.

Unless you use a laptop..in which case power management should be builtin,
there isn't much point worrying too much about cpu use , just get a tool
you think is reliable, get a benchmark of resting and turning sluggish, call
that
0% and 100% and work with that.
Differences are what count.

This sounds even more reasonable and probably is what most people will
do (even if just subconciously).
Just to point out that I was never really "worried" about this (in the
sense of tearing out hair or having restless nights). But it did seem a
worthy target for intellectually motivated curiosity.

Your posts have cleared up a lot for me in that regard.
Many thanks for that.
Paul