Re: processor cache and performance

ChadA via .NET 247 wrote:
> With today's fast processors, it seems performance optimizations have as much
> to do with good memory usage as with efficient algorithms. Just one memory
> "page miss" can consume many processor cycles. So, my question is how does
> the processor cache actually work, and how can I tune my app to effectively
> use it?

From a .NET app you surely won't have control of where objects get placed in
physical memory. So the short answer is that it is a waste of time trying to
optimize things at this level, AFAIK. Worry about your application's algorithms
instead if performance is actually a problem.

That said - if what you are really interested in is how various levels of memory
management work and interact from onboard processor physical memory caches up
through translation buffers, page frames, virtual memory translation, paging
files ,etc. - well, this is all fascinating and engrossing stuff indeed (to some
of us, YMMV )- but it's computer architecture questions you are into here,
far below the level of application development.

> I know it depends on processor cache size, but can multiple memory pages be
> in the processor cache at one time?

Depending on what you mean by "pages", yes. But are you asking about the paging
file and its corresponding "page frames" in physical memory (which are most
likely at the granularity of the virtual memory page size) or about the
processor's physical memory cache, where the granularity is typically a few words?

The best performance obviously comes
> when used memory is adjacent or has good locality of reference. Therefore,
> if all objects used by an algorithm are created about the same time, they
> will be nearby in memory.

Really there is nothing "obvious" about it, despite what intuition might tell
you. It depends on frequency of collisions between the cache tags, which you
have no control over. Your "therefore" is similarly an over-simplification - it
doesn't really follow at all in the way that you might think - but you will need
to study some details of memory management techniques to appreciate this.

However, what about new objects that must interact
> with very old objects? If the old objects are adjacent with all old objects,
> and the new objects are adjacent to all new objects, can the processor cache
> load BOTH the old and the new memory pages? Or will I constantly be
> switching back and forth?

All your questions are excellent and indicate that you actually want to think
about how computers work (how rare!) and about how the various levels of
hardware and software interact to create something as overall complex as Windows
or .Net. It is possible to do this and still maintain an overall perspective as
a programmer rather than a hardware engineer, believe it or not.

The best book I know on this subject at the level that you are asking is
"Structured Computer Architecture", by Andrew Tannenbaum. It's quite old by now
and may seem dated but it is still the most lucid explanation I have ever seen
of how a computer system is built up from transistors and gates through ALUs
physical memory, microocode, and eventually virtual memory, paging files and the
operating system.

Regards,
-rick-