Caching approach in API

[Anders Borum]

Hello!

I am developing an API that supports caching and would like your comments on
my implementation (and thoughts on the subject).

A set of Service classes have been designed, allowing the programmer to
access the functionality in the API. A Service could for instance allow the
programmering to retrieve Pages, Users or Modules (these are objects that
are cached in the API using their Guid as indexing key).

Each Service class inherits from a base CacheService class, which in turn
creates an internal cache (a synchronized Hashtable). For instance:

public class PageService : CacheService {}

Each Service is accessed from a Context object that exposes the different
Services. For instance:

Page page = Context.PageService.GetPage(..);

I am thinking about making a single cache available across all the Service
classes, instead of having them create their own instance internally. The
idea is to host the cache in the Context object, then making it available to
the Service classes.

The idea of having a single cache is interesting, as it allows for
monitoring, a single point of flushing and so on. However, I am a bit
worried that the synchronization on the Cache would slow things down -
compared to the distributed version, where each Service only locks its own
ressources when accessing the synchronized Cache.

Ideas and thoughts are greatly appreciated!

 

[Joerg Jooss]

Hello!

I am developing an API that supports caching and would like your
comments on my implementation (and thoughts on the subject).

A set of Service classes have been designed, allowing the programmer
to access the functionality in the API. A Service could for instance
allow the programmering to retrieve Pages, Users or Modules (these
are objects that are cached in the API using their Guid as indexing
key).

Each Service class inherits from a base CacheService class, which in
turn creates an internal cache (a synchronized Hashtable). For
instance:

public class PageService : CacheService {}

Each Service is accessed from a Context object that exposes the
different Services. For instance:

Page page = Context.PageService.GetPage(..);

I am thinking about making a single cache available across all the
Service classes, instead of having them create their own instance
internally. The idea is to host the cache in the Context object, then
making it available to the Service classes.

The idea of having a single cache is interesting, as it allows for
monitoring, a single point of flushing and so on. However, I am a bit
worried that the synchronization on the Cache would slow things down -
compared to the distributed version, where each Service only locks
its own ressources when accessing the synchronized Cache.

Ideas and thoughts are greatly appreciated!

IMO, the main issue is not even lock contention, but manageability. Runtime
environments that use garbage collection are quite susceptible to memory
leaks due to excessive caching. You didn't describe what expiration policy
your cache employs (LRU, aging, ...). I hope there is one, otherwise you've
almost certainly opened the flood gate...

Using a single cache makes most sense if your cached objects have a similar
life span and have similar object sizes. Otherwise, managing your cache
becomes mere guesswork.

Assume you have a LRU cache that is bounded by the number of objects
referenced (N). If your cached object's size varies a lot, it's hard to tell
how to set N, because N big objects can consume way more memory than N tiny
objects. If you use separate caches, you can tune their maximum sizes
individually. Depending on your application's profile, you may also find
that your expiration policy (e.g. LRU) favors certain objects. Thus,
"non-favored" objects are not really cached like you expect them to be.

The same applies for runtime monitoring. Ideally, you'd want to have a
performance counter for each cache that tells you the current numberof
objects cached. This information is the more useful, the more specialized
caches you have, meaning you can use the numbers monitored in system test or
production to further tune your system.

Cheers,

 

[Anders Borum]

Hello Joerg!

Thanks for the ideas and very interesting comments.

I'll get back to you later today with further information on the caching
needs.