Amazing Caching Proxy in Java

Use Case

Imagine you have some Java code that does lots and lots of computation. All the time intensive calculations is performed by the class SlowCalculator which implements the interface Calculator:

You notice that calculate() is often called with the same parameters which lead to the exact same result (SlowCalculator is stateless). This means it is possible to cache values so there’s no need to recompute. Using the generic CachingProxy™ described below, you can create a cached proxy for any class with just one single line of code:

That’s it, and the application is blazingly fast again.

UPDATE: Support for null values, transparently handles exceptions, better hash, nullpointer-bugfix.

UPDATE: Here is an article “Memoization in Java Using Dynamic Proxy Classes” that does (almost) exactly the same as this code.

How To Do This

All this sounds nice, but can you do this in java? Turns out you can and it is not that difficult either. The feature that makes it all possible is Dynamic Proxy. With it you can implement interfaces at runtime. You take an interface, create a proxy for it with Proxy.newProxyInstance(...), supply an InvocationHandler that implements the invoke() method, and you are done.

The code for the CachedProxy.create() method is this:

  1. First I create a HashMap that is the cache for the return values. I have written a class Args (omitted here) that is as the key to map from the method and the parameters to the cached output.
  2. A Proxy is created for the interface cl, with an InvocationHandler that does all the magic.
  3. The magic is actually very simple: If there is no cached result already available (line 25), perform the computation (line 26) and store the result in the map, then return the result.

Download

You can get the full code at my github repository:

Benchmarks

In my benchmark I can run about 8 million calls per secons via the cached proxy. That’s not too bad, given all the additional overhead with reflection and the HashMap.

Do you know of any way to improve this? Any ideas or suggestions are welcome!

10 Comments on "Amazing Caching Proxy in Java"

Notify of
avatar
ben
Guest

Improvements:

1. Use a referencemap
2. What happens if there is a large running process that returns null? 😛

Merry xmas

Martin Ankerl
Guest

Hi ben, good points! I have never used referencemap before. And if it returns null it always recalculates. I could use containsKey but then I need two lookups, one for contains and one to get the element. Merry christmas!

fede
Guest
public Object invoke(final Object proxy, final Method method, final Object[] args) throws Throwable { final Args input = new Args(method, args); Result result = argsToOutput.get(input); if (result==null) { result= new Result(method.invoke(code, args)); argsToOutput.put(input, result); } return result.getValue(); };
Jim White
Guest

This is already a widely known technique called memoization.

http://en.wikipedia.org/wiki/Memoization

Heh, just looked at the Java implementation link, and it is this same proxy method published five years ago.

http://www.onjava.com/pub/a/onjava/2003/08/20/memoization.html

Martin Ankerl
Guest

thanks for the link, Jim!

Anonymous
Guest

This is not thread safe (HashMap has to be synchronized).
That’ll make it potentially much slower in a multi-threaded application. ConcurrentHashMap might help.

– Anon

cheind
Guest

hi martin & merry christmas,

you might consider some features of the memcache library for ruby (i.e. expiration you’ll need sooner or later). Here’s an application quite the same as yours http://refactormycode.com/codes/609-caching-methods.

Joined github? Can anyone here my echooooooo? 🙂

Amir
Guest

Martin, are you still active on this thread? If so I can point you to some modifications that I made to improve this caching logic.

Amir
Guest

Hey Martin,

I had been working on something similar. We initially had been using @Cacheable that uses Spring and AOP to do some caching via ehcache. But as I made a project open source, I didn’t want to depend on Spring. So I started with some of your code but cleaned it up a lot. With the use of Guava, I was able to make the code a lot less. I also used Arrays.deepEqual() which works really well. If you want to check it out here it is

https://github.com/opower/jpile/blob/master/src/main/java/com/opower/persistence/jpile/reflection/CachedProxy.java

wpDiscuz