GarlicSim blog

15th January 2011

Text

`caching.cache`: A caching decorator that understands arguments

GarlicSim module of the week

This is the first post in the GarlicSim module of the week series. This will be a series of blog posts in which I explore different modules from the garlicsim.general_misc package. These are modules which are not specific to computer simulations; they can be relevant to any kind of Python program, and you are welcome to import them from garlicsim.general_misc and use them in your projects.

Part one: caching.cache: A caching decorator that understands arguments

Part two: caching.CachedType: A metaclass for sharing instances

Part three: address_tools: More powerful replacements for eval and repr 

Part four: ContextManager and the manage_context method

Part five: cute_profile: Profile your Python code on the fly

caching.cache: A caching decorator that understands arguments

Some time ago I saw an example for a memoization decorator for Python functions. (I prefer the term “caching” over “memoization”, so I’ll use “caching” from now on.)

The idea for caching is very cool. You decorate your function with a caching decorator:

>>> @my_caching_decorator
... def f(x):
...     print('Calculating...')
...     return x**x # Some long expensive computation
>>> f(4)
Calculating...
256
>>> f(5)
Calculating...
3125
>>> f(5)
3125
>>> f(5)
3125

As you can see, after the first time we calculate f(5) the result gets saved to a cache and every time we’ll call f(5) Python will return the result from the cache instead of calculating it again. This prevents making redundant performance-expensive calculations.

So I wanted to use a caching decorator for a few functions in GarlicSim. But… they all annoyed me. I just couldn’t find a good caching module that I liked, and I asked a few experienced Python programmers whether they know any good caching decorator on PyPI. They didn’t. So I made my own, released it with GarlicSim 0.6, and now I’ll explain why it’s so good.

garlicsim.general_misc.caching.cache

The main thing that annoyed me about the various caching decorators out there is that they don’t understand function arguments. For example, calling f(3) is exactly the same as calling f(x=3). And for a more complicated function like this:

def g(a, b=2, **kwargs):
    return whatever

There can be many different ways to make the same call: g(1), g(1, 2), g(b=2, a=1) and even g(1, 2, **{}) are all equivalent. They give the exact same arguments, just in different ways. But most caching decorators out there can’t understand that. If you call g(1) and then g(1, 2), they will calculate the function again, because they don’t understand that it’s exactly the same call and they could use the cached result.

Enter caching.cache:

>>> from garlicsim.general_misc import caching
>>> @caching.cache()
... def g(a, b=2, **kwargs):
...     print('Calculating')
...     return (a, b, kwargs)
... 
>>> g(1)
Calculating
(1, 2, {})
>>> g(1, 2) # Look ma, no calculating:
(1, 2, {})
>>> g(b=2, a=1) # No calculating again:
(1, 2, {})
>>> g(1, 2, **{}) # No calculating here either:
(1, 2, {})
>>> g('something_else') # Now calculating for different arguments:
Calculating
('something_else', 2, {})

That’s the main cool thing about caching.cache; It has other interesting features that I will not go in depth into:

• It has a max_size argument which limits the size of the cache dict; old values get thrown away according to an LRU algorithm implemented using (my bundled version of) Python’s OrderedDict, specifically the new OrderedDict.move_to_end method.
• It stores arguments with sleekrefs. Sleekrefs are a more robust variation of weakrefs. This prevents memory leaks when using potentially-heavy arguments. More about sleekrefs in a future blog post.

• It preserves the original function’s signature using Michele Simionato’s excellent decorator module.

Source and tests for caching.cache

Source for caching.cache, well-documented. Here are the tests.

There is also a Python 3 version of caching.cache, and here are its tests. It’s available with the Python 3 fork of GarlicSim. Note that I haven’t tested it on Python 3’s new kinds of function signatures; I’ll be happy to get patches with tests and/or implementation of support for those.

Please give feedback!

I’ll be happy to get any opinions, critiques and code reviews on caching.cache!