Friday, December 22, 2006

Reload modules in Python environment

Problem:
You create a module and run it using the python interpreter. Now you modify this module and try to run it again, surprise surprise, the new changes are not reflected.

What really happens:
The python interpreter loads modules as and when they are used in a script. Once these modules are loaded they are cached in memory for reuse later. Now if you modify a pre-loaded module and try to run it, the interpreter does not bother loading it again as it already has a cached copy.

Solution:
One technique that i have used effectively for (indirectly) reloading the modules is to delete the module object from the cache. This forces the interpreter to reload the module when it executes the script again and you can test your new changes easily.

How its done:
The interpreter caches the module objects in a PyDictionary (a map) called the sys.modules. This map stores the module objects in the following format:
key : the fully qualified class/module name
value : the actual module object

The basic trick here is to delete the required object from this map and your jobs done.

Code:

if "<the fully qualified class/module name>" in sys.modules.keys():
    del(sys.modules["
<the fully qualified class/module name>"]

line 1 : test if the module exists in the cache
line 2 : actually remove the module from the map based on its key

eg.
Say, you want remove the module "com.test.testfile" from the cache.

if "com.test.testfile" in sys.modules.keys():
    del(sys.modules["com.test.testfile"]

Put these lines of code after the import statements or before you make a reference to your changed module, this ensures that the module is reloaded before execution.

There may be better ways of reloading modules but i found this most effective when working with a system which contains a lot of scripts.

No comments: