Jupyter / IPython: After editing a module, changes are not effective without kernel restart

A powerful feature of Jupyter / IPython (indeed, of Python itself) is that you can interact with your data from the (I)Python command line. In order to provide this functionality, the modules which hold your data remain alive between invocations. Therefore once a module is imported, it stays imported, and re-importing it has no effect at all.

However this great feature can have puzzling consequences, especially to programmers who are more used to compiled languages with make-style dependency change detection. In particular, if you change a python module which has already been imported within ipython, and then re-import that module (e.g. by using ipython to run a program which uses it), python will think "I've already imported this module, no need to read that file again", so your changes will not be effective. (Note that this does not apply to your main program file, which IPython runs directly, rather than importing, so that changes are always effective once they are saved.)

There are several ways to work around this issue.

1) The simplest and most certain is to restart the ipython kernel after changing an imported module. But this has downsides as well, notably losing the data which exists in your ipython namespace and in any other imported modules.

2) For simple cases, you can use python's reload function. (This function is builtin in Python 2. Starting with Python 3.4, it is imported from standard module "importlib"). In many cases, this suffices after editing a module. It is described briefly here:  http://dpb.bitbucket.org/reloading-modified-code-when-using-the-ipython-interactive-shell.html and elsewhere online in more detail.

3) For more complex cases, where reloading the module that you have edited also requires that its dependent/imported modules be reloaded (e.g. because they must be initialized as part of your edited module's initialization), ipython's autoreload extension can be useful. See  http://ipython.readthedocs.org/en/stable/config/extensions/autoreload.html including the important caveats at the bottom of the page.

 

Have more questions? Submit a request

Comments

  • Avatar
    Phillip Feldman

    Clear and consise.  Thanks!

  • Avatar
    Arika Ligmann-Zielinska

    Thanks a ton!

  • Avatar
    Phillip Feldman

    If a module is edited using the Canopy editor and the changes are saved, it seems as though Canopy should be smart enough to figure out that that module needs to be reloaded. It would at least be great if there were an option to make this behavior the default.

  • Avatar
    Jonathan March

    Phillip, this is certainly an appealing idea, but the details would be tricky. First, recall that the Canopy GUI's Python is quite strictly an IPython QTConsole. All interactions with it, e.g. Run, cd, edit, and the GUI debugger, occur through the IPython magics and/or API. So this would need to be an IPython enhancement, not just a Canopy one. Second, it's not clear exactly what implementation would be most useful. Remember that when you reload a module, it loses all its global state. Suppose main imports A imports B imports C, and you edit B. There's a strong argument to be made that unless all the others are reloaded too, that you could end up with inconsistent object state between modules. But there's also a strong argument to be made that this would be intrusive and surprising. And what about module D, which also imports C and is using its global objects? IPython's autoreload tries to address these issues, but is a bit fragile for reasons such as the ones I just described.

  • Avatar
    Thej Kiran Ravichandran

    Wow. Man. Thanks a ton. Works like a charm... the thrid option. Was looking for this a few days back!

Powered by Zendesk