Enhanced Interactive Python with IPython

by Jeremy Jones
01/27/2005

Python is a multipurpose programming language: it is object oriented, is dynamic, can accomplish much in few lines of code, is syntactically clean and elegant, "fits the brain well," and is an excellent language for programmers of all ages and skill levels. These characteristics have contributed to building a loyal, knowledgeable, and vibrant community. In addition to the language syntax and the community, a benefit of using Python is its interactive interpreter. The standard interactive interpreter that ships with Python allows a user to type Python code at a command prompt and have it execute while the user watches, and allows various levels of introspection into the source code. (Python's introspection is its ability to inspect code objects at runtime; determine information about them such as their type, attributes, and defined methods; and make that information available to the user.) The Python interactive interpreter is useful for testing pieces of code, such as algorithms or getting acquainted with a particular library, or for accomplishing specific tasks, such as creating 500 directories named dir_000 through dir_499.

The standard Python interpreter is unquestionably a highly valued item in the toolbox of many Python programmers. However, the IPython developers have taken the fundamental ideas found in the standard interpreter and improved upon them so considerably that they have created a tool of amazing utility. The project's web site describes it as "an enhanced Interactive Python shell." This description is far too modest. IPython is elegant, thoughtfully crafted, and extremely extensible. Some of the thoughtfully crafted features that make IPython so useful are tab completion, object introspection, a powerful history mechanism, inline editing of source code, integration with the Python debugger, the %run mechanism, macros, the ability to create multiple profiles, and system shell access.

IPython began life with Fernando Pérez attempting to unify his own work, IPP by Janko Hauser, and LazyPython by Nathan Gray. Pérez cites Stephen Figgin's article on IPP and LazyPython as inspiration. Fernando is currently the sole developer for the IPython project.

While Fernando likes to emphasize that IPython is "meant to be a highly configurable tool for building problem-specific interactive environments," this article will focus almost exclusively on its ability to replace the Python standard interactive interpreter.

Installing

For this article, I assume you already have Python 2.2 or higher installed. I created and tested all examples using IPython 0.6.6, Python 2.4, and Fedora Core 3 Linux.

In order to run IPython, you must first download and install it. At the time of this writing, IPython 0.6.6 was the latest. Download an appropriate version and format for your system. (For example, because I installed Python 2.4 from source, IPython-0.6.6.tar.gz was best for me.) Windows users should probably download the .zip file and unzip it. In another week or so, another alternative for Windows users is to download the Python Enthought Edition, which will contain IPython as one of its default shells. If you installed an RPM, it should be as simple as rpm -Uvh <filename>. Otherwise, unpack the compressed file and change into the resultant directory, which should contain a setup.py file. Installation should be as easy as typing (as root):

python setup.py install

If on a Unix system, this will put an executable file named ipython in the same directory as your python executable. Installation on Linux systems will put documentation in /usr/share/doc/ipython/<VERSION>. Running ipython for the first time will create a configuration directory at $HOME/.ipython containing several configuration files (ipythonrc, ipythonrc-math, ipythonrc-numeric, ipythonrc-physics, ipythonrc-pysh, ipythonrc-scipy, and ipythonrc-tutorial) for different profiles:

[jjones@cerberus ~]$ /usr/local/python24/bin/ipython
**********************************************************************
Welcome to IPython. I will try to create a personal configuration directory
where you can customize many aspects of IPython's functionality in:

/home/jjones/.ipython

Successful installation!

Please read the sections 'Initial Configuration' and 'Quick Tips' in the
IPython manual (there are both HTML and PDF versions supplied with the
distribution) to make sure that your system environment is properly configured
to take advantage of IPython's features.
Please press <RETURN> to start IPython.

In summary, for installation:

Download the appropriate package.
Uncompress or untar it.
Execute python setup.py install.

As mentioned above, in order to start IPython, execute the installed ipython executable. That will yield a prompt that may appear unfamiliar to those who already work with the standard Python shell prompt:

[jjones@cerberus ~]$ /usr/local/python24/bin/ipython
Python 2.4 (#2, Nov 30 2004, 09:22:54)
Type "copyright", "credits" or "license" for more information.

IPython 0.6.6 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]:

To exit IPython (from Linux), type Ctrl-D (which will request confirmation before editing) or either Exit or Quit (note the case), both of which exit without prompting.

Features

Magic

IPython has several "magic" keywords:

%Exit, %Pprint, %Quit, %alias, %autocall, %autoindent, %automagic,
%bookmark, %cd, %color_info, %colors, %config, %dhist, %dirs, %ed,
%edit, %env, %hist, %logoff, %logon, %logstart, %logstate, %lsmagic,
%macro, %magic, %p, %page, %pdb, %pdef, %pdoc, %pfile, %pinfo, %popd,
%profile, %prun, %psource, %pushd, %pwd, %r, %rehash, %rehashx, %reset,
%run, %runlog, %save, %sc, %sx, %system_verbose, %unalias, %who,
%who_ls, %whos, %xmode

IPython works by checking the command issued to it against its list of magic keywords. If the command is a magic keyword, IPython knows what to do with it. If it's not a magic keyword, it lets Python figure out to do with it. If automagic is on (the default), you don't need to prepend the % symbol to magic keywords. If automagic is off, you need to prepend them. Type magic at the shell prompt to see a list of all magic keywords and some words of help with their usage. Good documentation is a key feature of any piece of software, and between the IPython user manual and the inline documentation (%magic), IPython certainly does not lack in this area.

Tab Completion

A very powerful feature in IPython is tab completion. If you are familiar with Python, you may be thinking, The standard Python interactive shell has tab completion. All you have to do is:

[jjones@cerberus ~]$ /usr/local/python24/bin/python
Python 2.4 (#2, Nov 30 2004, 09:22:54)
[GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rlcompleter, readline
>>> readline.parse_and_bind('tab: complete')
>>>

That's correct. Both the standard Python interpreter and IPython support "regular" completion as well as menu completion. To use completion, type a pattern you would like the shell to match, followed by the Tab key. When in "regular" completion mode (the default), the Tab key will:

expand the pattern to as many characters as it can while still honoring the original match criteria
list all items matching that pattern

For example:

In [1]: import os

In [2]: os.po<tab>
os.popen   os.popen2  os.popen3  os.popen4

In [2]: os.popen

Typing os.po followed by the Tab key expands os.po out to os.popen (as shown at the In [2]: prompt) and displays all the modules, classes, and functions under os that begin with po, which happen to be popen, popen2, popen3, and popen4.

Menu completion has a slightly different behavior from regular tab completion. In order to change from the default tab completion behavior to menu completion, edit the ipythonrc config file located in $HOME/.ipython/. Comment out:

readline_parse_and_bind tab: complete

and uncomment:

readline_parse_and_bind tab: menu-complete

Rather than display a list of matches for the current command, menu-complete cycles through each matching item in the list with each press of the Tab key. For example:

In [1]: import os

In [2]: os.po<tab>

will result in:

In [3]: os.popen

Each subsequent press of the Tab key will cycle through to the next matching item in the remaining list of popen2, popen3, popen4, and back to po. To see a list of all items matching the specified pattern while in menu-complete mode, hit Ctrl-L:

In [2]: os.po<Control-l>
os.popen   os.popen2  os.popen3  os.popen4

In [2]: os.po

Introspection

Python has several built-in functions that facilitate introspection. IPython has access to all of the standard Python functions as well as some extras built into the shell. A typical introspective technique with the standard Python shell is using the built-in dir() function:

>>> import SimpleXMLRPCServer
>>> dir(SimpleXMLRPCServer.SimpleXMLRPCServer)
['__doc__', '__init__', '__module__', '_dispatch',
'_marshaled_dispatch', 'address_family', 'allow_reuse_address',
'close_request', 'fileno', 'finish_request', 'get_request',
'handle_error', 'handle_request', 'process_request',
'register_function', 'register_instance',
'register_introspection_functions', 'register_multicall_functions',
'request_queue_size', 'serve_forever', 'server_activate', 'server_bind',
'server_close', 'socket_type', 'system_listMethods',
'system_methodHelp', 'system_methodSignature', 'system_multicall',
'verify_request']

Well, that's nice. Very helpful, in fact. I have done just that for years now and have been very pleased with it. This is a nice list of all of SimpleXMLRPCServer.SimpleXMLRPCServer's methods, classes, modules, and so on. Because dir() is a built-in function, IPython can do the same thing, but its ? and ?? operators are more powerful:

In [1]: import SimpleXMLRPCServer

In [2]: ? SimpleXMLRPCServer.SimpleXMLRPCServer
Type:           classobj
String Form:    SimpleXMLRPCServer.SimpleXMLRPCServer
Namespace:      Interactive
File:           /usr/local/python24/lib/python2.4/SimpleXMLRPCServer.py
Docstring:
    Simple XML-RPC server.

    Simple XML-RPC server that allows functions and a single instance
    to be installed to handle requests. The default implementation
    attempts to dispatch XML-RPC calls to the functions or instance
    installed in the server. Override the _dispatch method inherited
    from SimpleXMLRPCDispatcher to change this behavior.

Constructor information:
Definition:     SimpleXMLRPCServer.SimpleXMLRPCServer(self, addr, 
                     requestHandler=<class
				SimpleXMLRPCServer.SimpleXMLRPCRequestHandler at 
				     0xf6b2c5cc>, logRequests=1)

If long strings are present, the ? operator will trim them. The ?? will not trim long strings, and it will display syntax highlighted source code if the source is available.

Pages: 1, 2