Embedding a Python Shell in a Python Script

I am huge of advocate of command-line programs and domain specific languages. Back when I worked in statistical genetics, there were many fine programs that ran this way, allowing for easily repeatable analysis*, and easy scripting. Python has some great tools for creating programs driven by mini-languages (such as the cmd module), but they seem to date from a kinder, gentler time when people took what they could get for documentation, and they liked it. Finding simple examples of how they work is tough**. I’m sure in some future post, I’ll tackle this in more depth, but for now I want to focus on a simpler problem: embedding python shells into python scripts.

Should be trivial, right? IDLE exists, and python comes with a bundled interpreter. Searching for help here fails because “embedding a python shell” calls up documentation on how to get a shell in C environments, which is not what I mean at all. How often has it happened that there is some complex script that one wants to introspect, partway through, maybe after some data structures are created or loaded? A typical solution is something like:

if flag:
    import pdb
    pdb.set_trace()

This is fine and dandy, except that it’s not always “debugging” that one wants to do. Sometimes one wants to explore data, or gods forbid, enter new code as an experiment.

I read Rick Muller’s recipe about how to embed a code.InteractiveConsole into a python script, and I thought I could do a little better. The following snippet shows how a --shell command line flag (read using optparse) drops the user into a shell, using IPython if it’s available, or falling back on code.***

#!/usr/bin/python

## filename:  cmdDriven.py
import sys

# some imports, data, and like, so we have something to explore
Info = dict(author="Gregg Lind", blog="Write-Only")
data1 = set(range(100))

def tickle(foo, willing=False):
    if willing:  return "%s just got tickled" %foo
    else: return "I don't think %s would like that so well" % foo

if __name__ == "__main__":
    from optparse import OptionParser
    parser = OptionParser()
    parser.add_option('--shell', action="store_true", default=False, help='get an (ipython) shell to explore the data before output')
    (options, args) = parser.parse_args()

    if options.shell:
        try:
            from IPython.Shell import IPShellEmbed
            ipshell = IPShellEmbed(argv=[''],banner="hello!",exit_msg="Goodbye")
            ipshell()
        except ImportError:
            import code
            # calling this with globals ensures we can see the environment
            shell = code.InteractiveConsole(globals())
            shell.interact()
    else:
        sys.exit("No shell requested")

One of the nice benefits to this coding pattern is that we can do bash scripting using this quite easily:

$ python cmdDriven.py --shell << JUNK
> print Info
> print tickle("bob")
> JUNK
Python 2.5.1 (r251:54863, May 18 2007, 16:56:43)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> {'blog': 'Write-Only', 'author': 'Gregg Lind'}
>>> I don't think bob would like that so well

Footnotes

* another favorite topic! Cf: Literate programming, Sweave for R-project.

** for cmd: http://www.eskimo.com/~jet/python/examples/cmd/index.html

*** Edit (4/28/09), here is my function I tend to use for this in projects these days:

def prompt(vars, message ):
    #prompt_message = "Welcome!  Useful: G is the graph, DB, C"
    prompt_message = message
    try:
        from IPython.Shell import IPShellEmbed
        ipshell = IPShellEmbed(argv=[''],banner=prompt_message,exit_msg="Goodbye")
        return  ipshell
    except ImportError:
        ## this doesn't quite work right, in that it doesn't go to the right env
        ## so we just fail.
        import code
        import rlcompleter
        import readline
        readline.parse_and_bind("tab: complete")
        # calling this with globals ensures we can see the environment
        print prompt_message
        shell = code.InteractiveConsole(vars)
        return shell.interact

p = prompt(locals(),'')
p()

Note: the ipy interface has been updated in version 0.11 (July 2011) https://writeonly.wordpress.com/2011/08/31/updated-bash-ipython-alias-that-works-for-0-10-and-0-11/ and this code might no longer work.

Advertisements

---