As mentioned in my introduction to this series, learning to program is a series of lurching steps forward, sideways, and back, leading (if all goes well) to a better holistic understanding of the process.
In many rationalist disciplines, there are learning exercises that require grokking the whole and the parts in an iterative process, until a fullness is achieved. As learning occurs, one filters out more or the irrelevant detail. Think of learning to drive, or to speak a new language, or music theory. Without knowing “the point”, the big picture, the details are just noise, and tedia. Most programming (cf: math, stats) books and courses stay focused on the details far too long. What’s missing are activities that combine the micro and macro views fluidly. Code analysis fits the bill nicely.
One of the unsung marvels of the open-source movement is that it exposes mountains of good and bad code for all to see, of every imaginable pattern and anti-pattern. Some of the code even works, despite of (or because of) its ugliness.
Learning to read others’ code is an under-explored pedagogy tool. It’s well and good to read the sort of well-commented, rational, toy examples that one finds in books, but it’s quite another thing entirely to learn to read other peoples’ eccentric, human-created code, warts and all.
Python has some nice advantages for this sort of exercise*. It has an interactive interpreter, and even the worst python code is still pretty easy to pick apart, for most reasonable length examples, using common modules. Give me Twisted, and I can give you write-only code, but that’s my failing, not the tool.
So, as my service / penance for all the bad code I’ve unleashed, I’ve decided to comment and describe the weird mishmash of ideas and patterns existing in a first generation, but working, application I wrote.
More people should know how to program*. More resources are available than ever to help one learn. Pc’s are ubiquitous, and the open source movement and the internet make powerful languages of every flavour (C, Python, Haskell, R…) and difficulty easily available, along with extensive documentation. It should be a cakewalk, right? So why aren’t there more programmers?
I am huge of advocate of command-line programs and domain specific languages. Back when I worked in statistical genetics, there were many fine programs that ran this way, allowing for easily repeatable analysis*, and easy scripting. Python has some great tools for creating programs driven by mini-languages (such as the
cmd module), but they seem to date from a kinder, gentler time when people took what they could get for documentation, and they liked it. Finding simple examples of how they work is tough**. I’m sure in some future post, I’ll tackle this in more depth, but for now I want to focus on a simpler problem: embedding python shells into python scripts.
Should be trivial, right? IDLE exists, and python comes with a bundled interpreter. Searching for help here fails because “embedding a python shell” calls up documentation on how to get a shell in C environments, which is not what I mean at all. How often has it happened that there is some complex script that one wants to introspect, partway through, maybe after some data structures are created or loaded? A typical solution is something like:
if flag: import pdb pdb.set_trace()
This is fine and dandy, except that it’s not always “debugging” that one wants to do. Sometimes one wants to explore data, or gods forbid, enter new code as an experiment.
I read Rick Muller’s recipe about how to embed a
code.InteractiveConsole into a python script, and I thought I could do a little better. The following snippet shows how a
--shell command line flag (read using
optparse) drops the user into a shell, using IPython if it’s available, or falling back on