On Lisp
slight diversion from State machines
For my undergraduate research project I tried to create some tools to generate three dimensional computer models of buildings (or whatever) from photographs. I used AutoLISP and let AutoCAD's graphics engine manage the user input and the display. My mentor was Mark Gross who among other things was an accomplished lisp hacker. I was hoping to teach myself Common Lisp after graduation, but was discouraged when Mark observed that languages, even programming languages, are grounded in a community. He said it would be hard to learn lisp in the absence of a lisp community. He was right. Before I finally admitted defeat I had bought CLTL2, Mark's suggestion, and On Lisp which I found for myself in the Fall of 1994.
Both books have collected dust on my bookshelf in the many years since. Every time I've moved since then I've considered getting rid of them, but each time I've felt my stomach uncomfortably tighten up. I couldn't do it. For just less than a decade I have been moving them and their dust from one place to the next.
My recent exploration of PyBlosxom has rekindled my interest in lisp because of the lambdas in its callback chain. I have been wanting to reconnect with what I used to know about functional programming, but had still not made the time. Then I blog-hopped my way to Paul Graham's article Hackers and Painters.
There on his site I found his page about On Lisp . The reviews quoted there are glowing. I went to Amazon to get other opinions which were even more glowing. I'm glad my intuition wouldn't let me get rid of it. In the past month I've finally cracked it open and have been slowly ploughing my way through it.
It's a fairly short book, but quite dense. I'm sure Paul would consider that flattery because he suggests that code should have just those qualities, short but dense. The reading is slow going. Most of the prose is brief explanations of what a code example is doing, or why you might want a utility to do that. But understanding the code and connecting it with the prose takes time and attention. And it's been a long time since I parsed parentheses.
Macros, creating languages, and other meta-programming. The discussion of closures and functions that return functions are quite illuminating. I've understood the what of closures for a long time. The Camel book explains them pretty clearly. But only now do I think I understand the why bother of closures. They're akin to lightweight objects with only one method. The variables closed within are encapsulated properties of the object. Lightweight and encapsulation are Good Things. Paul emphasizes how lisp invites the programmer to extend the language into their problem domain. Functions and macros that generate functions and macros are powerful tools. Another recent bloghopping session led me to Guy Steele's OOPSLA '98 inspired presentation on Growing a Language. Mark said something similar at the Sundance Lab in the early 90's: "sooner or later all programs want to become their own language."
All those lisp hackers have been pointing me in this direction, but only with some experience under my belt can I understand the wisdom. AutoLISP was trying to guide me in that direction too. Toward the end of my time working with Kim Veltman, the animation tools I had developed were becoming a language. I wasn't doing that in any intentional way. I was just gradually removing some of the repetition in creating the animations. As I sharpened my tools, there began to emerge an animation scripting language.
While developing Envoy, we were conscious of the force pulling towards a language. In fact we resisted that force, trying to expressly not write a language. But the requirements were really begging for it. Envoy administrators would create macros (think Excel macros, not lisp macros) to munge CAD data. There were pre-processing macros, format conversion macros, post-processing macros, and delivery macros. (I thought we needed a composite pattern -- macros of macros -- but was accused of more pattern zealotry). In spite of our efforts to avoid it we joked sarcastically that we had created a not-exactly-visual, distributed batch file system, about as impoverished a language as DOS. In spite of our sarcasm, it was actually pretty fun to build. Good lessons in concurrency and messaging among other things.
Is all programming meta-programming, just that some things are more obviously meta- than others? When test-firsting I write programs to test programs. It's not quite like functions that return functions, but there's something meta- about it. Test-driving often leads quickly to interfaces each of which is an API -- a micro language for communicating with an object. That's what Guy Steele's presentation was getting at, that good programming languages encourage growth in the language. The Ace project web site remarks that we don't write ones and zeros anymore, and very few write assembly. It boasts of being the next logical step in that progression of abstracting away another layer of implementation details. When Grady Booch spoke at DJUG he called it increasing layers of abstraction.
On Lisp's theme of bottom-up programming is a metaphor for the profession as a whole. Programmers grow their tools from the root elements of lisp toward a domain specific language at the same time that the system is being decomposed from high level requirements into appropriately digestible chunks. Similarly, the programming's progression has been inside-out recursion. Instead of working in from the outer lists down to the atoms, those who went before wrote in atoms and in each decade those elements have been wrapped it in a higher level list.
Heh heh heh. I'm not even half-way through this book and I've already connected it to the programming profession as a whole. Consider that another strong recommendation. :)