thinair Boulder, Colorado. elevation 5400 feet.

Playing Nice - what Perl's got that LISP and Smalltalk lack

A long while back, David observed two features of Perl which have made it more successful than LISP or Smalltalk. (Defining "successful" is left as an exercise for the reader. :-) There's always been only one Perl; differing versions, sure, but you have never had to spend any time choosing whose Perl interpreter to use. More importantly, Perl began with the assumption that it would not be the only tool in your toolbox: it set out from the beginning to play well with others. By contrast, there have always been many flavors of LISP and Smalltalk any of which have tried to be pretty much everything you ever need.

Painting in broad strokes, David says Smalltalk's weakness is "at the boundaries:" when you want to try to do some typical unix system maintenance, or interfacing with underlying C libraries, or something similar. As long as you're staying within the Smalltalk environment, it completely rocks. But it's definitely painful if you try to reach outside. And it's especially painful if you want your code to work with different Smalltalks. What Perl got right was making it completely painless to integrate with its environment.

In some sense LISP wants to be on a LISP Machine and Smalltalk wants to be in its virtual machine, whereas Perl wants to go out and play with the other kids. The former languages are introverted and Perl is extroverted.

I'm focusing on LISP and Smalltalk and Perl 'cos that's where David started, but I think the notion generalizes. Java clearly follows in the LISP and Smalltalk tradition. It's easier to reach out to a different machine than it is to reach into the OS of computer hosting your JVM. ('Course that begs the question "why is Java more successful than LISP and Smalltalk?") Python and Ruby and TCL follow Perl's example for integrating well with the host environment. Along these lines, it seems worth applauding Groovy for having gotten this right, at least within the confines of the JVM. Instead of re-inventing a host of libraries, Groovy aims to just make it easier to use the existing plethora of Java APIs. Whereas, for example, Jython is inherently stuck playing catch up with C-Python because it takes time to port the standard libraries, Groovy will always just advance with the Java platform.

At bivio our entire business is written in Perl. The system maintenance, release process, unit tests, acceptance tests, mail processing, backup procedures: it's all Perl. We use RPMs for package management, but even there we generate the .spec files and kick off the builds and releases with perl. bOP is pretty monolithic. There's a lot of great stuff in there, but we're not playing along with many of the popular CPAN libraries. Within the development world of bivio, bOP is our universe, not unlike those alternate universes of LISP and Smalltalk. Even though perl plays well with others, bOP really doesn't, it's almost a world unto itself.

Having simmered these thoughts for probably over a year I feel like I have some insight to how Smalltalk and LISP got where they are. Even if I'm really only a tourist in those worlds, I think I see how they evolved into introverted and, in some sense, self-centered worlds. If you'll forgive the immodesty, we've got a small, but incredible team of programmers. [1] Where we've rolled our own code instead of using something from CPAN there have been good reasons to do so. We get a lot of efficiencies because our code is tuned to our way of solving problems. We really get an amazing amount of re-use out of our framework. And yet, it seems like we've fallen into the same mode as LISP and Smalltalk in that we live in our own world, building and expanding the reach of our bucket of tools, seldom reaching out to the rest of the world. We accomplish some amazing things, but in an isolated way.

Teach me to write at one in the morning... I didn't actually say anything about how I think it happened! Bear with me on a bit of tangent. There's been one particularly notable change in my programming since working with bivio. Rob and Paul have shown me that it's all just a simple matter of programming. Whereas I used to see specialized programming domains, I now just see code. For example I used to buy into the story that a business needs developers, and a QA team, DBAs, sysadmins. I used to think that people programming embedded systems were doing something fundamentally different than web developers which was different again from operating systems and distributed systems and device drivers. But all of it is ultimately just code. This understanding has given me a great deal more courage to tackle problems that I used to think were beyond my reach. In fact, I could describe my journey along the programming career path as one of passing through veil after veil obscuring something I thought I couldn't handle. I no longer really see programming obstacles and boundaries. Obviously there are many layers of interfaces between different components, but it feels a bit like the scene in The Matrix when Neo starts to see everything as raining lines of code. Suddenly the entire world is in reach.

I've met a few people who were Smalltalk or LISP programmers. Those I've met personally have been some of the best programmers I've met. I'm interested in LISP and Smalltalk in large part because there are programmers I respect who prefer those languages. Similarly, the guys at bivio are amazing programmers. What many of these people have in common is a kind of confidence that comes from knowing that its all just code and you get to choose for yourself which parts you want to change. Paul doesn't seem to get excited about any technology. He's got a detached confidence about his skill as a developer. The important questions for him are about business problems. He could program anything he wanted, but a real business problem brings focus. His super power is being able to sustain a laser focus on the business problem at hand.

bOP is its own world in part because Rob and Paul haven't needed to play well with others in order to solve their business problems. Where other developers might have needed the help from CPAN libraries, Rob and Paul can probably write their own faster than they can learn the quirks of the API of someone else's code. It's not that they program in a vacuum. For a long time we were using Text::CSV as a simple example. But a few months back Paul finally got fed up with it's limitations and just rolled his own that works the way he wants to work and integrates seamlessly with our other infrastructure. Frankly, it's been a huge improvement. But it's also a glimpse into the process that leads to this kind of self-centered tool building. Great programmers can and do roll their own tools. For serious LISP hackers, the fact that there are many different LISP platforms is an almost irrelevant complaint. For them it's a trivial investment to build up the compatibility layers they need to work across platforms. An accumulation of little decisions like that lead naturally to fragmentation and in some sense isolation.

But David is right that playing well with others is important. For all the other virtues of LISP and Smalltalk, Perl does the the play-well-with-others thing right.

[1] As long as I'm being immodest, I might as well be specific. We are five programmers and one customer support person actively developing and maintaining eleven different software systems of varying complexity.