Once you learn lisp, you almost immediately want to convert ten friends to the church of lisp. But there's a fundamental problem. You've learned things about programming that your friends haven't. You will use vocabulary and metaphors and examples which will have absolutely no meaning whatsoever to your intended converts. This is hugely frustrating and leads most people to just give up the evangelism (or at least back off a little ;-). You either wallow in the frustration or re-adjust your expectations for a longer-term conversion. Your friends aren't going to get it until they walk down the path to lisp on their own.
defmacro has made a valiant attempt to explain lisp using things you already know. If you're not already a fan of lisp, go read The Nature of Lisp. Hopefully you'll see the light. :-)
Thanks to James Duncan Davidson for pointing to this great link.
Commenting on the content itself, it was a thing of inspiration to use XML and Ant in particular for this bootstrapping. But I can't resist the need to rehash my most recent complaint about Java. It's a code smell that Duncan had to invent another language in order to build tomcat. Java isn't flexible enough on its own to let you build your java apps. Sure the parser is in Java, but the build language is still Ant or Maven.
We generally bill $125/hour. That income gets divided as follows:
* 65% to the developer
* 10% to whomever brought in the client
* 10% to whomever is managing the contract
* 15% to the house
It has been rough coming from a good-salary-with-benefits culture to what I usually call a commission culture. It is a different way of life, professionally. When I was working on salary, the only time I really needed to think about my income was in the salary negotiation, or when asking for a raise. Once I was past that, my paycheck was directly deposited and I would pay my bills a couple times a month. The day-to-day work at the office was stressful for all the reasons that make Dilbert funny: political maneuverings, fights between marketing and development and QA, schedule and feature and testing insanity and whatnot. By contrast, there are three day-to-day stresses at bivio. Am I getting the features out in a timely manner for our customers? Have we lined up the next pile of work? Am I billing enough hours this month to cover my bills next month? That last question is terrifying. "Am I providing for myself and my family?" has been a daily source of stress, sometimes even hourly. Not an easy transition. For most of 2005 I was regularly worrying that I wouldn't be able to hack it as a software businessman. Fear of failure loomed.
I kept reminding myself about the experience with Spatial/PlanetCAD. I had been paid a big, fat salary with benefits and responsibilities with a medium-sized, well established business. But there were major corporate events almost every quarter after I got hired. Just months after my first day on the job, the company sold off its main business asset to finance a dot com play (in November of 2000 -- great timing, huh?), and then we went through round after round of layoffs. Two years later my number finally came up, about a week after I got back from my honeymoon. That's the short version of my dot-com-crash-and-burn story. Notice that I did not get a job at a start-up. Spatial was 15 years old when I was hired. Enron is an even bigger example of every appearance of big and established and growing. Illusions of stability.
Even if it was terrifying facing the daily reality of billing enough hours at bivio, at least I knew exactly where I stood. There are no board room maneuvers and there's no rumor mill. We practice open book management. We all know exactly where the company stands. It is no secret where the money comes from nor where it goes. At one level it's scary as hell, but it's also completely real. No illusions. If the company is struggling, we all feel it directly. If we're doing well we feel that too. I've learned a bit about reading balance sheets and income statements and I'm starting to understand what they say about where we stand financially. I've learned a lot about cash flow and exactly how critical it is that we deliver value to our customers. I've always had a strong belief in customer service, but there's a profoundly different feeling when you can connect the dots between the check the customer writes and your own paycheck.
Even with this awareness, the culture shift has been difficult. One uncomfortable conversation with my Mom stands out in particular. Probably last Thanksgiving she commented that its often hard to take the risk of changing jobs, that it's easier to play it safe and stay with what you know. At the time I had been worrying out loud about whether I could hack it financially. But somehow Mom's comment about playing it safe really struck a chord, but not the way she intended. The idea of getting a traditional job with a salary and benefits felt much more like the play-it-safe path than sticking with bivio. It contradicted my story about Illusions of stability, but rang true nonetheless. The more courageous thing to do was stick with bivio.
This year has been a much different experience than last year -- I'm finally getting the hang of it. I've had month on month improvements in the number of hours I've billed. And unlike in the salaried jobs, when I really apply myself (like I did last month), it shows up in my paycheck. That's also real. The company does better -- bivio gets the 15% cut of a bigger pie. But I get almost immediate feedback about my effectiveness. Tight feedback loops allow you to adjust more quickly to the feedback. In that way, this arrangement is so much better than an end-of-year bonus. It's money in my pocket right now that lets me make decisions about what I do this month.
The conversation at our regular Friday lunch last week inspired this post. We had a guest to whom we explained how we work. I'm the brunt of a running joke who's punch line is "get back to work, Eric". At lunch Rob delivered the punch line. Everyone laughed. A few moments earlier Rob had just mentioned that he really doesn't want to be in the business of telling people what to do. He just wants to enjoy going to work. Think about that situation for a moment. I'll bet the image it has painted in your mind is almost completely opposite of what actually happened. If you work in a typical salaried position, you're probably imagining an awkward moment of silence around the table, or some knowing glances between co-workers. Eric just got publicly scolded by the boss in front of everyone. But that's not at all how it happened. Instead it was a scene of a playful joke among friends. This running joke is fundamentally about a bunch of guys who care about the fact that I spent most of last year worrying about my billable hours. They want me to succeed, to be able to make this work. I may be the brunt of the joke, but it's a kind of camraderie and encouragement that makes it really great to work here.
Sarah and Elliott are on vacation without me so I have some time to myself. I'm sitting in a coffee shop trying to get a few things into writing that have been on my mind for months. But there's a lot of beautiful people in a Boulder coffee shop and it can be really distracting in the most pleasant way. And then I find myself blogsurfing around various threads of conversation. Theoretically I'm trying to re-discover some links related to the topics, but my mind is just swimming in a web of connections and disconnections some in the blogsphere and some in the conversations at other tables or in line and at the counter.
Really fun experience, but very hard for my fingers to follow my trains of thought whether geeky and technical, philosophical and introspective, or lusty. Can I write about the mostly very fit bodies in all sorts of shapes and sizes that walk in and out? Mannerisms and postures and gaits of every rythm displaying every nuance of confidence and insecurity. Some are cool, some are obviously too cool, some are geeky, some casually sophisticated, some fashionable, but neither New York nor Hollywood fashionable. Some faces are relaxed, some stressed, some desperately in need of caffeine, some worried, some happy. Some of the beautiful people have horrible posture and some of the people who would look horrible in a photograph move in the most beautiful way.
Just soaking it all in, and even getting a little bit written.
With his own code he connects existing systems in unexpected ways (e.g. Library Lookup). He creates simple tools that encourage us to create and share our metadata (e.g. transparent feedreading, recent experiments in screencasting and InfoWorld's metadata explorer). For that matter, he regularly shines his journalistic light on projects that illustrate the emergent magic of collaboration and metadata (for example, wikipedia, bloglines, del.icio.us , flickr, and of all things the inspired heavy metal umlaut screencast).
Jon doesn't need my applause, but there's a reason I'm pointing to so much of his stuff here. I've been a fan for a long time and David's observation gave me a name to put to my favorites of Jon's work: it's all about playing well with others.
A long while back, David observed two features of Perl which have made it more successful than LISP or Smalltalk. (Defining "successful" is left as an exercise for the reader. :-) There's always been only one Perl; differing versions, sure, but you have never had to spend any time choosing whose Perl interpreter to use. More importantly, Perl began with the assumption that it would not be the only tool in your toolbox: it set out from the beginning to play well with others. By contrast, there have always been many flavors of LISP and Smalltalk any of which have tried to be pretty much everything you ever need.
Painting in broad strokes, David says Smalltalk's weakness is "at the boundaries:" when you want to try to do some typical unix system maintenance, or interfacing with underlying C libraries, or something similar. As long as you're staying within the Smalltalk environment, it completely rocks. But it's definitely painful if you try to reach outside. And it's especially painful if you want your code to work with different Smalltalks. What Perl got right was making it completely painless to integrate with its environment.
In some sense LISP wants to be on a LISP Machine and Smalltalk wants to be in its virtual machine, whereas Perl wants to go out and play with the other kids. The former languages are introverted and Perl is extroverted.
I'm focusing on LISP and Smalltalk and Perl 'cos that's where David started, but I think the notion generalizes. Java clearly follows in the LISP and Smalltalk tradition. It's easier to reach out to a different machine than it is to reach into the OS of computer hosting your JVM. ('Course that begs the question "why is Java more successful than LISP and Smalltalk?") Python and Ruby and TCL follow Perl's example for integrating well with the host environment. Along these lines, it seems worth applauding Groovy for having gotten this right, at least within the confines of the JVM. Instead of re-inventing a host of libraries, Groovy aims to just make it easier to use the existing plethora of Java APIs. Whereas, for example, Jython is inherently stuck playing catch up with C-Python because it takes time to port the standard libraries, Groovy will always just advance with the Java platform.
At bivio our entire business is written in Perl. The system maintenance, release process, unit tests, acceptance tests, mail processing, backup procedures: it's all Perl. We use RPMs for package management, but even there we generate the .spec files and kick off the builds and releases with perl. bOP is pretty monolithic. There's a lot of great stuff in there, but we're not playing along with many of the popular CPAN libraries. Within the development world of bivio, bOP is our universe, not unlike those alternate universes of LISP and Smalltalk. Even though perl plays well with others, bOP really doesn't, it's almost a world unto itself.
Having simmered these thoughts for probably over a year I feel like I have some insight to how Smalltalk and LISP got where they are. Even if I'm really only a tourist in those worlds, I think I see how they evolved into introverted and, in some sense, self-centered worlds. If you'll forgive the immodesty, we've got a small, but incredible team of programmers. [1] Where we've rolled our own code instead of using something from CPAN there have been good reasons to do so. We get a lot of efficiencies because our code is tuned to our way of solving problems. We really get an amazing amount of re-use out of our framework. And yet, it seems like we've fallen into the same mode as LISP and Smalltalk in that we live in our own world, building and expanding the reach of our bucket of tools, seldom reaching out to the rest of the world. We accomplish some amazing things, but in an isolated way.
Teach me to write at one in the morning... I didn't actually say anything about how I think it happened! Bear with me on a bit of tangent. There's been one particularly notable change in my programming since working with bivio. Rob and Paul have shown me that it's all just a simple matter of programming. Whereas I used to see specialized programming domains, I now just see code. For example I used to buy into the story that a business needs developers, and a QA team, DBAs, sysadmins. I used to think that people programming embedded systems were doing something fundamentally different than web developers which was different again from operating systems and distributed systems and device drivers. But all of it is ultimately just code. This understanding has given me a great deal more courage to tackle problems that I used to think were beyond my reach. In fact, I could describe my journey along the programming career path as one of passing through veil after veil obscuring something I thought I couldn't handle. I no longer really see programming obstacles and boundaries. Obviously there are many layers of interfaces between different components, but it feels a bit like the scene in The Matrix when Neo starts to see everything as raining lines of code. Suddenly the entire world is in reach.
I've met a few people who were Smalltalk or LISP programmers. Those I've met personally have been some of the best programmers I've met. I'm interested in LISP and Smalltalk in large part because there are programmers I respect who prefer those languages. Similarly, the guys at bivio are amazing programmers. What many of these people have in common is a kind of confidence that comes from knowing that its all just code and you get to choose for yourself which parts you want to change. Paul doesn't seem to get excited about any technology. He's got a detached confidence about his skill as a developer. The important questions for him are about business problems. He could program anything he wanted, but a real business problem brings focus. His super power is being able to sustain a laser focus on the business problem at hand.
bOP is its own world in part because Rob and Paul haven't needed to play well with others in order to solve their business problems. Where other developers might have needed the help from CPAN libraries, Rob and Paul can probably write their own faster than they can learn the quirks of the API of someone else's code. It's not that they program in a vacuum. For a long time we were using Text::CSV as a simple example. But a few months back Paul finally got fed up with it's limitations and just rolled his own that works the way he wants to work and integrates seamlessly with our other infrastructure. Frankly, it's been a huge improvement. But it's also a glimpse into the process that leads to this kind of self-centered tool building. Great programmers can and do roll their own tools. For serious LISP hackers, the fact that there are many different LISP platforms is an almost irrelevant complaint. For them it's a trivial investment to build up the compatibility layers they need to work across platforms. An accumulation of little decisions like that lead naturally to fragmentation and in some sense isolation.
But David is right that playing well with others is important. For all the other virtues of LISP and Smalltalk, Perl does the the play-well-with-others thing right.
[1] As long as I'm being immodest, I might as well be specific. We are five programmers and one customer support person actively developing and maintaining eleven different software systems of varying complexity.