In the middle of the 1980s, the same folks at Bell Labs who had previously invented Unix began a new OS-research project called
Plan 9. Although Plan 9 is not in wide use outside Bell Labs, and it's not the sort of operating system you'd expect large numbers of ordinary computer users to take to—at least, not in its present form—it's nevertheless a very interesting project.
The characteristic of Plan 9 that is perhaps easiest to describe is also one of its most interesting features. When the creators of Unix conceived it, they invented a novel way to control I/O resources: they represented them all as files. In earlier systems, different kinds of I/O devices had been represented with a wide variety of different APIs. In Unix, these all became files, and could all be addressed in the same way. If you wanted data from a device, you read its file. If you wanted to change its state somehow, you wrote to its file. Suddenly, thousands of different kinds of devices all became one thing.
It worked very well, greatly simplifying system programming on Unix, and making possible the evolution of a distinct style of system programming, in which system tasks are organized in terms of file reads and writes, and piping data through sequences of file-like objects.
The insight of the Plan 9 developers was that they had not taken their model far enough. Once Unix escaped into the wild, it began to acquire new features, and those new features were not, by and large, carefully designed to preserve the Unix vision that everything was a file. Soon there was once again a bewildering variety of APIs that any Unix system programmer had to understand.
In designing Plan 9, its creators resolved to return to the roots of Unix, and this time push the file metaphor as far as they could. The result is an operating system in which everything is a file. Files on disk, of course, are files. But so is the network: every network-accessible system in the world is a directory in the Plan 9 filesystem. Websites are directories in the filesystem. Windows are files; to display something in a window, you write the something to the window's file. To burn data to a CD, you copy the data to the CD's file in /mnt/cd/.
Everything is a file.
When I first began to learn to program in Lisp, I was fascinated by Lisp machines. So much so, in fact, that I wangled a couple of different Symbolics machines to play with for several years. The Symbolics operating system, Genera, was not very much like Plan 9, but they both took a completely original, clean-slate approach to making computing hardware usable. That has been done relatively few times in computing history, and for good reason. Developing an operating system is time-consuming and expensive, and its odds of financial success are low. Technical excellence is no guarantee of success. Plan 9 itself, for example, is certainly technically excellent, but it hasn't seen the wide adoption of its ancestor Unix, in part precisely because Unix is so widely adopted and, whatever its failings, is adequate for the tasks to which its put. The cost of adapting to a system as new and different as Plan 9 are manifestly not worth the expense of giving up substantial investments in Unix.
Still, for a certain kind of mind, the lure of rethinking computing environments is irresistible, despite its costs and risks. I had the very good fortune to work for a few years on a clean-slate OS project at Apple. Apple's Newton engineers wrote an entirely novel OS for their handheld computer. Much of the work was done in a dialect of Lisp, my favorite language, and all of it was original, ignoring compatibility with any previously-existing operating system. It's delicious to try to think of everything in an OS as brand new; to try to imagine how you can do things in terms of the resources you have available, instead of in terms of what has been done before.
One lesson of Plan 9 is that the right organizing metaphor is extremely powerful. It was that same principle that made the Mac successful: Apple stumbled over the graphical desktop metaphor at Xerox, and realized it was just the kind of organizing metaphor that could pull together a great product.
So the whole time I was playing with Symbolics machines and working on Newton and fiddling with Plan 9, I was thinking about what organizing metaphor captures the essence of Lisp.
When Linus Torvalds created Linux, I thought that was great; he released a free Unix-like kernel that could be used to build up a Unix-like operating system, reusing the metaphors that had grown up in that world. If someone released a similar Lispy kernel, around which a Lispy culture of computing could grow, what would it look like? What would be the organizing metaphor of a new free Lisp-machine OS?
The old Lisp machines didn't have a single clear organizing metaphor in the way that Plan 9 or the Mac had. They had Lisp, of course; a machine whose system programming language is Lisp is quite wonderful, in that everything about the system is manifest and accessible right down to the hardware. You can easily inspect and modify everything, and with much greater safety than in, say, a Unix system. As Rainer Joswig likes to say, everything in the system is alive; programming is not a matter of manufacturing dead, static things, but of live interaction with running code.
Still, there is not the central organizing vision that everything is a file, or that everything is a cartoon object that you can grab and manipulate with your hand. If there was such a concept for a Lisp OS, what wold it be?
Gradually, I came to think that is was something like a closure. I started out thinking about lists, of course, because that's the obvious place to start when asking this kind of question about Lisp. Lists are not quite the right thing, though. They're important, but a couple of things keep them from being the right starting point. A list is a tree, of course, and Plan 9 shows how powerful a tree is in organizing resources; it makes every file a node in a galactic network that, from any specific user's point a view, is a tree rooted at that user. Since everything computable in Plan 9 is a file, Plan 9 organizes the entire computational universe into a single vast tree with many virtual roots. That's pretty cool, but it's only part of the picture.
What about lambda? Lambda is also obviously very important. From the point of view of Lisp, lambda is computation. It's why computers do things;why they are more than just collections of inert data. I wanted a way to think about computing that combined lambda with lists, and I wanted one more thing.
Lisp machines were promiscuous. They had very little notion of security. They were single-user workstations, and the user was a programmer who wanted convenient access to everything the machine could do. Security was just another word for obstacles to productivity.
Nowadays it would be irresponsible to design an operating system that couldn't protect a user's resources from abuse. Networks are ubiquitous, and so is misuse of them. Any operating system from now on that is designed for general use has to take into account that its users will want protection against burglars and vandals.
It was this thought, combined with my general affection for the Common Lisp Object System, that led me to coin the name Closos for an imaginary Lisp OS built on a single organizing principle. In French, the world clos means a private, walled vineyard. In Catalan it means a walled garden. I imagined creating an operating system built on CLOS in the same sense that Dylan was built on CLOS (though Dylan was not Common Lisp). CLOS plus OS, became Closos.
So what's the organizing principle? I think it's the closure, more or less. A lambda wrapped in an environment. The lambda means the closure is
active; it can
do things. The lexical environment it's embedded in gives it memory—references to resources. If you imagine that various closures can be applied concurrently, and that they can pass data to one another, then the computing environment starts to look like the
Actor model. The Actor model has a long and distinguished history in distributed and concurrent computing, and Erlang has famously demonstrated how far it can take us in productivity and reliability. We might easily forget that a primary motivation for the design of the Scheme language was to support experimentation in Actor programming.
It's important to repeat that Closos is pretty much completely imaginary. From time to time I pick up a little piece of the idea and work it out in some code that I then squirrel away somewhere to think about and maybe come back to again later. I have the vague notion that I might get something really written some time in the future, but lately I've noticed that my age keeps advancing relentlessly, and the number of items on my To-Do list that are in front of Closos never seems to diminish. So perhaps it's destined to remain a fond daydream, just right for woolgathering on a rainy day.
engine of joy
Comments
Rayne (unauthenticated)
Feb 26, 2010
That last paragraph nearly made me cry. It's very poetic.
asenchi (unauthenticated)
Feb 27, 2010
I've also been thinking of how this would be possible. Lisp and Plan9 are two technologies that need to be merged and brought to the mainstream. I spend my "20%" time (in general) thinking about and honing my skills to achieve this. I am still a long way off but I hope to see it some day.
My one prevailing thought is that the lisp version needs to be a lisp-2, and it needs to not be Common Lisp. It needs to rival dynamic languages in it's simplicity but CL in it's complexity and macros.
I have to admit, it's exciting that someone else out there is thinking of this. However, you have much more experience in this area than I do. Someday I hope it becomes a reality. Someday...
matthias (unauthenticated)
Feb 27, 2010
Thx for the article. Great idea...why not publish some of your snippets on github?
mikel evins
Mar 1, 2010
I had a mildly bad experience with github. It's not really enough to justify avoiding it, but then, on the other hand, all of my code is available from repos on bywicket. If you want some of it, just ask.
meillo (unauthenticated)
Mar 2, 2010
asenchi wrote: ``Lisp and Plan9 are two technologies that need to be merged [...]''
I completely disagree. You better do not try to merge different world -- each of them of elegant pureness. If you though try to, you'll end up in some ``gray soup'' without elegance and without pureness.