Closos, Part 2

The first post about Closos generated a couple of interesting comments. Generally speaking, I direct my posts to a pretty small readership, consisting of my future self and a few close programmer and gamer friends. The posts are public, though, and because they're public, they sometimes generate some interesting comments from a wider readerhip. So it was with "Closos".

The first post was pretty much a confection: I got out of bed and wanted to just jot down something about the general idea of finding an organizing metaphor for a Lisp-flavored OS. I didn't say much of anything very specific, nor did I mean to; the topic of the post was the history of this idea—the idea of identifying an organizing metaphor that is Lisp-flavored.

One interesting comment was that the metaphor "everything is a lambda" doesn't mean anything. By contrast, mathrick wrote, "everything is a file" means something specific, because we have a concrete idea of what files are and what they do. But that's not quite fair, I think. We have a concrete idea of what files are and what they do because the metaphor has already been worked out concretely in practical detail. That's the trouble with a metaphor; it only communicates if the listener already knows what you're talking about.

Rainer Joswig knew what I was talking about. He provided a couple of comments that described the OS of the MIT Lisp Machines. He characterized it as "Flavoros"—an OS in which everything is a flavor. That's just about right. But just as the designers of Plan 9 moved on from Unix to design a new OS that takes the metaphor further, I imagine moving on from Genera to envision a modern Lisp OS with an updated vision of the organizing metaphor.

Let me just take a moment to emphasize again that I'm just imagining things now. I'm not actually working on anything. Or, rather, I'm working on a bunch of other things that either pay me right now or that I expect to pay me at some point in the future.

Just as Unix acquired features faster than they could be integrated into the everything-a-file metaphor, so too did the LispM systems acquire new features faster than they could be fully digested. Besides which, of course, the Unix computing culture thrived, while the LispM ecology, for various historical reasons, did not.

I come from a background of writing code for Apple Computer. I spent several years working on a Newton OS that was written in Lisp, and then on a great application-development system called SK8, also written in Lisp. Both of those systems (and indeed, the C++ Newton system, too) were built on frame languages.

In both Newton and SK8, everything was a frame, in the same way that in Genera, everything was a flavor. Newton had no files and no filesystem; instead, everything was stored in a "soup" of frames with automatic persistence. As in Rainer's description of the LispM system, all the system's resources were represented by objects linked in a graph.

Frames are something like Dictionaries in Smalltalk or Python; or like hashes in Perl, or like Maps in Clojure. They're objects that associate values with keys. Frames, though, have some semantics that these other associative arrays don't. A value in a frame is stored in a slot. A slot might be a simple container, as in these other examples, but it might not. It might not exist in the frame at all; it might be computed dynamically, on demand. Or it might exist, but not be just a simple container. In SK8's MacFrames language, for example, a slot can itself be a frame, with any number of additional slots describing what can be in the slot, how it can be gotten, what else happens when it's accessed, and so on.

Frames also support inheritance. A frame system can have many different kinds of inheritance—"is a", "has a", containment, implication, and others—operating all at the same time, and dynamically reconfigurable at runtime.

So why don't I say that "everything is a frame"? I don't think frames or flavors are quite right as the organizing metaphor. The file captures a great deal of what you want to think about in a system for computing. It enfolds the ideas of input and output, and when extended to directories, it includes the concept of network and hierarchical organization. At the same time, it remains appealingly simple. Frames and flavors don't achieve quite the same density of concept.

Files don't feel like Lisp, though. That's what had me looking for another metaphor. The one I settled on is clear in my mind, but doesn't have a one-word name that, in its technical sense, is in wide use. That shouldn't be surprising; the only reason that "file" has such a name name is that it's been worked out in practice for forty years, and the word and its technical usage are everywhere.

"Lambda" isn't the word I had in mind. Despite what mathrick appears to think, I'm not saying "everything is a lambda". As I said, the "Closos" post was about trying to think of this concept; it was not meant to be a clear explanation of the concept itself. A reasonably succinct description of the concept might be "a lexical closure over a lambda with an environment that includes zero or more possibly-lazy lists." Not exactly musical, I know; that's the problem.

It has been said that "objects are a poor man's closures." The lexical environment of a closure offers the same opportunities for managing references to data as a frame. The lambda in the middle of the frame provides a computational engine. Input and output streams are possibly-lazy lists, bound to variables in the closure's environment. Variable bindings provide the glue for connecting these objects into arbitrary graphs.

Moreover, because of the opacity of the closure's lexical contour, outsiders can only know what the closure wants them to know about its connections, and the graph that one closure sees might be completely different from the one that another sees. This quality is another insight pioneered by Plan 9: my filesystem doesn't have to look the same as your filesystem, even if they are made up of the same files. The entire computing world is connected in a single graph, sure; but that doesn't mean I see what you see. It's enough that we see enough of the same nodes in ways that are convenient to each of us.

How about, "everything is an actor"? That's what I said in reply to mathrick's objection. An actor, in the sense of the actor model, is an autonomous unit of computation that accepts inputs and sends outputs, and computes new states of itself. Scheme, from which I take the notion of a lexical closure over a lambda, was designed in part to serve as a language for exploring actors.

Aside from its utility in designing concurrent algorithms, the actor model offers a handy word and concept on which to hang a lot of computing—just as the concept of "file" does. Like the file metaphor, the actor metaphor offers convenient ways to talk and think about input, output, organization, access, resources, and so forth.

Maybe, instead of adopting a name that connotes a walled French vineyard, I should have swiped terminology from SK8. In SK8, the stage was the data structure that represented all objects that a user could interact with on-screen. Any data placed on the stage had to be rendered, and so it was wrapped in a data structure called a costume that facilitated the rendering. SK8 wasn't an actor system, but it certainly had the right vocabulary for one.

That must be why I named my hobby programming language "Bard". Maybe if it ever evolves a development environment, it should be called "Globe".