No, I Am Your Father

The line everybody gets wrong

Nobody in the film ever says “Luke, I am your father.” The line is “No, I am your father.” Millions of people remember it wrong, and they all remember it wrong in the same way, with great confidence.

The family tree of JavaScript gets the same treatment. Ask a room of developers where JavaScript's objects came from and you'll hear “Java” (the name was a marketing deal), “C” (the curly braces), or “Brendan Eich, in ten days” (true, but he didn't invent them from nothing). The answer is a research language most of them have never seen: Self, designed in 1986 by David Ungar and Randall B. Smith at Xerox PARC.

Self never had a large user base. Its ideas did. Every JavaScript object you've ever made follows Self's model, and the engine running this page (V8, if you're in Chrome) is built on tricks the Self team invented to make that model fast. Self is the parent nobody mentions at family dinners.

This mini book tells that story:

• Who the grandfather is. Smalltalk: classes, metaclasses and all.
• What Self threw away, and why throwing things away turned out to be the upgrade.
• How prototypes fixed the fragile class, and the part they didn't fix (it's a trap).
• Where the real revolution was. Not in the syntax: in the engine room.
• The reveal. JavaScript, and its later attempt to dress up as somebody else.

The grandfather

“I find your lack of classes disturbing.”

Every Smalltalk programmer, 1987 (allegedly)

Before there was a son, there was a grandfather, and before him a great-grandfather. In 1967, in Oslo, Ole-Johan Dahl and Kristen Nygaard gave the Simula language classes and objects to model ships, queues and customers. Object-oriented programming started there.

In the 1970s at Xerox PARC, Alan Kay, Dan Ingalls, Adele Goldberg and their colleagues took the idea much further. Smalltalk said: everything is an object, and the only thing objects do is send each other messages. Numbers are objects. Classes are objects. true is an object. There's no if statement; you send ifTrue: to a boolean and hand it a block of code. Smalltalk-80 shipped with overlapping windows, a mouse, a live image you could change while it ran, and browsers for reading and editing every line of the system. It was decades early.

This is how you make a kind of object in Smalltalk-80. You don't make the object; you make a class, and ask the class to make instances:

Object subclass: #Animal
    instanceVariableNames: 'name'
    classVariableNames: ''
    poolDictionaries: ''
    category: 'Zoo'.

Animal >> speak
    ^ name, ' makes a noise'

rex := Animal new.

Look at the first line. To create the class Animal, you send a message to another class, Object. Since everything in Smalltalk is an object, a class is an object too. Every object has a class. So what is the class of a class?

The tower of metaclasses

Smalltalk answers honestly, and the answer is a staircase:

Ungar and Smith looked at this and listed the costs of classes, even in a language as clean as Smalltalk:

• Two concepts where one would do. Programmers juggle objects and classes, instances and metaclasses, instance variables and class variables.
• No one-offs. You can't just make one special object. You must first describe its whole kind, then make the single instance.
• Two ways to touch state. Inside its class, a method reads name as a variable. From outside, you must send a message. The same state has two different kinds of access.
• Things to keep in sync. The class describes the shape of its instances. Change the class and every existing instance has to be converted to the new shape.

The power of simplicity

“Do, or do not. There is no class.”

Paraphrased, heavily

In 1987 Ungar and Smith published “Self: The Power of Simplicity” at the OOPSLA conference. The paper took Smalltalk's “everything is an object” and removed what it saw as the scaffolding. What was left is three ideas:

1. An object is a set of named slots. A slot holds a value or a method. Reading a slot and calling a method look exactly the same.
2. New objects are copies. There are no classes. You make an example object, the prototype, and copy it.
3. What you don't know, ask your parents. When an object gets a message it has no slot for, it passes the question to its parent objects. Parents are just slots marked with a star.

Here is a whole object, written between (| and |). x <- 3 is a slot you can change, y = 4 is a constant, and sum is a method. The method reads x by sending the message x to itself; you don't write self, because Self assumes it.

Meet the father

Let's make a real prototype and put it in the lobby, the object where all the global names live. Its parent* slot points at traits clonable, a shared object that knows how to copy.

No class, no constructor, no new. To get another one, copy it. The copy has its own name slot, so changing it doesn't change the original. (Assigning is a message too: name: 'Self 4.0' sends name: to the copy, and the answer is the copy itself, so we can keep talking to it.)

Meet the son

Now a second object whose parent is father. It has a name and a birth year, and nothing else. It doesn't define reveal.

The son doesn't understand reveal, so he asks his father, and the father's method runs. But inside it, self is still the son, so name means the son's name. This is called delegation: the behaviour is borrowed and the identity stays put.

Self can look inside any object. A mirror reflects on an object without disturbing it. Ask what the son actually owns:

Three slots. Everything else, he borrows. That's the whole object model: slots, copies and parents. Smalltalk's classes, metaclasses, instance variables, class variables and constructors are all replaced by those three ideas.

What Self did to Smalltalk

“When I left you, I was but the learner. Now I am the master.”

Self, to Smalltalk, probably

Self didn't reject Smalltalk. It's obviously Smalltalk's child: the same unary, binary and keyword messages, the same blocks in square brackets, the same ^ for returning, the same comments in double quotes, the same live world of objects. The changes are all in one direction: take a distinction away.

Three of these changes do most of the work. Let's run them.

Upgrade 1: there is only one way in

In Smalltalk, a method inside Droid reads the instance variable battery directly. If you later decide the battery level should be computed rather than stored, every method that touched the variable must change, including methods in subclasses you didn't write.

In Self, nobody touches a variable directly, ever. droid battery is a message. So you can swap a stored value for a computed one and nobody who asks can tell:

Bertrand Meyer later named this the uniform access principle. JavaScript got its own version in 2009, when ES5 added getters. Python calls it @property. Self simply never had a second way.

Upgrade 2: control flow is just messages

This one Self inherited intact from Smalltalk, and it deserves the credit: there's no if. A comparison answers true or false, which are ordinary objects, and they answer ifTrue:False: by running one block or the other.

(Self writes the second keyword with a capital, False:, so the parser knows ifTrue:False: is one message and not two.)

Upgrade 3: parents are slots, so they can change

With resend, a child runs its parent's version of a method, which is what other languages call super. Here is the third generation:

Now the part Smalltalk couldn't do without tricks. The son's parent slot was written parent* <- father, with an arrow, so it's assignable. An object can change who it inherits from while the program runs. Self programmers call this dynamic inheritance and use it for objects whose behaviour depends on their state: a window that is open or closed, a parser in one mode or another.

It's a trap: the fragile class

“It's a trap!”

Admiral Ackbar, reviewing a pull request to a base class

The fragile base class problem is the name for a familiar pain: you make a reasonable change to a base class, and code you've never seen, in subclasses somebody else wrote, breaks. It comes in two kinds, and it's worth being precise, because prototypes fix one of them and not the other.

Kind 1: structural fragility

Here the subclass depends on the base class's shape. In C++, an object of a subclass contains the base class's fields at fixed offsets. Add one field to the base class and every offset in every subclass moves, so everything that inherits from it must be recompiled. Ship a library with a new private field and you can break programs that were compiled against the old version.

Smalltalk is far more forgiving, because the whole world is live, but the structure is still there. Compiled methods reach instance variables by their position. Add an instance variable to Animal and the system must recompile the methods of every subclass of Animal and convert every existing instance of every one of them to the new layout. Smalltalk does that for you, by mutating the world. And because subclasses read inherited instance variables directly, renaming or removing one in the base class breaks the subclasses' methods.

How prototypes take the structure away

Self removes every one of those dependencies:

• Children don't contain their parents. A parent slot is a pointer, not a copy of the parent's layout. Adding a slot to a parent changes the parent and nothing else.
• Nobody reaches into anybody's fields. Every access is a message (chapter 3), so a parent can change how it stores things without breaking a single child.
• There's no class to keep in sync. A prototype is just an object. Changing it affects the copies you make next. Existing copies aren't converted, because there's nothing to convert them to.

Watch the father learn something new while his whole family is alive. The son and grandson have it at once, with nothing recompiled and no instance converted:

Now change a prototype itself, with a copy of it already out in the world. (We experiment on a copy of the son called rebel, so the real son stays as he is for later chapters.)

Nothing broke, nothing migrated, and the old object says so plainly instead of being silently reshaped. If Luke needs the slot, you add it to Luke. The layout bookkeeping that C++ makes you recompile for, and that Smalltalk does by rewriting the world, still exists in Self. It lives in the virtual machine, which tracks object layouts in hidden structures called maps, and it never reaches your program. Chapter 5 is about them.

Kind 2: semantic fragility, the trap

The second kind isn't about layout. It's about behaviour: the base class calls its own methods, a subclass overrides one of them, and the base class's behaviour changes in ways neither author intended. The textbook example is a set that counts insertions by overriding both add: and addAll:. If the base class's addAll: happens to call add:, every element is counted twice, and if a later version of the base class stops doing that, the count silently becomes correct again.

Prototypes do not fix this. They can't, because it's the same mechanism we celebrated in chapter 2. The father's reveal sends name to self. Any child can change what that means:

And one new trap of its own

Self added a trap that Smalltalk doesn't have. If a child doesn't have a slot and assigns to it, the assignment is found in the parent and changes the parent's slot:

The kid tried to rename himself and renamed his father. Self programmers avoid this with a discipline described in the 1991 paper “Organizing Programs Without Classes”: put shared behaviour in a traits object, which is a parent, and put state in a prototype, which gets copied, so every copy carries its own slots.

The revolution was in the engine room

“She may not look like much, but she's got it where it counts, kid.”

On the Self virtual machine

By the standards of 1986, everything you've read so far is a performance disaster. Smalltalk was already considered slow. Self is worse in every respect: reading a variable is a message send, every if is a message send with blocks, and there are no classes for a compiler to reason about. Any object can gain slots or change parents at any moment. Critics said it couldn't be made fast, and on the evidence of the time they were right.

The Self group, which moved to Stanford and then to Sun Microsystems, spent the next eight years proving them wrong. They built on the best Smalltalk work of the day, especially L. Peter Deutsch and Allan Schiffman's 1984 Smalltalk-80 implementation, which already compiled to machine code on the fly and remembered method lookups at each call site. Then they went much further:

1. Maps (1989). Objects copied from the same prototype share one hidden “map” that records which slot sits where. Copies become as compact as instances of a class, because, underneath, the VM keeps something very like a class for them. The program never sees it.
2. Customised compilation (1989). Compile a separate version of each method for each kind of receiver. Inside that version, the compiler knows exactly what self is, so sends to self can be resolved early and inlined.
3. Polymorphic inline caches (1991). Each call site remembers the handful of receiver types it has actually seen and jumps straight to the right method. As a side effect, the caches record which types really occur where.
4. Dynamic deoptimisation (1992). When you open a debugger, or change a method the optimised code relied on, turn the optimised frames back into simple ones that match the source. You get aggressive optimisation and a fully debuggable, changeable language at the same time.
5. Type feedback and adaptive recompilation (1994). Start with a quick, simple compiler. Watch the program run. When some code turns out to be hot, recompile it, optimised for the types the caches actually saw, inlining aggressively.

The result, according to its authors, ran some benchmarks at around half the speed of optimised C, in a language with no classes, no static types and messages for everything.

So where was the revolution?

Not in the prototypes; other people had prototypes. Not in the syntax, which is Smalltalk's. The revolution was a reversal of who adapts to whom. Before Self, language designers bent the language to fit the compiler: declare your types, fix your classes, keep things static so the compiler can cope. Self made the language as simple and dynamic as people wanted and made the virtual machine learn the program's real behaviour while it ran: speculate, optimise, and back out when wrong.

Every serious dynamic-language engine works that way now. That is how Self ended up running nearly everywhere, without anyone running Self.

V8 still calls its hidden classes Maps, the same word the Self papers used. Every “initialise your properties in the same order” performance tip you've read for JavaScript is advice about keeping objects on the same map.

Now time something. The Self in this book is a straightforward interpreter, written in Rust and compiled to WebAssembly. It has none of the tricks above, so it's a good deal slower than the real Self VM was in 1995:

No, I am your father

“No. I am your father.”

The actual line

May 1995, Netscape. Brendan Eich has about ten days to build a language for the browser. He wanted to put Scheme in Netscape; management wanted it to look like Java, the hot new language from Sun. What he shipped has Java's syntax, Scheme's first-class functions, and Self's objects. It was called Mocha, then LiveScript, and in December 1995, after a marketing agreement with Sun, JavaScript.

Here's the reveal, in both languages. First Self:

And the same family in JavaScript. Object.create(father) makes a new object whose parent slot, its [[Prototype]], is father:

It's the same object model: objects made of named properties, delegation to a parent, and this bound to the original receiver, so the father's method speaks in the kid's name. Squint and the two programs are the same program.

What the son did better

Remember the trap at the end of chapter 4, where the kid renamed his father? JavaScript fixed it. Assigning to a property the object doesn't have creates a new property on that object, which shadows the parent's instead of overwriting it:

It was a good decision. In return, JavaScript has only one parent per object (Self allowed many) and no non-local return. It also has no live environment, unless you count the developer tools: expand an object in your browser's console and you'll find a little outliner with a [[Prototype]] slot at the bottom.

What the son did to look respectable

To make prototypes feel familiar to Java programmers, JavaScript added new and constructor functions. Every function got a prototype property, which becomes the parent of the objects that new makes. That arrangement meant that for two decades JavaScript developers had to learn the difference between prototype (a property of functions) and __proto__ (the actual parent link). The plain, Self-style way to make an object with a given parent, Object.create, wasn't standard until 2009, after Douglas Crockford spent years arguing for it.

And the father's two gifts fight each other inside the son. Changing an object's parent at run time is a Self feature; hidden classes are a Self optimisation. Combine them in JavaScript and the optimisation punishes you for using the feature:

It works. MDN's documentation for setPrototypeOf opens with a warning that changing an object's prototype is very slow in every engine, because of how engines optimise property access. That optimisation is the Self VM's.

The son puts on the helmet

“Search your feelings. You know it to be true.”

To anyone who thinks JavaScript has classes

In 2015, twenty years after it was born with prototypes, JavaScript got the class keyword. The son dressed up as the very thing his father had deleted. But a helmet only changes how you look:

A class is a function. extends builds a prototype chain. luke owns nothing and borrows everything, exactly like the son in chapter 2. ES2015 classes do add real things, such as stricter constructors and cleaner super, but the object model under the helmet is still Self's.

One more family resemblance. Strongtalk, the Smalltalk built by Self's VM team at Animorphic, also had an optional type system: you could add types to a dynamic language, have them checked, and the program would run the same either way. Today the most popular optional type system in the world is TypeScript's, and like Self it cares about what an object can do rather than what it's called. The grandson who said “now with types” in chapter 3 wasn't wrong.

Why it matters

“Pass on what you have learned.”

Good advice for languages, too

As a product, Self was a failure. It never went mainstream, never had a big community and never shipped in a browser. As a set of ideas it was one of the most successful projects in the history of programming, and it won three separate times:

• Its objects won. Every JavaScript object, which means most objects created on Earth today, is a Self object with Java's haircut.
• Its virtual machine won. Adaptive optimisation, inline caches, hidden classes and deoptimisation run in HotSpot, in V8 and in many other engines. The idea that the runtime should learn from the running program is now simply how fast dynamic languages are built.
• Its liveness won, partly. Self 4.0's Morphic, by John Maloney and Randy Smith, let you grab any object on screen, look inside it and change it while it ran. Morphic moved to Squeak and Pharo Smalltalk and to the browser in the Lively Kernel, and every “edit and see it instantly” tool is a distant cousin.

Underneath those wins are three lessons that still apply:

1. Keep the language simple and let the implementation be clever. Programmers shouldn't have to contort their code so a compiler can understand it. Self showed that the machine can do the understanding.
2. Concrete beats abstract. People find it easier to copy an example and change it than to write a description of every possible example. Design tools rediscovered this: a component with instances that override some of its properties is a prototype with children.
3. One way in makes change cheap. When everything is a message, a parent can change how it works without its children noticing. That's the structural half of the fragile class problem, solved by removing a distinction rather than adding a rule.

So the next time someone tells you JavaScript is Java's little brother, you can correct them, and get the quote right while you're at it:

“No. Self is your father.”

Further reading

The primary sources are short and surprisingly readable. They are listed in the order this book uses them.

1. L. Peter Deutsch and Allan M. Schiffman, Efficient Implementation of the Smalltalk-80 System, POPL 1984.
2. Henry Lieberman, Using Prototypical Objects to Implement Shared Behavior in Object-Oriented Systems, OOPSLA 1986.
3. David Ungar and Randall B. Smith, Self: The Power of Simplicity, OOPSLA 1987.
4. Craig Chambers, David Ungar and Elgin Lee, An Efficient Implementation of SELF, a Dynamically-Typed Object-Oriented Language Based on Prototypes, OOPSLA 1989.
5. David Ungar, Craig Chambers, Bay-Wei Chang and Urs Hölzle, Organizing Programs Without Classes, Lisp and Symbolic Computation, 1991.
6. Urs Hölzle, Craig Chambers and David Ungar, Optimizing Dynamically-Typed Object-Oriented Languages With Polymorphic Inline Caches, ECOOP 1991.
7. Urs Hölzle, Craig Chambers and David Ungar, Debugging Optimized Code with Dynamic Deoptimization, PLDI 1992.
8. Urs Hölzle and David Ungar, Optimizing Dynamically-Dispatched Calls with Run-Time Type Feedback, PLDI 1994.
9. Leonid Mikhajlov and Emil Sekerinski, A Study of the Fragile Base Class Problem, ECOOP 1998.
10. David Ungar and Randall B. Smith, Self, History of Programming Languages III, 2007. The story told by the people who lived it.
11. Allen Wirfs-Brock and Brendan Eich, JavaScript: The First 20 Years, History of Programming Languages IV, 2020.
12. The Self project, still maintained by its community: selflanguage.org.