A Conversation With Doug Engelbart

Dr. Dobb's Journal September 2000

Solving complex problems as a life goal

By Eugene Eric Kim

Doug Engelbart is a man who thinks big. Fifty years ago, he dedicated his career to designing systems that could help the world solve its most difficult problems. Along the way, he invented the mouse, hypertext systems, two-dimensional display editing, collaborative video teleconferencing, and a host of other technologies that today form the basis of interactive, collaborative computing. Many of these inventions were part of the NLS System, which Engelbart first presented at the 1968 Fall Joint Computer Conference, a demonstration that later became known as "The Mother of All Demos."

While the World Wide Web and related technologies were directly inspired by Engelbart's work, they only scratch the surface of the capabilities of NLS and its successor, Augment. Today, Engelbart is developing an open-source version of the next generation of his system for collaborative knowledge work, dubbed the "Open Hyperdocument System." His initial goal for the OHS is to help programmers collaboratively develop software, a problem that DDJ readers will agree is as complex as it gets.

Engelbart is currently director of the Bootstrap Institute (http://www.bootstrap.org/), which he founded in 1988 to help organizations learn how to improve their ability to solve complex problems using tools such as interactive computing. Engelbart recently spoke with Eugene Eric Kim about his life's work and ways in which we can augment the collective human intellect.

DDJ: What's been the underlying goal of your research for the past 50 years?

DE: Well, it really hinges on one thing: How to help mankind get collectively better able to cope with complex, urgent problems.

DDJ: Was this the target as you thought of it back in the early 1950s?

DE: Almost word for word.

DDJ: What were the things that got you in that direction in the first place?

DE: It was because of some woman. [Laughs] She said she'd marry me. So the following Monday, driving to work -- I was working over at NASA Ames Laboratory as an electrical engineer -- I just got to saying, okay, my personal life now is getting all set, I'll get married happily ever after and have a family. And for some reason I started thinking about my professional life, and just got to seeing the long haul. Nice people, nice place, secure job, but just no events. It was just unclear to me what kind of goals I was after professionally; and it was just humiliating to think I didn't have anything other than a pleasant, steady job. But I grew up in the Depression, when this was almost every kid's thing -- get a steady job. Well anyway, this was so much that I said, gee, that will never do. So what do you want out of a career? So I think I made a quick estimate. I was 25, and I figured, well, 65 is hopefully when I'll quit. So that's 40 years, 2000 hours a year, it ends up being about 5 million minutes, I think. That's quite a bit to invest. So how do you want to get the best return on your investment? For some reason, I set aside money very quickly. Pick something real and assume that there will be enough money to raise a family. And that's how I got to this point. Sometime the next day, I picked that goal.

DDJ: Picked the goal of solving man's complex problems?

DE: No, of maximizing investment about humanity in terms of how much good it'd do. And I looked for all kinds of crusades, so several months of crusade hunting. And then, one Saturday, I said, well, boy, crusades are really complicated. In the first place, you don't just walk down the street and announce you have a crusade and put out advertisements for production managers of crusades.

And in the second place, you hear stories of people who have gone after fixing something for society and by the time they got done, the side effects were almost as bad as what they were trying to fix. These things are so complex. And that's when it dawned on me, the basic thing of complexity and that the really complex problems have to be dealt with collectively, and our collective ability to deal with complex, urgent issues is not increasing as fast as you can predict that problems are.

I was an electrical engineer, but I had also been fortunate enough that in World War II, I'd been drafted into the Navy, and they had given me the opportunity to go through their training for electronic technicians -- radio, radar, sonar. In radar, a lot of the circuits are similar to those used in computers. I didn't know about computers in detail, but this radar used a cathode ray tube display, and its electronics make the display images. So, if a computer can punch cards and print, then it certainly could drive the electronics that would put what you want to see on the screen.

And if the radar could sense the operator turning a crank and pushing buttons, then computers could certainly do that, too. Then you could sit there and interact with the computer. And you could get all kinds of new portrayals, and really very different ways in which you could get your knowledge there to see. Other workers could sit and connect to the same computer complex, and you could be working together in really radical new ways. Oh boy! Okay!

DDJ: When you adopted this crusade in the first place, did you know that computers were going to play a crucial role in what you were trying to do?

DE: For sure. I just assumed that if the world realized what it can do in that respect, and can see it working, that yes, computers would be crucial. And later, by the time I got out of graduate school, the military was getting more interested in computers. And I thought, wow, if all that potential that I see starts to be realized, there's just going to be so much of a market that there was no question that industry would start to produce cheaper and cheaper and better computers.

DDJ: One of the things that you often say is that today, mankind is facing the most pervasive change that it's ever faced. Was this your feeling back in the '50s as well, or is this a more recent sentiment?

DE: The unprecedented change scale came out for me after I went to SRI (http://www.sri.com/) in 1957. I started -- as I termed it to myself -- working for my room and board, doing whatever research they'd pay me for, to get a chance to do my own work eventually. So, I kept looking for things that would be more relevant to what I wanted to do, and one of them turned out to be a study about the dimensional scaling of electronic components -- so, what would happen if you could learn how to make components smaller and smaller? That opened up a whole field of physics, how to deal with the change of scale. And then it turned out, that's related to what I had bumped into at NASA Ames, where they test models in a wind tunnel. In order to make the data relevant, they have to keep something called the Reynold's number constant.

It's a quotient of a term that has to do with the different dimensions that you change -- size, speed, pressure, and other things. So, if you get the right ratio of those things established, and put the dimensions on, the dimensions all cancel and you get this dimensionless number. So, if you change all these dimensions in such a way that that number stays constant, then for some interesting abstract reason, the data you take are applicable to the full-scale airplane. Otherwise, if you just build a scale model of an airplane, it won't fly.

Then, the more I looked into that and realized that some people back in the '20s and '30s had written some really interesting articles about things like that, that kind of illustrated how, when scale was changed, what interesting things happened. And I began to realize that the results of large scale changes would usually be full of surprises.

I believed that what's now called nanotechnology was a sure thing. Well, what's going to happen when the components get much smaller and the computers get much cheaper and faster? I just realized, my god, the huge, foreseeable changes in that tool system, in technology, were bound to create surprising changes, throughout society. How was this going to change all the other things, organizational, etc.? These changes in scale are going to cause all kinds of surprises. We're not really oriented for that scale of change and the kinds of complex interactions between multiple changes that will be involved.

DDJ: When you started doing your research, did you think that artificial intelligence had a future, or did you reject it from the beginning?

DE: No, I didn't reject it. They rejected me.

DDJ: Why did they reject you?

DE: I was saying, look, we can make machines smarter and smarter, but their value will be in how much smarter they make people. And people's environments are going to change as we automate what they used to do. These sentiments got me in trouble with almost all the rest of the world. In the '70s, all of the proposals going into the NSF and DARPA were what computers were going to do. And it was all about how they were going to make it so easy. The AI guys, for instance, they claimed that, hey, our smart computer will sit there and interact with the user, analyze what the user knows, and adapt so the user won't have to learn anything new. And they pointed over to my stuff, where we had links and structure and all kinds of new functions going on, and of course the interface looked strange, because how do you manage those things without different verbs and nouns? And they all started pointing to our stuff as the example of hard-to-learn bad stuff. Our backing just caved in. DARPA finally notified me, in 1977 I think, we're cancelling your funding.

DDJ: When did you start working on NLS?

DE: We started to get some money to actually build things in the very early '60s sometime. We went through two different machines.

DDJ: Did you build your own machines?

DE: No. But we had to make our own display systems with the first four early experimental computers. The first machine was a CDC 160A. The next one was a CDC 3200 or something like that. That's the display system [that] we then did the screen-selection experimenting on -- when the mouse got born. It cost us $80,000 to make it and the interface to the 3200.

DDJ: What programming language were you using to develop the system?

DE: Well, we wrote our own assemblers at first. Fortran was already available, but it was too clumsy for what we were trying to do. Then, some of our programmers started saying, look, we're learning about compiler compilers, so there soon was an embryonic compiler compiler, at least we can define our own compiler. And look what else it will do for us. That's when we started developing our own command-language interface system. So, over the two decades since the late '60s, everything was written in our own multilevel languages. And it's all still running, on a DEC-20 emulator.

DDJ: How many people did you have working on NLS's software?

DE: Let's see, at the peak we were 47 people and I think there were, like, 18 of them who were building the software. In '74, we started actually being in the business of selling support service to anybody on the ARPAnet, where people could start setting up their own way of working. So, they'd get a package of electronic mail (they called it), word processing, and collaboration software, the internal system, and all of that. The e-mail was all interlinkable with anything else, and everything was hyper-link citeable down to the character. There were a lot of things in there that have recently gotten people seriously interested about building a community of participants to evolve what we call a scalable, Open Hyperdocument System.

DDJ: You're concerned with solving complex problems. Developing software is clearly a complex problem. Did you find yourself developing new techniques and applying your ideas to the development of your software?

DE: Basic ideas like citation linkage that would point to any file object, that was there since the early '60s. And also the basic idea of optional view specifications as part of a link's syntax. And then the idea of providing for different grades of user capability, actually providing different interfaces, so that you get your own interface, so that the people that wanted to learn to use what you could call a "big workshop vocabulary," could actually do that; but also, people could start out with a very simple one. Basic architectural things like these were put in there by 1970.But that was just basic. There were also special applications for manipulating files, and some especially for programming. The source-code files stayed is the NLS basic environment for doing all the studying, searching, editing, moving around, when you wanted to compile it or check things or do something special, you'd just call in a special application. The software's all hyperlinked and structured.

DDJ: The source code is all hyperlinked?

DE: Yes. And in any comment field, you could put a link pointing to anyplace else. Every e-mail message is also targetable. If somebody comes in from the field with a trouble-call message about a problem, that will become interlinked with the following analysis, bug fix, change to requirements, fixup of documentation, etc. Or they could say, "Hey, Fred, why don't you look into that?" So they'd come back then saying, "Okay, obviously this part of the software has a problem, and it looks like it probably is something where the requirements might need to be changed." And then, there's a view you can take of things, a filtered view, and say, "Okay, it's a bug. So it wasn't there three months ago, so let's just ask for all the statements that have been changed in the last three months." Well, that's one of the viewing options. So you'd sort of look at that. Another part of the embedded properties is who did the last change to a given statement. "Oh, Fred, was in there." All we have to do is just say, "What did Fred change?" and we're going to find the bug. [laugh] Which was one of the jokes. But anyway, all of those things were just part of the daily operations, and they all evolved during that time.

DDJ: Had you come up with things like hypertext, backlinking, change management, the outlining views, all that stuff, before you built the display systems for the machine? Or did you build the display and think, "Oh wait, now I can do some of this, now I can do some of that."

DE: The whole orientation from 1951 on was display interaction. The very first things we built were to do the editing and the viewing and the linking. We knew that there were a lot of surprises in store for us when we really started to harness this business, and we knew it would be very handy to start looking at some of those, especially the ones that would make the most difference to our ongoing work. Things like every object being addressable, viewing options being flexibly controlled, hierarchical file structuring, they just came out intuitively from the "look for surprises" part of the scaling study. So okay, how do you get started in that? Well, you've got to learn how to work in a very different, augmented way; so when you do, the hierarchy and the addressability and the optimal views and the open-ended vocabulary just all emerge.

DDJ: So we talked about a lot of what was motivating you, what you were working on, a lot of the stuff that you developed, and the tools to help address this problem, but we haven't really talked about what the grand solution actually is. What is the solution that you're proposing for augmenting the human intellect?

DE: Because it's such a big scale thing, you have to find an appropriate evolutionary environment and get that established. There are a lot of things that need to coevolve, so what's an appropriate environment to support that? And as you're learning how to do better at collective work, you'll have choices about where you're going to apply your early gains, because you're not just going to be able to lift up all of society, boom, like that. Say, "Oh, look, what if, as we paid early, special attention to learn how to be more effective collectively, we applied as much of the new capability as possible to improving selected parts of society's improvement infrastructure." From the early '60s, I'd been calling that "bootstrapping." The better that gets, the better we're going to improve all the other things we want to improve, including how you're going to improve improvement.

DDJ: Over the past 50 years, just to have been able to watch how technology and society have evolved must just be extraordinary in and of itself. But you've not only been able to watch this, you've seen some of your ideas actually become accepted and become mainstream. Is it frustrating to have to wait so long for a certain idea to take off and to see things being reinvented? What is it like to have an office at Logitech and look around and see a mouse on every desktop? [Editor's Note: Mouse-maker Logitech generously provides office and research space for the Bootstrap Institute.]

DE: It's funny. Sometimes I wonder why it doesn't turn me on more. It's mainly that the thing is just dominated by, damn it, how can we get the world to really go after it? It's hard to take seriously so much. It's like saying, really, I'm a failure.

DDJ: You mean people aren't thinking big enough?

DE: Right. I really feel like the kinds of problems, the complexity the world's getting into, are such that if we don't really get going on something explicit about bootstrapping, we could just see the whole human race tripping and sprawling.

DDJ: Does it bother you that you're so closely associated with things like the mouse as opposed to some of these broader ideas?

DE: Yes, I guess it does. I'm bothered that the world doesn't seem to see the potential and decide, let's go after it.

DDJ: Do you have better success today selling your ideas to people?

DE: I get them to listen longer. [Laugh] I keep thinking, gee, maybe we can get moving now.

DDJ: Do you think that's because we've got this paradigm shift going on right now?

DE: Yeah. The experience people have with the Web moves their paradigms. It's really interesting how much it is that the current prevailing cutting-edge technology is sort of implicitly held as if people weren't there? We've still got a long way to go.

DDJ: What is the Bootstrap Institute?

DE: Well, it's an entity that was created about 10 years ago to be a platform for launching Collective IQ bootstrapping. And then the Bootstrap Alliance emerged a few years ago as a specifically not-for-profit envelope for Network Improvement Communities (NICs), organizations focusing on improving their improvement process. These NICs are an important part of the evolutionary improvement infrastructure that is a central part of the bootstrapping strategy.

DDJ: And is one of the purposes for the Bootstrap Institute to build the Open Hyperdocument System?

DE: For fifteen or more years, it has been clear to me that an OHS -- Open Hyperdocument System -- has to emerge as a common basis upon which to build the world's evolving knowledge base: handling every kind of knowledge domain, with embedded properties and structural conventions whose nature and usage has yet to evolve. And maximum-function interoperability has to prevail. The emergence of XML provides a candidate growth base for content form, but that has to coevolve with how people are going to learn to manipulate and view.

I've been talking here about coevolution and such as optimal evolutionary environments. That's because the target capabilities are going to involve changing many many of the ways we couple our basic human sensory, perceptual, cognitive, and motor capabilities into the augmented way of doing our knowledge work. And since I don't believe that any person or group can possibly be smart enough to design the best end system that we should use, I favor an approach [that] seeks the best large-scale evolutionary process.

And so, launching an OHS Project along with a proactive user community seems like the only way to get going on creating this large-scale evolutionary process. So, we have crafted a prototype model, with an open-source approach, and a rough-sketch evolutionary path involving a succession of expanding user communities.

DDJ: Do you have any final thoughts?

DE: That's the direction, world, why aren't you going after it? Here's an important grand challenge, and if you go after it, it will make the capability of dealing with that other grand challenge all the better. So why wouldn't this be about the best grand challenge that you could start out with? -- improving your capability to deal with grand challenges.

DDJ