“The Pervasiveness of Technology Degrades Personal Responsibility”

Joseph Weizenbaum
Joseph Weizenbaum

Mr. Weizenbaum, What concerns do you have regarding a world of widely networked computers?

Weizenbaum: Pervasive computing is a condition, just like heat in Africa. No one planned it, there was no conference to decide upon it, and no one can say, “We’re getting rid of it.” The condition has grown, just like we use automobiles today as a matter of course. But even with this example, you can ask yourself if that makes sense, given the traffic jams, exhaust, and use of oil resources. Today, many people use a huge number of computers – many of them networked – with exactly the same lack of reflection. This condition has consequences: many of them a blessing, others the opposite.

I see a certain analogy to the telephone there. At the beginning, there were doubts about ever being able to do anything useful with it. The ability to call everyone is not a convincing argument – at least not then, when there was a total of only 200 telephones. The actual benefit of the telephone is that it is everywhere today: it’s pervasive. Conditions with the PC are similar. We’re at the threshold of pervasiveness; PCs are spread widely and are often networked. That changes our society. By that I mean the pressure that burdens us all to react very quickly. For example, an e-mail demands a quick reaction.

I always thought that the advantage of e-mail was that I didn’t have to react to it immediately.

Weizenbaum: E-mail is another example of the condition of expecting a quick reaction. I’m not complaining about e-mail here. It’s just a metaphor for the speed – or lack of an ability to pause – that we’ve created. The ability to pause: that’s the core of the matter. Everyone can be reached at any time. The entire world speeds up, everything is urgent. And when everything is urgent, nothing is urgent, and thus we slither into meaninglessness.

Is that a fundamental criticism of the economizing of the world, that we also subordinate our private life to considerations of efficiency?

Weizenbaum: The short answer is “yes.” Let’s say I want a cell telephone. My position in society “forces” me to have one. I could rebel, but that would exclude me from the mass society. Now, someone could say that that’s not the fault of technology and computers. And it’s not, of course. Computers have no guilt. It’s the ease with which we slip into this condition. And that’s irresponsible.

Do you see a degradation of personal responsibility because of pervasive computing?

Weizenbaum: Yes, and I have a prime example of it. In the 1980s, the American stock market collapsed after an electronically induced market crash. The market lost a huge amount of value. Why? Systems like that of the stock market are unstable. Consider the example of a sailboat. The greater the tilt of the boat as it goes into the wind, the smaller the working surface of the sail, and the more that the weight of the boat lifts the sail. The market system does not have a centerboard or a way to stabilize itself. It can tilt; it is unstable. No one had thought of this before, and then it simply occurred. Now, the main questions are: Who “manufactured” the market system? Who “discovered” it? Who’s responsible? Who can stop it? And the answer is “no one.” No one is responsible. And there’s no off switch.

Do you fear a new collapse?

Weizenbaum: I can’t say that. It wasn’t a computer system that collapsed; there was no computer error. But the loss of control nonetheless did a great deal of harm. And if various systems are networked over computers, the entire network becomes all the more sensitive. It always takes a smaller means to create immense harm. It takes only a little bit of effort to create damage; the power to destroy has become cheaper. Less efficient systems would also be less sensitive.

In my opinion, all this is the result of our naïve interaction with technology. I mean the erroneous belief that the answer to defective technology is more technology. Better technology makes systems even more complex and more unmanageable. I think that companies like Microsoft and SAP know that very well. If you find an error, you find a patch. In itself, the patch can be completely free of errors. Nonetheless, the patch can lead to unforeseeable reactions because of its interplay with the rest of the software.

In this context, what do you think of a vision of the future that includes things like an intelligent refrigerator that always knows what’s in it?

Weizenbaum: That’s nonsense! If I had a real enemy, I’d send him that kind of a refrigerator as a gift. This case once again raises the question: Do you need this technical bauble? Obviously, we don’t. And think about what such a system would replace. Someone who opens the refrigerator and says, “Hey! We need milk!” or who throws out an open container before it spoils. Someday I’ll come home after a weekend away and find spoiled milk in my refrigerator. The computer acted logically and did not remove the container from the refrigerator. It might even have issued a polite warning. Once again, no one did anything wrong, but damage has nonetheless been done.

So you don’t believe that such a high-tech system can function?

Weizenbaum: No, I don’t. It’s all about an interaction between a human being and a machine. High tech doesn’t work there. Take a microwave oven, for example. You can set the starting time, the cooking time, and the temperature. When my wife used to go out to play tennis, she told me to program the microware in such and such a way, and to turn off the timer two hours later. I told her that she could do it herself, but she replied that it was too complicated for her.

I think that in the world of technology, there are often cases where technical instruments are not used at all. A few days ago I read an article in the newspaper about how many electronic devices were offered for auction on eBay: scanners, computers, and cameras. First, that shows that electronic devices sell well – how else could they wind up on eBay? Second, however, it shows that ultimately, people don’t use these devices. There are two possible reasons for that: either they don’t really need the devices, or the devices are too complicated for their users.

What advice would you offer someone who works as a developer at SAP?

Weizenbaum: My advice can be summarized in one word: slower. Companies should speak about responsibility in their advertisements – something like a two-page spread in large-circulation newspapers with a headline: “It takes longer with us.” I mean that in all seriousness.

People want solutions to handle specific tasks. But always faster and cheaper? That leads to a situation in which everything must happen immediately. That approach presses everyone for time so that promises are made that can’t be kept.