Joseph Weizenbaum On Tools

Posted 3/7/2015

Joseph Weizenbaum, one of the great computer scientists of the mid-twentieth century (and author of the instantly-famous ELIZA program), in 1976 wrote Computer Power and Human Reason: From Judgment to Calculation, a brilliant examination of the impact of computers on society and human thought.

In his first chapter, “On Tools,” he examines the impact which tools have on the processes they are used on. The way in which man does something, he says, is intimately tied to how he does it. “Man can create little without first imagining that he can create it” (pg. 18). Tools shape the imaginations of their users, including creating the belief that the tools are “necessary” to the processes that machines aid. In the same way, Weizenbaum argues, the computer became “necessary” the tasks it was used for.

When the first telegraph line connecting Texas with New York was laid doubts were expressed as to whether the people in those places would have anything to say to one another. But by the time the digital computer emerged from university laboratories and entered the American business, military, and industrial establishments, there were no doubts about its potential utility . . . Unprecedentedly large and complex computational tasks awaited American society at the end of the Second World War, and the computer, almost miraculously it would seem, arrived just in time to handle them. (pg. 27)

The indispensability of computers is in many ways a mirage created by the existence of computers. “Only rarely,” he says, “if indeed ever, are a tool and an altogether original job it is to do, invented together” (pg. 32). The impact of computers on society, he says therefore, is not that they transformed what we do, but that they transformed how we do them. “If the triumph of a revolution is to be measured in terms of the profundity of the social revisions it entailed, then there has been no computer revolution” (pg. 32).

In perhaps the most illuminating passage of all, he argues that this transformation of method can actually prevent work on the real problem from being accomplished:

The capacity of the human mind for sloppy thinking and for rationalizing, for explaining away the consequences of its sloppy thinking, is very large. If a particular technique requires an enormous amount of computation and if onlyk a limited computational effort can be devoted to it, then a failure of the technique can easily be explained away on the ground that, because of computational limitations, it was never really tested. The technique itself is immunized against critical examination by such evasions. Indeed, it may well be fortified, for the belief that an otherwise faultless and probably enormously powerful technique is cramped by some single limitation tends to lead the devotee to put effort into removing that limitation. When this limitation seems to him to be entirely computational, and when a computer is offered to help remove it, he may well launch a program of intensive, time-consuming “research” aimed simply at “computerizing” his technique. Such programs usually generate subproblems of a strictly computational nature that tend, by virtue of their very magnitude, to increasingly dominate the task and, unless great care is taken to avoid it, to eventually become the center of attention. As every more investment is made in attacking these initially ancillary subproblems, and as progress is made in cracking them, an illusion tends to grow that real work is being don on the main problem. (pgs. 35-36)

Additionally, he argues, all tools presuppose a “language,” and themselves constitute a kind of “language.” Because a tool can only be used for certain things, and is manifestly inadequate for others, it naturally dictates, by its nature, what sort of problems are fed through it. The conclusion we can draw from this idea, related to the modern truism of computers as the predominant processors of information, is apparent. Computers, because they are the dominant means of processing information, have also become the definition and the language of “information processing”, and indeed, of thought itself. “A computing system that permits the asking of only certain kinds of questions, that accepts only certain kinds of ‘data’, and that cannot even in principle be understood by those who rely on it, such a computing system has effectively closed many doors that were open before it was installed” (pg. 38).