Posted on behalf of Lars Wood
The literature of artificial intelligence generally fails to distinguish different classes of machine reasoning. The explicit assumption, made originally by Paul Armer of the RAND Corporation, that all intelligent behavior is of the same general type, has encouraged workers in the field to generalize and not distinguish between what is now known to be brute force computer “information processing” and what is human-like machine intelligence. This confusion has had two dangerous consequences. First there is the tendency, exemplified by Herbert Simon, to think that heuristics discovered in the field of computer information processing, such as theorem proving, must tell us something about true machine intelligence, such as machines capable of human perception and insight. The second result is the problem of exponential paradigm growth that ultimately causes failure when the techniques of brute force computer information processing are sophistically extended to implement human intelligent machines.
In this context it is helpful to distinguish categories of intelligent activity. We can then determine to what extent intelligent behavior in each category presupposes human intelligence. This enables us to account for what successes have been attained and predict what further progress can be expected in a particular category. One can delineate and distinguish four categories of human intelligent activity. The first three categories are amenable to brute force computer information processing, while the fourth is intractable using these brute force methods.
Categories of Machine Reasoning and Learning
Category I, the lowest, is where the situation-response psychologists are most at home. It includes all forms of elementary associationistic behavior where meaning and context are irrelevant to the activity concerned. Rote learning of non-sense syllables is the most perfect example of such behavior so far programmed, although any form of conditioned reflex would serve as well. Also some games, such as the game sometimes called Geography (which simply consists of finding a country whose name begins with the last letter of the previously named country), belong in this area. In language translating, this is the level of the mechanical dictionary; in problem solving, that of pure trial-and-error search routines; in pattern recognition, matching patterns against fixed templates.
Category II encompasses the conceptual rather than the perceptual world. Problems are completely formalized and completely calculable. For this reason, it might best be called the area of the simple-formal. Here “artificial intelligence” as found and advertised by various digital game companies in their products are possible in principle and in fact. In Tier III, natural language is replaced by a formal language, of which the best example is logic. Games having precise rules that can be calculated out completely, as in the case of nim or tic-tac-toe are examples. Pattern recognition on this level takes place according to determinate types, which are the class in question. Problem solving takes the form of reducing the distance between means and ends by repeated application of formal rules. The formal systems in this area are simple enough to be manipulated by algorithms that require no search procedure at all (for example, Doom or Wang’s logic puzzle). Heuristics are not only unnecessary here, they are a positive handicap, as the superiority of Wang’s algorithmic logic program over Newell, Shaw, and Simon’s heuristic logic program demonstrates. In this tier of machine artificial intelligence, academia and industry have had their only true unqualified successes.
Category III, what we shall define as complex-formal systems, is the most difficult to define and has generated most of the misunderstandings and difficulties in the field of artificial intelligence and machine learning. It contains behavior that is in principle formalizable yet is in fact exponentially intractable. As the number of elements increases, the number of transformations required grows exponentially with the number of elements involved. As used here, “complex-formal” includes those systems that in practice cannot be dealt with by exhaustive enumeration of algorithms (chess and GO etc.) and thus require heuristic programs, i.e. using a problem-solving technique in which the most appropriate solution of several found by alternative methods is selected at successive stages of a program for use in the next step of the program by an algorithmic “rule of thumb” principle, which generally has wide problem domain application but is not generally universal or intended to be strictly accurate or reliable in every situation.
Category IV might be called the area of nonformal behavior. This includes all those everyday activities in our human world that are regular yet not governed by rules. The most striking example of this controlled imprecision is our disambiguation of natural languages in the context of understanding. Pattern recognition in this domain is based on recognition of the generic, or of the typical, by means of a paradigm case. Problems on this level are open-structured, requiring a determination of what is relevant, and insight into which operations are essential, before the problem can be addressed.