recognized prominence of Cornell’s program in information theory.
Interestingly, at IBM, Jelinek got the opportunity to pursue research in the two fields of most interest to him—linguistics and information theory—through the challenge of designing an automatic typewriter that would respond to the human voice. He saw speech recognition as an information theory problem, rather than the traditional view that speech recognition could (or to some, should) be solved using fundamental principles in acoustics and/or linguistics (or more likely a combination of the two technologies). As an admirer of Claude Shannon, the “Father of Information Theory,” Jelinek’s approach followed the fundamental teaching of the “Shannon game,” in which a person competes with a computer, which keeps an array of conditional probabilities that define the odds of a given word following a specified sequence of words, in completing a partial sentence.
Information theorists believe that a machine that chooses the most probable word to succeed the three (or more) preceding (already seen) words would be more statistically likely than the guess made by a human, thereby winning the game. The machine need not actually understand or guess the meaning the sentence really intends to express. The computer-selected word is thus based on the so-called n-gram statistical grammar commonly used in speech recognition systems today. With such types of grammar as the discrete source models, Jelinek treated the speech recognition problem, or rather the natural language processing problem, as a noisy-channel discrete decoding problem and advocated the method of maximum likelihood decoding. This statistical approach to linguistics, built on the framework of information theory, stayed in the course of the axiomatic probability theory, avoided the mathematically intractable problem of human intelligence, but clashed with the conventional discipline of linguistics.
Jelinek was well known for his wry humor, most especially this oft-quoted quip: “Every time I fire a linguist, my payroll goes down and the performance of my speech recognizer goes up.” Jelinek’s engineering results, as manifested in systems