Textbook of AI

Hearsay-II was the speech-understanding system developed at Carnegie Mellon University from 1971 to 1976 (with a final summary paper by Erman, Hayes-Roth, Lesser and Reddy in ACM Computing Surveys, 1980), under the DARPA Speech Understanding Research (SUR) programme. The five-year, multi-million-dollar SUR effort had set demanding targets, continuous speech, 1000-word vocabulary, multiple speakers, near-real-time response, less than 10% error, and Hearsay-II was the system that came closest to meeting them. Applied to a chess-position-naming task and a 1011-word document-retrieval task, it achieved roughly 90% sentence-level accuracy.

The blackboard architecture

Hearsay-II's lasting contribution to AI was the blackboard architecture, a problem-solving paradigm in which several independent knowledge sources cooperate by reading from and writing to a shared global data structure (the blackboard), with a separate scheduler (or control component) deciding whose turn it is to act. In Hearsay-II the knowledge sources spanned multiple linguistic levels:

• acoustic (raw signal → parameters)
• phonetic (parameters → phoneme hypotheses)
• lexical (phoneme strings → word hypotheses)
• syntactic (word sequences → grammatical phrases)
• semantic (phrases → meanings consistent with the task domain)
• pragmatic (meanings consistent with task context and discourse)

Each knowledge source watched the blackboard for hypotheses at its input level, posted refined or new hypotheses at higher levels, and could revise or rate hypotheses already present. Crucially the architecture supported both bottom-up (data-driven) and top-down (expectation-driven) processing within the same control loop, allowing, for example, a high-confidence syntactic prediction to direct a renewed phonetic search in a noisy region of the signal.

The architecture was a major paradigm of multi-agent AI for two decades, with descendants including HASP/SIAP (sonar signal interpretation), OPM (multi-agent planning), GBB (a generic blackboard toolkit), and aspects of the DVMT distributed sensor-network testbed. Echoes of the blackboard idea persist in the modern agentic multi-LLM systems that maintain a shared scratchpad / message buffer.

People and lineage

Hearsay-II's principal architects were Lee Erman (lead developer), Frederick Hayes-Roth, Victor Lesser, and Raj Reddy. Reddy went on to win the 1994 ACM Turing Award (with Edward Feigenbaum), partly for the Hearsay/SUR work. The system's specific speech-understanding techniques were superseded by hidden-Markov-model approaches in the 1980s (BBN BYBLOS, IBM Tangora, CMU Sphinx, also from Reddy's group), and those in turn by deep-neural-network acoustic models from 2010 onwards (Hinton, Deng et al.) and the modern wav2vec / Whisper / Conformer family. The architectural ideas, heterogeneous knowledge sources, asynchronous cooperation, mixed bottom-up / top-down control , remain influential.