Multimodal Cognitive Architecture: Making Perception More Central to Intelligent Behavior


B. Chandrasekaran
The Ohio State University

I propose that the notion of cognitive state be broadened from the current predicate-symbolic, Language-of-Thought framework to a multi-modal one, where perception and kinesthetic modalities participate in thinking. In contrast to the roles assigned to perception and motor activities as modules external to central cognition in the currently dominant theories in AI and Cognitive Science, in the proposed approach, central cognition incorporates parts of the perceptual machinery. I motivate and describe the proposal schematically, and describe the implementation of a bi-modal version in which a diagrammatic representation component is added to the cognitive state. The proposal explains our rich multimodal internal experience, and can be a key step in the realization of embodied agents. The proposed multimodal cognitive state can significantly enhance the agent's problem solving.

B. Chandrasekaran is Professor Emeritus Computer Science and Engineering and Director of the Laboratory of Artificial Intelligence Research in the Department of Computer Science and Engineering at The Ohio State University. He is a Fellow of the Institute of Electrical and Electronic Engineers, Association for Computing Machinery and American Association for Artificial Intelligence. His major research activities are in diagrammatic reasoning, causal understanding, knowledge systems, decision support architectures and cognitive architectures. He and David Brown authored "Design Problem Solving," (Morgan Kaufmann), and he is co-editor of Diagrammatic Reasoning: Cognitive and Computational Perspectives (MIT Press). Chandrasekaran was Editor-in-Chief of IEEE Expert/Intelligent Systems from 1990 to 1994. He is currently a technical leader in an ARL-supported Government-Industry-University Collaborative Technology Alliance on Advanced Decision Architectures.