As a first step in involving user emotion in human‐computer interaction, a memory‐based expert system (JANUS; Kearney, 1991) was designed to interpret facial expression in terms of the signaled emotion. Anticipating that a VDU‐mounted camera will eventually supply face parameters automatically, JANUS now accepts manually made measurements on a digitized full‐face photograph and returns emotion labels used by college students. An intermediate representation in terms of face actions (e.g., mouth open) is also used. Production rules convert the geometry into these. (...) A dynamic memory (Kolodner, 1984; Schank, 1982) interprets the face actions in terms of emotion labels. The memory is dynamic in the sense that new emotion labels can be learned with experience. A prototype system has been implemented on a Sun 2/120 system using POPLOG. Validation studies on the prototype suggest that the interpretations achieved are generally consistent with those of college students without formal instruction in emotion signals. (shrink)