A persistent controversy in language evolution research has been whether language emerged in the gestural-visual or in the vocal-auditory modality. A “dialectic” solution to this age-old debate has now been gaining ground: language was fully multimodal from the start and remains so to this day. In this paper, we show this solution to be too simplistic and outline a more specific theoretical proposal, which we designate as pantomime-first. To decide between the multimodal-first and pantomime-first alternatives, we review several lines of (...) interdisciplinary evidence and complement it with a cognitive-semiotic experiment. In the study, the participants saw – and then matched to hand-drawn images – recordings of short transitive events enacted by 4 actors in two conditions: visual and multimodal. Significantly, the matching accuracy was greater in the visual than the multimodal condition, though a follow-up experiment revealed that the emotional profiles of the events enacted in the multimodal condition could be reliably detected from the sound alone. We see these results as supporting the proposed pantomime-first scenario. (shrink)

The role of figurative languages such as metaphor and metonymy in creativity has been studied in cognitive linguistics. These methods can also be applied to analyze non-linguistic data such as pictures and gestures. In this paper we analyze fashion design by focusing on visual metaphor and metonymy. The nature of creativity in fashion design is not fully studied from a cognitive perspective compared to other related fields such as art. We especially focus on the aspect of fashion design as a (...) communication tool between the designer and the audience to convey designer’s image of human. Photos from two fashion shows are analyzed. We carried out an experiment to compare how human images in two shows are interpreted by those who are familiar with fashion and those who are not. We got three results: (1) As far as figurative (metaphorical and metonymic) interpretations of human image are concerned, two groups with different familiarity to fashion have significantly different patterns of responses to two shows. (2) For non-figurative interpretations (such as physical or personal attributes), no significant difference in the pattern of response to the show was observed between two groups. The participants as a whole, however, responded to the two shows differently. (3) In addition for non-figurative interpretation, fashion experts found significantly more attributes in human images than the others. The results show that the analysis of figurative interpreations is effective in seeing how the familiarity to fashion affects the mode of seeing fashion shows. (shrink)

Many types of everyday and specialized reasoning depend on diagrams: we use maps to find our way, we draw graphs and sketches to communicate concepts and prove geometrical theorems, and we manipulate diagrams to explore new creative solutions to problems. The active involvement and manipulation of representational artifacts for purposes of thinking and communicating is discussed in relation to C.S. Peirce’s notion of diagrammatical reasoning. We propose to extend Peirce’s original ideas and sketch a conceptual framework that delineates different kinds (...) of diagram manipulation: Sometimes diagrams are manipulated in order to profile known information in an optimal fashion. At other times diagrams are explored in order to gain new insights, solve problems or discover hidden meaning potentials. The latter cases often entail manipulations that either generate additional information or extract information by means of abstraction. Ideas are substantiated by reference to ethnographic, experimental and historical examples. (shrink)

How might a human communication system be bootstrapped in the absence of conventional language? We argue that motivated signs play an important role (i.e., signs that are linked to meaning by structural resemblance or by natural association). An experimental study is then reported in which participants try to communicate a range of pre-specified items to a partner using repeated non-linguistic vocalization, repeated gesture, or repeated non-linguistic vocalization plus gesture (but without using their existing language system). Gesture proved more effective (measured (...) by communication success) and more efficient (measured by the time taken to communicate) than non-linguistic vocalization across a range of item categories (emotion, object, and action). Combining gesture and vocalization did not improve performance beyond gesture alone. We experimentally demonstrate that gesture is a more effective means of bootstrapping a human communication system. We argue that gesture outperforms non-linguistic vocalization because it lends itself more naturally to the production of motivated signs. (shrink)