Artificial emotional intelligence beyond East and West

Daniel White, Department of Social Anthropology, University of Cambridge, United Kingdom
Hirofumi Katsuno, Department of Media, Journalism, and Communications, Doshisha University, Kyoto, Japan

PUBLISHED ON: 11 Feb 2022 DOI: 10.14763/2022.1.1618

Abstract

Artificial emotional intelligence refers to technologies that perform, recognise, or record affective states. More than merely a technological function, however, it is also a social process whereby cultural assumptions about what emotions are and how they are made are translated into composites of code, software, and mechanical platforms that operationalise certain models of emotion over others. This essay illustrates how aspects of cultural difference are both incorporated and elided in projects that equip machines with emotional intelligence. It does so by comparing the field of affective computing, which emerged in the North-Atlantic in the 1990s, with kansei (affective) engineering, which developed in Japan in the 1980s. It then leverages this comparison to argue for more diverse applications of the culture concept in both the development and critique of systems with artificial emotional intelligence.
Citation & publishing information
Received: September 17, 2021 Reviewed: December 12, 2022 Published: February 11, 2022
Licence: Creative Commons Attribution 3.0 Germany
Funding: Support for this research has been provided by Freie Universität Berlin, Doshisha University, the University of Cambridge, and a JST RISTEX Grant (number JPMJRX19H5).
Competing interests: The author has declared that no competing interests exist that have influenced the text.
Keywords: Artificial intelligence, Affective computing, Emotion, Engineering
Citation: White, D. & Katsuno, H. (2022). Artificial emotional intelligence beyond East and West. Internet Policy Review, 11(1). https://doi.org/10.14763/2022.1.1618

This article belongs to Concepts of the digital society, a special section of Internet Policy Review guest-edited by Christian Katzenbach and Thomas Christian Bächle.

1. INTRODUCTION

There is a robot in Japan called LOVOT that is designed, so its makers advertise, ‘to be loved by you’ (Groove X, 2019, n.p.). To facilitate this, its engineers at its parent company Groove X have equipped LOVOT with trademarked ‘emotional robotics’, which refers to ‘a robot technology that was created to stir people’s feelings, through the ways that the robots look, feel and behave’ (Groove X, 2019, n.p.). LOVOT is small, soft, furry, and is designed to be about the same weight and temperature of a human baby. It can roll around on wheels that fold into its body when it is picked up and held. Tactile sensors on its body register human-robot interactions that are recorded in a ‘Diary’ accessible by a smartphone application for other registered users—such as a curious parent, for example—to track. Through a ‘horn’ on its head, which hosts a video camera and processor, LOVOT can map its surroundings, navigate obstacles, and record up to 1,000 distinct human faces, as well as facial expressions which signal different emotions—although this function has not yet been activated according to the authors’ conversations with company staff. Based on interactions with users, the robot can also assign values to people and rank them according to a hierarchy of ‘preference’: those users offering the most positive interactions, such as through regular displays of tactile affection, receive the highest rating. Through these mechanisms and others, LOVOT exemplifies what many robot researchers, engineers, and marketers call ‘artificial emotional intelligence’.

What do emotions become when transcribed into digital platforms? Evaluating the emergence of new technologies, algorithms, and digital platforms incorporated into machines like LOVOT requires tracing the processes by which the definitions, meanings, and significance of emotional experience change when emotions are converted into meanings that can be processed in digital form. Adding a critical perspective to this process is important given the degree to which such technologies can mislead users on the accuracy and purported universality of their emotion-recognition abilities. Although the quantification of emotion has a long history (Bollmer, 2019; Crawford, 2021; Lupton, 2016; Wilson, 2010), more recent are practices of digitalising emotion that combine smart cameras, social robots, wearable devices, and other technologies with machine learning and algorithmic forms of analysis. This approach to interpreting emotion through quantitative metrics, combined with increased computing potential, has made the concept of ‘artificial emotional intelligence’ into a powerful sociotechnical tool. It is for this reason that we aim to contribute considerations of cultural diversity that can make the term into a powerful critical tool as well.

‘Artificial emotional intelligence’ is an umbrella concept used by digital technology developers and researchers to designate technologies estimated by engineers or users to have emotional capacities. Defined by Andrew McStay (2018, p. 3), ‘artificial emotional intelligence’ can be understood by the ‘capacity to see, read, listen, feel, classify and learn about emotional life’. It can incorporate ‘reading words and images, seeing and sensing facial expressions, gaze direction, gestures and voice. It also encompasses machines feeling our heart rate, body temperature, respiration and the electrical properties of our skin, among other bodily behaviors’ (McStay, 2018, p. 3).

In this essay we build on this definition as a constructive critical starting point. Most importantly, because different groups of people disagree on what emotions are, how they work, why they matter, and even how they physically feel, we argue that there is good reason to incorporate into this definition a culturally diverse perspective in order to better evaluate the significance as well as the threats posed by the rise of technologies with emotional capacities. This task is even more critical when one considers how different groups of people approach not only emotional experiences in various ways but also the very technologies, such as LOVOT, that mediate them. For this reason, we focus our attention on some ways that emotion has in different cultural contexts become differently digitalised through emerging technologies equipped with artificial emotional intelligence. We do so not to establish a fixed definition of the term ‘artificial emotional intelligence’ that can be uniformly or universally applied across comparable contexts. However, neither do we propose to substitute for this universal approach an equally simple one of cultural differences, such as those between a so-called ‘East’ and ‘West’. Rather, we aim to broaden the meaning and critical acuity of the term ‘artificial emotional intelligence’ to better encapsulate the diversity and complexity of the cultural conditions under which emotion and technology are increasingly combined today. Writing from the perspective of cultural anthropology and media studies, we argue that while economic structures underlying the development of emotional technologies incentivise engineers to build universal models of emotion recognition, the ethnographic record demonstrates a diversity of emotional experience that proves more difficult to capture through code.

In the following sections we draw out the importance of this dialectic between universal and particular models of emotion by, first, summarising the recent historical context for efforts in measuring emotion in digital landscapes (section 2); second, describing the rise of the field of affective computing that has come to dominate artificial emotion research largely in anglophone cultural contexts (section 3); third, considering alternative approaches to artificial emotional intelligence, such as those from Japan, where we conduct ethnographic fieldwork on emotional robotics and affective engineering (kansei kōgaku) (section 4); and finally, advocating for the diversification of the concept of ‘culture’ itself so that developers might better incorporate aspects of cultural diversity into emotional technology design and researchers might further refine their critique of emotional technology development (section 5).

2. KEY CONCEPTS AND HISTORICAL CONTEXT

The question of whether manufactured objects can perform, understand, or even ‘have’ emotions is an old theme in the diverse literary and philosophical narratives of artificial intelligence (see Cave et al., 2020). Many of these narratives draw heavily from even earlier efforts to formalise a psychological science of emotion. In the late nineteenth century, for example, from naturalists such as Darwin to early neurologists like Guillaume Duchenne, a broad range of researchers combined an analytical view of an evolving natural science with emerging media technologies. Using tools such as illustrated and, later, photographic ‘books of faces’ (Bollmer, 2019), European and North American scientists applied an experimental lens to interpret the philosophical puzzle of the body’s affective states. These precedents established a sociotechnical legitimacy around using the face and body to decode emotion. With later technological developments that enabled digital data collection, such decoding processes could be easily incorporated into an accelerating technological science of emotion detection. This historical process highlights the increasing importance today of rapidly advancing practices of datafication and digitisation—concepts that are critical to our analysis and covered earlier in this series.

According to Viktor Mayer-Schönberger and Kenneth Cukier (2013, p. 78), ‘datafication’ refers to the process of putting data into a ‘quantified format so it can be tabulated and analyzed’. This refers as much, the authors discuss (2013, pp. 76–80), to US Navy officer Matthew Maury’s catalogue of sailing data in the mid-nineteenth century that radically rationalised marine navigation as it does to engineer Koshimizu Shigeomi, who collected 360 points of pressure data from car seat sensors in order to produce a digital ID code for individual drivers.

The clarity and simplicity of this definition of datafication is helpful, but it also leaves out important social dimensions critical to the datafication of emotion in particular. As Ulises Mejias and Nick Couldry explain, ‘datafication has major social consequences’ (2019, p. 1), and incorporates ‘the wider transformation of human life so that its elements can be a continual source of data’ (p. 2). Built into technical processes of datafication, then, especially since the rise in scientific, corporate, and then home computing beginning most prominently in the 1950s, are social processes by which data is manufactured out of human interactions both with other humans and with emerging technologies like LOVOT. Even more importantly, these processes expand and accelerate practices of technological ‘enclosure’ (see Roquet, 2020) by which increasing aspects of daily life are rendered codable for machines.

Also accelerating processes of datafication are those of ‘digitisation’, which ‘turbocharge datafication’ by ‘turning analog information into computer-readable format’ (Mayer-Schönberger and Cukier, 2013, p. 85). Combined with the rapid acceleration of computer processing, mobile computing, and machine learning, digitisation incorporates datafication’s social emphasis on quantification with material technologies that automate processes of data processing and analysis.

The at once social and technological processes of datafication and digitisation enabled early work in computing and control systems that has come to be associated with the term ‘artificial intelligence’, and whose origins are often ascribed to the ‘The Dartmouth Summer Research Project on Artificial Intelligence’ in New Hampshire in 1956.1 The ascension of the term ‘artificial intelligence’ in English has foregrounded cognition in processes of representing human intelligence in machines, reinforced by the popularisation of Alan Turing’s 1950 paper ‘Computing Machinery and Intelligence’, from which is derived an enduring legacy that associates the measure of intelligence on a measurement of ‘thought’. However, other scientific traditions have just as readily proposed ‘emotion’ as an equally important marker of intelligence.

For example, when scientific literature on artificial intelligence began to enter Japan in the 1960s, because Japanese terms for ‘intelligence’ (chinō) and ‘mind’ (kokoro) refer symbolically to the heart as much as to the brain, the task of representing intelligence in machines in Japan had long been entangled with representing emotion (Katsuno and White, forthcoming 2022).2 Accordingly, because emotion was understood to be an embodied capacity, intelligence itself was understood to require a body to best represent it. This is why engagements with artificial intelligence in Japan have largely relied on concepts of ‘embodied intelligence’ and ‘embodied cognition’ (Robertson, 2018, pp. 82–86), and often gone hand in hand with the development of humanoid robotics.

When anglophone research began more explicitly engaging emotion with works like Rosalind Picard’s Affective Computing (1997) and Marvin Minsky’s The Emotion Machine (2006), coming arguably much later than in Japan,3 they were initially seen as exceptional and even marginal perspectives on intelligence. Such cultural differences in the approach to representing intelligence in machines suggests the important role that social context plays in the production not only of technologies that are manufacturing and collecting new forms of emotional data but also of the theories of emotion on which those technologies rely. To draw out the significance of certain cultural differences in defining artificial emotional intelligence, in the next two sections we compare the formation of the field of affective computing, originating mainly in North America and Europe, with approaches to emotional robotics and affective engineering (kansei kōgaku), which emerged in Japan. To reiterate, however, we set up this comparison in the next sections between a so-called ‘West’ and ‘East’ that is all-too-common in cultural scholarship on robotics in order to seek ways to better think beyond it in the final sections.

3. AFFECTIVE COMPUTING IN NORTH-ATLANTIC SCHOLARSHIP

In a 1995 working paper and later in a discipline-establishing book titled Affective Computing in 1997, the MIT computer scientist and entrepreneur Rosalind Picard argued that programmers, coders, and computer engineers need to ask questions about the relationship between emotions and computing. She proposed calling this field affective computing and summarised its principal concerns as the investigation and engineering of computers and software that can ‘recognize’ human emotion, can ‘express’ and perform emotion, and that can in some way even ‘“have” emotion’ (Picard, 1995, p. 1). In featuring these three objectives, Picard rendered the question of emotion as one that computer scientists could elucidate with their particular tool kit of coding, natural language processing, and automation—even to the extent of answering philosophical questions about the very nature of emotion.

In combination with the increasing ubiquity of social media platforms, as well as with wearable and mobile devices that offer a variety of physiological tracking capacities, computer scientists have recognised in emotional data an interest among manufacturers in potential profit generation and an appeal among consumers for tracking, self-development, and self-care. Accordingly, within the field of emotion research, there is an enormous incentive in terms of both monetisation and professional development to build systems that automate the detection of human emotion.

Such systems are applied to various uses today, albeit while also raising questions about their accuracy and legitimacy. For example, smart wristbands made by Picard’s company Empatica are designed to measure levels of anxiety and other affective patterns in the body through indicators like skin conductance and heart rate variability, as well as help ‘detect a possible tonic-clonic seizure’ for those who have epilepsy (Empatica, 2021). A software application by Affectiva called Affdex (another spin-off company from Picard’s lab) purports to record emotions through facial expressions. If advertisers want to know how consumers feel about the video content companies are producing, they can collect this data by willing participants. Finally and most controversially, similar interpretive methods of facial expression recognition have been employed by security personnel, such as in American airports in the wake of the 9/11 attacks on the World Trade Center (Crawford, 2021, p. 170). A version of facial expression and physiological recognition technology called ‘I-BORDER-CTRL’, made by European Dynamics, has also been tested at EU border gates to offer ‘lie-detecting avatars’ and ‘advanced analytics’ for ‘risk-based management’. Project summaries state that this ‘unique approach to “deception detection” analyzes the micro-gestures of travellers to figure out if the interviewee is lying’ (Boffey, 2018; European Dynamics, 2021; also see König, 2016 and Hall and Clapton, 2021).

Despite the impression projected by these technologies that they provide objective measures of internal feeling states, it is important to clarify that such technologies record only visible signs of such states. As psychologists like Lisa Feldman Barrett have made clear (2017; also see Le Mau, Hoemann and Lyons et al., 2021), these are not nearly the same thing. Consequently, critics argue that such technologies can mislead users on the accuracy of these platforms and suggest a degree of authority and certainty not reflected in psychological literature. Such conditions engender multiple tensions among marketers, developers, and researchers between those who are encouraged by the presumed but misleading universality of emotional AI platforms and those who aim to deliver tools to support the cultivation of emotional intelligence that are also sensitive to cultural diversity.

Sustaining this tension is a social practice of modelling emotion in machines whereby certain psychological theories of emotion most conducive to quantifiable—but not always reliable—emotion interpretation are selected over others. In this process, programmers and roboticists interested in building a machine capable of registering feeling states must start with a psychological model of how emotions work. Computer scientists Ruth Aylett and Ana Paiva (2012, p. 253) artfully summarise the technological, ethical, and social implications of this challenge:

In order to implement any model on a computer, the model itself must be sufficiently specific. From this perspective, many psychological models are not usable as they stand, but must be operationalized. Qualitative relationships must be quantified...Thus, when computer scientists select models from psychology, they tend to favour those that are already sufficiently specific or that can be made so relatively easily.

The social result of efforts to technically model emotion in software is that engineers gravitate toward those models that tend to be easily implementable in autonomous systems and generative of quantitative data. A popular example of such a universal model is that of psychologist Paul Ekman’s theory of ‘basic emotions’. In over forty years of research on the expressions of the emotions, Ekman (1999) developed a model of six basic emotions that he considered universally identifiable in facial expressions across cultures. Even more importantly, he also developed a rigorous coding system to render emotions uniformly readable. In conjunction with Wallace Friesen, and drawing on the work of anatomist Carl-Herman Hjortsjö, Ekman and Friesen (1978) designed the Facial Action Coding System (FACS). With its second published edition over 500 pages in length and outlining specific facial Action Units (AU) and exercises to recognise them, FACS provides programmers with a systematic means to code facial expression in a way that is easily implemented in software. It is thus this technical system, and in turn Ekman’s model of emotions more generally, that has become the fundamental basis within affective computing for integrating many machine systems and robots with a form of artificial emotional intelligence.

Ekman’s model of universal basic emotions, despite critical perspectives from psychology (Barrett, 2017), anthropology (White, 2017), and cultural studies (Rhee, 2018, pp. 101–109), has globalised. Ekman’s model has even inspired the emotional AI architectures—i.e., the way sensory data is processed and layered in systems of software—of many humanoid and companion robots in Japan. These include the early Face Robot of Kobayashi Hiroshi and colleagues (1994), SoftBank’s companion robot Pepper (SoftBank, 2014), Fujisoft’s communication robot Palro (Fujisoft, 2021), and Sony’s pet robot AIBO (Sony, 2021). However, other fields of emotional technology design in Japan have followed alternative trajectories.

4. AFFECTIVE ENGINEERING IN JAPAN

While some roboticists in Japan have applied Ekman’s and Picard’s approaches to emotion modelling, albeit by modulating the ends to which they were applied (see White and Katsuno, 2021), other engineers have taken less universal approaches. In this section we add to affective computing and the model of ‘basic emotions’ upon which it often relies an alternative trajectory to emotional technologies and artificial emotional intelligence from Japan.

As alluded to above, emotion and embodiment have long been at the heart of affective technology development in Japan. In fact, the task of building a humanoid robot with heart (kokoro) has long served as a signifier of Japan’s distinctive approach to technology, as well as of the nation’s signature technological contributions (Katsuno, 2011, 2015). This practice of representing Japanese technoculture through images of human-robot affinity has often been as common among Japanese roboticists as it has been among non-Japanese scholars writing about them. To offer a less caricaturised depiction of cultural difference, however, and to avoid the impression that the field of emotional robotics in Japan can serve as an equal counterpart to affective computing in the ‘West’, we focus here on a less-studied field engaging with artificial emotional intelligence: kansei kōgaku (kansei engineering or affective engineering).

Kansei engineering is a methodological approach to technology and product design emerging most prominently in the 1980s, largely associated with the work of Mitsuo Nagamachi at Hiroshima University. Nagamachi first established a programme he called Jōcho Kōgaku (Emotional Engineering) in 1970, and later changed its name to Kansei Engineering in 1988 in order to more comprehensively integrate human sensibility into engineering. In his own words, ‘Kansei Engineering is defined as “translating technology of a consumer’s feeling (Kansei in Japanese) of the product [into] design elements”’ (1994, p. 468; 1995, p. 4). Summarised later by kansei engineering researcher Yuuki Shigemoto, it is an approach to design that yields an ‘absolute emotional bond between product and consumers’ (2020, p. 1). Combining the disciplines of affective psychology with mathematics, engineering, and machine learning, kansei engineering aims to adapt to, analyse, and reveal the capacities of human sensation through both qualitative and quantitative metrics. In its attention to the subjectivity of human emotion, to its understanding of emotion as dependent on one’s interaction with objects and the environment, and to processes of quantification and automation, kansei engineering has emerged as a field as interested in the distinctiveness of human emotion as it is in generalising processes of emotion detection for automation.

Kansei is a combination of the words kan 感 (feeling, sensation, sense) and sei 性 (faculty, disposition). Reflecting the transnational processes of scientific practice in Japan, kansei was originally a translation of the English word ‘sensibility’ and the German word ‘Sinnlichkeit’ (sensuality/sensualness). For Japanese researchers, the English ‘sensibility’ referred to the ability to receive emotional and affective impressions from external stimuli, with a particular focus on the ability to aesthetically appreciate or to discriminate values. Sinnlichkeit, on the other hand, was a term derived primarily from Kantian epistemology, referring to ‘object-directed representations’ that are ‘sense-related, object-dependent, immediate or directly referential, and non-conceptual’ (Lévy, 2013, p. 86; also see Nakamura, 2013). In other words, the conceptualisation of kansei in Japanese was inspired by varying views of the sensory process from two different foreign language contexts from which they were drawn.

As the field of kansei engineering progressed, ‘kansei’ has become a term that encompasses a variety of meanings defined through the projects of kansei engineering researchers. However, like the field of humanoid robotics that similarly emphasises sensation and emotion, it has also gathered attention as a key term within a field of cultural politics, invoked in projects involving government agencies and research institutes to promote a culturally distinct approach to design. In this usage, the term is imagined as the repository of a uniquely Japanese approach to engineering that emphasises sensitivity in relation to manufactured objects. Indeed, that the term ‘kansei’ was left untranslated by some Japanese academics formative of the field, much to the disappointment of colleagues who thought this might reflect a self-Orientalising trend (including one of our astute reviewers), highlights the contested nature of the term in and beyond Japan.

It was in the mid-1980s that ‘kansei’, understood as subjective human sensory experience (Nagamachi, 1989, 1990), became most formalised as a target of research and technological design. Motivating this focus was a recognition that ergonomics-oriented engineering for mass production based on the uniformity of the human body was insufficient for customer needs. What was needed instead, researchers proposed, was a new form of engineering that could respond and appeal to the diverse sensibilities of individual consumers (The Science Council of Japan, 2005). This sense of urgency for a new human-centred engineering approach was reflected in Japan’s national policy on manufacturing, and in 1991 a national project called ‘The First Stage of Human Sensory Measurement and Application Technology’ was launched with the aim of developing technology for simple and quantitative measurement of human sensory experiences. Early studies in this period produced devices that would, for example, measure stress in salivary hormones and skin temperature (Yoshioka, 1996), reflecting methods of measuring physiological indices of emotion in affective computing.

However, different from approaches in affective computing that sought to identify objective physiological metrics for human emotion, researchers in kansei engineering employed measurement techniques in order to better identify and design products, services, and social systems that ‘give people comfort and pleasure’ (Nagamachi, 1990). This emphasis on human comfort has gradually shifted the definition of ‘kansei’ to increasingly incorporate aspects of subjective sensation and emotion that are constructed interactively between an agent and environment.

For example, Yoshio Shimizu defined kansei as ‘the ability to extract information from stimuli from the outside world, and at the same time to transmit information to the outside world’ (Shimizu, 1996, p. 184, emphasis added). Later, he expanded on this, emphasising that ‘a person’s kansei is nurtured by culture’ (Shimizu, 2016, p. 6). Toshio Kuwako defined kansei as ‘the ability to comprehensively perceive space and the body’ (Kuwako, 2001, p. 5). Furthermore, Akira Harada (1999) conducted a survey on the definitions of kansei among a range of researchers engaged in kansei engineering and organised responses along five key themes: 1) subjective and unexplainable action; 2) cognitive action by knowledge and experience in addition to inborn nature; 3) interaction between intuition and intellectual activity; 4) the ability to intuitively respond to and evaluate features such as beauty and pleasure; and 5) the action of the mind to create images. Compared with approaches in affective computing, these themes suggest an orientation to the dynamic and culturally particular possibilities of mutual influence between subject, technological object, and environment.

Based on the various conceptualisations of kansei proposed by researchers, the 19th Technical Committee on Kansei Engineering of the Science Council of Japan, convening in 2005, formalised the importance of interaction and subjectivity to kansei engineering in a report. One notable passage reads: ‘Among its various meanings, we use the term “kansei” in the sense of the ability to relate between humans, between humans and the environment, the ability to exchange information (receiving and transmitting), and the ability to form relationships. In this sense, kansei is the basic human capacity to shape society” (The Science Council of Japan, 2005, p. 4). By connecting the principles of human emotion not to individual biology as a product of evolution but rather to evolving socio-technical contexts, kansei engineering emphasised the relational and particular rather than universal capacities of human affect.

Like the terms ‘affect’, ‘emotion’, and ‘artificial emotional intelligence’ in anglophone academic literature, ‘kansei’ can take on different meanings for different researchers. While at times it captures the cultural diversity of approaches to modelling emotion even within Japan, at other times it can be leveraged in a more homogenous and nationalist frame. During our fieldwork, we noted a tendency among some kansei engineering researchers to characterise Japanese research as distinguished by a focus on emotion and heart (kokoro) that was contrasted with Western intellectualism, rationality, and rationalisation. For many, this was seen as of particular value in the current context of the global environmental crisis, where it is argued that values of sensitivity and coexistence are of increasing importance.

The process by which kansei engineering has constructed kansei as a uniquely Japanese concept that emphasises subjectivity, the senses, and their relational engendering within distinct environments adds cultural diversity to the concept of artificial emotional intelligence. This becomes particularly clear when compared with some of the basic-emotion approaches to affective computing we introduced above. Although new social robots like LOVOT (the winner of the 2019 Grand Prize of the Japan Society of Kansei Engineering’s ‘Cute Kansei Design Award’) incorporate mechanisms such as facial-expression recognition inspired by basic-emotion and emotional robotics approaches, LOVOT also incorporates exploratory and experimental mechanisms of mutual human-robot comfort more typical of kansei engineering. These include its warm body, soft fur, and button-like nose that vibrates when pressed in order to establish a sense of machine-inclusive multispecies connection (see White and Katsuno 2021, p. 245). From this perspective, emotionally intelligent robots like LOVOT, like the engineers in Japan who design them, embody multiple, diverse, and complex cultural subjectivities.

5. CONCLUSION

The diversity of cultural perspectives on artificial emotional intelligence challenges its viability as a tool to build universally applicable systems that automate emotion recognition and performance. Seeking not to rebuke the term but rather to build into it a more critical purchase, we have aimed to illustrate some of the cultural differences surrounding artificial emotional intelligence while also avoiding simplified cultural comparisons and contrasts. In other words, we hope this consideration of cultural differences in the analysis of artificial emotional intelligence ‘East’ and ‘West’ helps frame a way of thinking beyond this overly common trope of cultural comparison. Indeed, we hope this perspective also encourages researchers to think beyond the limitations of the ‘culture’ concept altogether at precisely the time it is being both overlooked and misappropriated in various fields of emotion engineering research and development. As the anthropologist Lila Abu-Lughod famously argued in 1991 (p. 466), the concept of ‘culture’ often ‘operates in anthropological discourse to enforce separations that inevitably carry a sense of hierarchy. Therefore anthropologists should now pursue, without exaggerated hopes for the power of their texts to change the world, a variety of strategies for writing against culture’. In practical terms, this means not that anthropologists and others writing about culture should abandon the ‘culture’ concept altogether but rather diversify it so it can more adequately capture the social complexity embedded in applications of globally shared terms like ‘artificial emotional intelligence’.

Paying attention to different cultural approaches to modelling and inscribing emotion in machines facilitates important capacities for cultural critique. This is especially critical at a time when companies guided by affective computing and inspired by the potential of ‘artificial emotional intelligence’ as a possible reality rather than critical concept are producing emotional technologies that increasingly rely on universal models of emotion. We suggest that drawing on culturally diverse examples helps render terms like ‘artificial emotional intelligence’ into a broader, more variable, and more critically robust object of empirical research within digital societies research.

Acknowledgments

The authors gratefully acknowledge the financial support of Doshisha University, the University of Cambridge, The Leverhulme Centre for the Future of Intelligence, Freie Universität Berlin, and the funders of a JST RISTEX Grant (number JPMJRX19H5) for this research.

References

Abu-Lughod, L. (1991). Writing against culture. In H. Moore & T. Sanders (Eds.), Anthropology in theory: Issues in epistemology (pp. 466–479). Blackwell.

Aylett, R., & Paiva, A. (2012). Computational modelling of culture and affect. Emotion Review, 4(3), 253–263. https://doi.org/10.1177/1754073912439766

Barrett, L. F. (2017). How emotions are made: The secret life of the brain. Houghton Mifflin Harcourt.

Boffey, D. (2018, November 2). EU border ‘lie detector’ system criticised as pseudoscience. The Guardian. https://www.theguardian.com/world/2018/nov/02/eu-border-lie-detection-system-criticised-as-pseudoscience

Bollmer, G. (2019). Books of faces: Cultural techniques of basic emotions. https://doi.org/10.25969/MEDIAREP/4191

Cave, S., Dihal, K., & Dillon, S. (Eds.). (2020). AI Narratives: A History of Imaginative Thinking about Intelligent Machines (1st ed.). Oxford University Press. https://doi.org/10.1093/oso/9780198846666.001.0001

Cave, S., Dihal, K., Hollanek, T., Katsuno, H., Liu, Y., Taillandier, A., & White, D. (2022). The meanings of AI: A cross-cultural comparison. In S. Cave & K. Dihal (Eds.), Imagining AI: How the World Sees Intelligent Machines (forthcoming). Oxford University Press.

Crawford, K. (2021). Atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.

Ekman, P. (1999). Basic emotions. In T. Dalgleish & M. Power (Eds.), Handbook of cognition and emotion (pp. 45–60). John Wiley & Sons.

Ekman, P., & Friesen, W. V. (1978). Facial action coding system: Investigator’s guide. Consulting Psychologists Press.

Empatica. (2021). Embrace 2. Empatica. https://www.empatica.com/en-gb/embrace2/

European Dynamics. (2021). About iBorderCtrl. https://www.iborderctrl.eu

Fujisoft. (2021). Palro. Palro. https://palro.jp

Groove X. (2019). Official website. Groove X. https://lovot.life/

Hall, L. (2021). Programming the machine: Gender, race, sexuality, AI, and the construction of credibility and deceit at the border. Internet Policy Review, 10(4). https://doi.org/10.14763/2021.4.1601

Harada, A. (1999). Kansei no teigi (Definition of kansei). In A. Okazaki (Ed.), Research organization for the special project on the construction of a structural model for sensitivity evaluation, evaluation of kansei 2 (pp. 41–47). University of Tsukuba.

Katsuno, H. (2011). The Robot’s Heart: Tinkering with Humanity and Intimacy in Robot-Building. Japanese Studies, 31(1), 93–109. https://doi.org/10.1080/10371397.2011.560259

Katsuno, H. (2015). Branding humanoid Japan. In G. Kirsch, D. P. Martinez, & M. White (Eds.), Assembling Japan: Modernity, technology and global culture (pp. 205–230). Peter Lang.

Katsuno, H., & White, D. (2022). AI in Japanese. Chapter section of The meanings of AI: A cross-cultural comparison. In K. Dihal & S. Cave (Eds.), Imagining AI: How the world sees intelligent machines (forthcoming). Oxford University Press.

Kobayashi, H., Hara, F., Uchida, G., & Ōno, M. (1994). Akuteibu hyūman intafēsu (AHI) no tame no kao robotto no kenkyū (Study of face robot for active human interface: Mechanisms of face robot and facial expressions of 6 basic emotions). Robotto Gakkaishi (Journal of the Robotics Society of Japan), 12(1), 155–163. https://doi.org/10.7210/jrsj.12.155

König, M. (2016). The borders, they are a-changin’! The emergence of socio-digital borders in the EU. Internet Policy Review, 5(1). https://doi.org/10.14763/2016.1.403

Kuwako, T. (2001). Kansei no tetsugaku (Philosophy of kansei). NHK books.

Le Mau, T., Hoemann, K., Lyons, S. H., Fugate, J. M. B., Brown, E. N., Gendron, M., & Barrett, L. F. (2021). Professional actors demonstrate variability, not stereotypical expressions, when portraying emotional states in photographs. Nature Communications, 12(1), 5037. https://doi.org/10.1038/s41467-021-25352-6

Lévy, P. (2013). Beyond kansei engineering: The emancipation of kansei design. International Journal of Design, 7(2), 83–94.

Lupton, D. (2016). The quantified self: A sociology of self-tracking. Polity.

Mayer-Schönberger, V., & Cukier, K. (2013). Big data: A revolution that will transform how we live, work, and think. Houghton Mifflin Harcourt.

McStay, A. (2018). Emotional AI: The Rise of Empathic Media. SAGE Publications Ltd. https://doi.org/10.4135/9781526451293

Mejias, U. A., & Couldry, N. (2019). Datafication. Internet Policy Review, 8(4). https://doi.org/10.14763/2019.4.1428

Minsky, M. (2007). The emotion machine: Commensense thinking, artificial intelligence, and the future of the human mind (1. Simon & Schuster trade paperback ed). Simon & Schuster.

Nagamachi, M. (1989). Kansei kōgaku (Kansei engineering). Kaibundō Shuppan.

Nagamachi, M. (Ed.). (1990). Kaiteki kagaku: Ningengawa kara mita shōhinzukuri e (Science of comfort: Toward a creation of production from a human-centered perspective). Kaibundō Shuppan.

Nagamachi, M. (1994). Kansei kōgaku to wa (What is sense engineering? Kansei to seni seihin tokushū (Sense and soft products special issue), 468–472.

Nagamachi, M. (1995). Kansei Engineering: A new ergonomic consumer-oriented technology for product development. International Journal of Industrial Ergonomics, 15(1), 3–11. https://doi.org/10.1016/0169-8141(94)00052-5

Nagasawa, S. (1998). Kansei kōgaku no kiso to genjō (The fundamentals and current status of kansei engineering. Japan Society for Fuzzy Theory and Systems, 10(4), 647–661. https://doi.org/10.3156/jfuzzy.10.4_67

Nakamura, T. (2013). The scope of aesthetics for comparative aesthetics: An examination of kanseigaku in Japan. Aesthetics, 23(1), 135–154.

Ōhashi, T., Oda, S., Hidaka, T., & Murakami, Y. (Eds.). (1985). Jōcho robotto no sekai (The world of feeling robots). Kōdansha.

Picard, R. W. (1995). Affective Computing (Report No. 321; MIT Media Lab Oratory Perceptual Computing Section Technical Report). MIT. https://hd.media.mit.edu/tech-reports/TR-321.pdf

Picard, R. W. (1997). Affective computing. MIT Press.

Rhee, J. (2018). The robotic imaginary: The human and the price of dehumanized labor. University of Minnesota Press.

Robertson, J. (2018). Robo sapiens japanicus: Robots, gender, family, and the Japanese nation. University of California Press.

Roquet, P. (2020). Telepresence Enclosure: VR, Remote Work, and the Privatization of Presence in a Shrinking Japan. Media Theory, 4(1), 33–62.

Shigemoto, Y. (2020). Meaning and Approach of New Product Designing Through Kansei Engineering. In G. Di Bucchianico, C. S. Shin, S. Shim, S. Fukuda, G. Montagna, & C. Carvalho (Eds.), Advances in Industrial Design (Vol. 1202, pp. 621–626). Springer International Publishing. https://doi.org/10.1007/978-3-030-51194-4_81

Shimizu, Y. (1996). Kansei kōgaku to sono yakuwari (Kansei engineering and its role). 47(2), 183–188.

Shimizu, Y. (2016). Kansei kōgaku no hōhō (Methods of kansei engineering). Journal of the Society of Biomechanisms, 40(1), 5–11. https://doi.org/10.3951/sobim.40.1_5

Shinohara, A., Shimizu, Y., & Sakamoto, H. (1996). Kansei kōgaku e no shōtai (Invitation to kansei engineering). Morikita Shuppan.

SoftBank. (2014, June 5). SoftBank mobile and Aldebaran unveil “Pepper”—The world’s first personal robot that reads emotions [Press]. SoftBank. https://www.softbank.jp/en/corp/group/sbm/news/press/2014/20140605_01/

Sony. (2021). Aibo. Sony. https://aibo.sony.jp

The Science Council Japan. (2005). Gendai shakai ni okeru kansei kōgaku no yakuwari (The role of kansei engineering in contemporary society). Kansei Engineering Professional Committee, Division of Human and Engineering Partnership.

Turing, A. M. (1950). Computing machinery and intelligence. In R. Epstein, G. Roberts, & G. Beber (Eds.), Parsing the Turing test: Computing machinery and intelligence (pp. 23–65). Springer. https://link.springer.com/chapter/10.1007/978-1-4020-6710-5_3

White, D. (2017). Affect: An Introduction. Cultural Anthropology, 32(2), 175–180. https://doi.org/10.14506/ca32.2.01

White, D., & Katsuno, H. (2021). Toward an Affective Sense of Life: Artificial Intelligence, Animacy, and Amusement at a Robot Pet Memorial Service in Japan. Cultural Anthropology, 36(2). https://doi.org/10.14506/ca36.2.03

Wilson, E. A. (2010). Affect and artificial intelligence. University of Washington Press.

Yoshioka, M. (1996). Ningen kankaku keisoku ōyō gijutsu (Human sensory measurement application technology). Journal of the Society of Fiber Science and Technology, Japan, 52(6), 242–246.

Footnotes

1. The first use of the term ‘artificial intelligence’ is attributed to the American computer scientist John McCarthy. In 1955 McCarthy and colleagues proposed to host a two-month workshop that tested the hypothesis that nearly all aspects of learning and intelligence could be simulated in machines. This workshop became the Dartmouth Summer Research Project, which took place in 1956. Although the term was not initially embraced by all of the workshop’s participants, it was increasingly accepted and advanced by many of its leading researchers working at MIT, such as Marvin Minsky, where AI research became prominent (Cave et al., forthcoming 2022).

2. Much of this early translation was stimulated by the mathematician Norbert Wiener’s visit to Japan in 1956, which attracted significant press in Japan, where Wiener’s work still has enormous influence.

3. As early as the 1980s the entertainment company NAMCO had sponsored a project run by Japan’s Foundation for Advancement of International Science (FAIS) that explored ‘the world of emotional robots’ (jōcho robotto no sekai) and sought to define how affect and emotion should be treated within human-robot relationships (Ōhashi et al., 1985).

Add new comment