Tangible Emotion: Playfulness in Communicating Emotional State in Hospital Context

For the first part of the project (February-March) we have explored the themes of Isolation, Imagined Space and Creative Play through the following activities/outcomes:

In the second part of our work (April-May), we are focusing on Emotion and Communication, exploring the following: how can we communicate emotional state in a playful and challenging way in both digital and physical context. For the second AHRC workshop day, Academic Perspectives on May 9th, our workshop entitled Tangible Emotions: Playfulness in Communicating Emotional State in Hospital Context will challenge participants to create their physical emoticons.

After looking at the timetable of the day, it seems like the workshops and talks will challenge the boundaries between physical and digital spheres with presentation on traditional and digital play but also, visual explanation will be emphasised with talk and workshop such as Emotion and the visual mode: case studies from graphic novels and Japan presented by Richard Finn and Dr Dylan Yamada-Rice, and Where’s the gore? Graffiting children’s picture books, workshop led by Dr Dylan Yamada-Rice and Prof Elizabeth Wood.

The Temple of Play, via Angelo Plessas. More picture of this on here.

We want to explore emoticons by challenging participants to respond to an emotion and create their physical emoticons. We were inspired by the following work: Theater of Play, a site-specific commission for Frieze Project 2013 that featured an “Emoticon Theater” where children were invited to create their Emoti-Cube Head. Below are some research and relevant links about emoticons.

For our workshop, we are working on designing a small kit for participants to get them started in designing their physical emoticon or what we call “tangible emotion”. The kit will feature a cube package with some materials and instructions to guide the activity. First participants will find out which emotion they have to deal with for the activity. They will be asked to write one personal story in relation to that emotional state featured in the package before starting to play with the different material and colour.  The use of different materials is very important as it conveys and affects user’s emotions in a tangible way. We are also interested in exploring potentials in materiality to create better emotional experience in design solutions. Below are some key bibliographies in our research and practice.

More about Emoticons:

  • Exploiting Emoticons in Sentiments Analysis (Hogenboom, Bal, Frasincar) qualitative analysis demonstrates that sentiment associated with emoticons typically dominates the sentiment conveyed by text (:7) + definition of emoticons as designed to express, stress, or disambiguate one’s sentiment.
  • Brain reacts to emoticon
  • Xu Bing – Book from the ground is an experimental project that deals with the natural formation of a global language using emoticons as context.
  • We feel Fine: An Almanac of Human Emotion shows an interactive storytelling and data visualisation. “Drawing from a database of more than 12 million human feelings collected over 3 years from personal blogs on internet, We feel Fine presents a comprehensive contemporary portrait of the world’s emotional landscape, exploring the ups and downs of everyday life in all its color, chaos, and candor.”
  • Re-twittering Machine
  • The Angelo Foundation: International Portrait Gallery is a platform where children can upload the faces and emotions they found in their daily life.

Emotions and Material

  • The Eyes of the Skin written by Juhali Pallasmaa in 1996. The book has become a classic theory in modern architecture. In the Multi-sensory experience session (:41), Pallasmaa talks about how does the sense of touch engage with visual experience in space and why it is important in spacial design.
  • Emotionally durable design – objects, experiences and empathy written by Jonathan Chapman, professor of sustainable design at Brighton University in 2005, who proposed the theory of ’emotionally durable’ design. This theory has some interesting aspects which relates to our research, such as how does product design triggers users’ emotional experience and make them feel connected to it.

Prior to our workshop we will talk about some references we looked at in order to give a context and inspire participants for the activity. We classified our research into the five following categories, each will be discussed briefly in our talk:

  1. General Study – Theory of Emotion
  2. Digital Interface – Analysing Facial Emotion
  3. Art Context – Drawing Faces and Emotion
  4. Patchwork/Quilt – Canvas of Emotion
  5. Digital App – Expressing and Recognising Emotion

Source here.

1 General Study – Theory of Emotion

We found that they were different theories of emotion that attempt to identify and classify the human emotional experience (source here). For example, Robert Plutchik’s wheel of emotions identifies eight basic “bipolar” emotion: joy/sadness, trust/disgust, fear/anger and suprise/anticipation. This theory explains how feelings are formed: when mixing or combining basic emotions such as anticipation and joy, it forms optimism. Other theories claim that there are six universal emotions: happiness, surprise, anger, fear, disgust and sadness.

The emotion expert Paul Ekman has focused for the past ten years on learning to spot how others feel and how to respond to others’ emotions. Through his life, he was a pioneer in the study of facial expression and found that the human face is capable of creating more than 7,000 facial expressions. He created training tool such as Ekman Micro Expression Training Tool 3.0 for people to improve their skills in spoting what he calls “micro expressions”. He contributed to research on micro expressions  which are “very brief facial expressions, lasting only a fraction of a second. They occur when a person either deliberately or unconsciously conceals a feeling. Seven emotions have universal signals: anger, fear, sadness, disgust, contempt, surprise and happiness”. They differentiate micro with “macro”, “false” and “masked” expressions and argue that micro expressions are important for improving emotional intelligence, developing capacity for empathy, improving relationship and developing social skills (…).

Artist Gert Germeraad inspired by Paul Ekman’s theory (website here).

More links in relation to emotion study:

  • Human Lie Detector Paul Ekman Decodes the Faces of Depression, Terrorism, and Joy, via Fastcompany.com
  • Appropriating New Technologies via GitHub
  • How many emotion are there? via psychology.about

Image through BBC.

2 Digital Interface – Analysing Facial Emotion :

MIT Media Lab: Affective Computing is looking at how new technology can measure and track emotion through time. Using computer interface, experts argue that this technology can apply to a wide range from medical (eg. read emotion of people with autism spectrum disorder) to commercial context (advertising evaluation). The study emphasises the role emotion play in people’s interaction and the importance of building machines and new technology that have a potential in reading and transmitting emotion.

In this BBC’s article, the enigmatic painting, The Mona Lisa was analysed with emotion recognition software to see whether the figure in it is happy or not. The study concluded that “she was mainly happy”.

Jean Dubuffet, Autoportrait II (1966) and (1947). Source here & here.

3 Art Context – Drawing Faces and Emotion:

For a long time, Art has been an important part of psychatric treatment. In the early 20th century, Hans Prinzhorn was an Art historian and young doctor when he recognized the talent in his patients and began collecting their artworks which some of them could be described as “canvas of emotion”. He published a book Artistry of the Mentally III which features some of his patients’ drawings, paintings and sculptures. This had a huge impact on the artworld and inspired artists such as Art Brut theorician, Jean Dubuffet who became fascinated by the pictural production of mentally ill people.

Below are some of our favorite artists who pictured faces and emotions in interesting and various ways. The references we picked cover from early 20th to 21st century encompassing painting, drawing, sculpture, collage and photography techniques.

Sans titre-1
Munch, The Scream (), Brancusi, Sleeping Muse (1910) and Francis Bacon, Number VII from Eight Studies for a Portrait (1953).

Picasso, The Weeping Woman (1937), Roy Lichtenstein, Drowning Girl (1963) and Boetti Faccine colorate.

The Art Critic 1919-20 by Raoul Hausmann 1886-1971
Raoul Haussman and Hannah Höch, Flight (1931)

JR Face 2 Face 6
Artist JR

4 Patchwork/Quilt – Canvas of Emotion

Patchwork, often associated with quilting involves sewing together pieces of fabric into a larger design. If you want to know more about patchwork making and quilting history check the V&A Quilts 1700-2010 app which takes you on a journey through three centuries of quilt making.

There are many examples of patchwork system used as tool for community where each individual can contribute to a “bigger picture”. The largest quilt/patchwork ever made is the AIDS Memorial Quilt and has just turned digital. The quilt was an enormous memorial with over 50,000 pannels created in the 80/90s for people who have died of AIDS (more info here). Recently the quilt turned digital and features a zoomable map of the quilt and interactive timeline where users can zoom in and out. Last year the (2015) Quilt was launched allowing users to create their own patch and carriyng on the tradition from the physical AIDS Memorial Quilt.

Bed Cover (1740-1800). Via V&A.

AIDS Memorial Quilt. Source here.

Microsoft brings AIDS Memorial Quilt online. Source here.

2015 Quilt

5 Digital App – Expressing and Recognising Emotion:

Below features different apps which explore various ways and design for children and adult to express and play with emotion.

screen480x480-1 copie4
SmallTalk Pain Scale App. SmallTalk Communication and Practice Apps website.

screen480x480-1 copie6
1-to-1 Dementia Carers app.

screen480x480-1 copie2
Choiceworks App.

screen480x480-1 copie3
Funny Feelings, the app is designed to give young people the tools and skills to express themselves well in the digital era. See video here.

screen480x480-1 copie3 copie2
VolaFriends, Faces presents randomly a grid of nine faces on the screen, each with one of the five emotions (happy, sad, angry, suprised, scared). Children can pick and see a quick animation of it.

screen480x480-1 copie3 copie
Autism Xpress
is designed to help people with autism to express and identify emotions. The interface proposes three different things:

  1. Feeling Finder where you select a face which answers the following “how do you feel today?”
  2. Emotion Matching (game)
  3. Expression Question  (game)

screen480x480-1 copie
Doctot Pain App

This symptom-checker app lets you keep a digital doctor in your pocket, via FastCompany. AskMD app takes the user though a personalized step-by-step consultation that narrows down the possible health conditions to the best possible matches for the user. The aim is summarized as to empower consumers to take greated control over their health information and personalise their experience. The app also features a pain scale of 1 to 10 which mimick those in real-life that doctor would ask customers during their visit.

SmilingMind is designed to gain an understanding of mindfulness and modern meditation. Here is the website with a very nice visual identity.

Relevant Links for Emotional App:

  • 5 iPad apps for Social and Emotional Learning, Teach Thought website.
  • Apps For Children With Special Needs, website.
  • Apps that Can Be Used by Children in the Hospital: For Communication and Diversion. Paper here.







Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: