The MUTABLE Research Project

A black-and-white drawing that illustrates an authentic situation of the data collection process for the Mutable corpus. The picture shows two men that are part of a blind-sighted team. The team is drafting an audio description.
A blind-sighted team is drafting an audio description. Copyright: MUTABLE Corpus & Eero Tiittula. All rights reserved.

MUTABLE: Multimodal Translation with the Blind

A postdoctoral research project at Tampere University (previously at University of Helsinki), funded by Academy of Finland

Project duration: 2017-2020

How are cognitive operations related to translating (e.g. problem solving, processing of source and target texts) displayed in the interaction between translating team members? How are the cognitive displays shared via different communication modes, such as speech, gestures, and gaze?

How does translating emerge and evolve as an interactive process between social actors?

What kind of expertise does the team bring in the translation process? How does the perception-related asymmetry between its members – some are sighted whereas others are blind – affect interaction and translation? Is blindness a resource rather than a handicap?

A black-and-white drawing that illustrates an authentic situation of the data collection process for the Mutable corpus. The picture shows two persons that are part of a blind-sighted team. One person is sitting on a chair, in front of a computer and holding hands on their face. The other person is pictured from behind and only their back of the head is shown in the picture.
Illustrations of the corpus.
Copyright: MUTABLE Corpus & Eero Tiittula. All rights reserved.

MUTABLE studies the interactive translation process by analyzing team audio description in which sighted and blind describers work on an audio description face-to-face. In the project, the particular topics of interest are the interface between cognition and interaction, the use of multimodal resources while translating, and the organisation of work in and the expertise of perceptually asymmetrical teams. The main research data are video recordings of authentic audio description processes in Austria, Finland and Germany.

The project’s results increase knowledge of translating as a multimodal and interactive process and of interaction between sighted and non-sighted persons. The project highlights the role of interaction and users in the making of translation. The project’s findings can help identifying visually-impaired persons’ expertise in relation to translation and multimodal communication. This new knowledge can be applied to foster inclusion in the society as well as diversity at workplace.

Here is a video of an interview in which Maija Hirvonen finds out about the craft of collaborative audio description from two Finnish audio describers (one sighted and one blind). The interview is in Finnish. You can find the transcription of the interview below the video.

Play video on YouTube (opens in new tab)

Litteraatti haastatteluvideosta (transcription of the interview)