Beyond Words: Multimodal Encounters in Translation

Abstracts

Elisabetta Adami and Sara Ramos Pinto (University of Leeds)

'Meaning-making in a world of untranslated signs: A way forward on Multimodality, Culture and Translation'

The development and increased transnational flow of multimodal products has brought Multimodality and Translation Studies (TS) closer in the last few decades. However, the links between the two areas of study have not been discussed in detail and TS has remained focused on the verbal. In this presentation we will discuss the fundamental principles of a multimodal approach to communication and how those challenge our current understanding of translation. We will also discuss how considering translation can further the discussion on multimodality and, finally, present a first sketch of a joint semiotic and translation research agenda/enterprise aiming at developing our understanding of meaning-making practices in a changed, increasingly transnational, and manifestly multimodal communicative landscape. We conceive its undertaking as a pre-requisite to derive implications for Translation Studies and provide indications that can support translation practice and training in today’s communicative landscape.

 

Klaus Kaindl (University of Vienna)

'A theoretical framework for a multimodal conception of translation'

In Translation Studies, the debate about multimodality is characterized by a variety of heterogeneous terminologies that ultimately all reflect a monomodal conception of the discipline. The first aim of this presentation is to lay the foundations for a systematic multimodal approach to translation. As a first step, the innovative potential of the concept of multimodality is critically analyzed against the background of the research tradition in semiotics. As a second step, the theoretical, textual and taxonomical needs and consequences of a multimodal understanding of translation are discussed. In order to develop a multimodal understanding of translation, it must be recognized that modes are intimately intertwined with medium and genre. As a last step, the interrelations among these three concepts are highlighted drawing on examples from popular music translation. Exploring the process of transfer, and this is the second aim of my presentation, can contribute to a better understanding of the functional specificity of modes, media and genres.

 

Gunther Kress (UCL)

'Re-constituting meaning: translation in a multimodal semiotic landscape'

Social Semiotics assumes that meanings are the product of social (inter-) action, hence meaning is seen as arising out of (and intricately linked with) social actions. Social Semiotics aims to outline the semiotic principles underlying meaning-making, and to provide accounts of the means for making meaning material. Two central terms are sign (as the basic unit of meaning) and mode (as the socially shaped material means for making meanings evident): e.g. sound as speech; movement (of part of the body) as gesture; marks on surfaces as image; etc. ‘Social life’ is a constant process of meanings made with apt means in one site and re-constituted with means apt in a different site. That requires accounts of the site-specific mode-resources for making meaning and of the site-specific mode-resources for re-constituting meaning. In my account I will provide examples of meanings made in one site and re-constituted as meanings apt for, and in, a different site. The process outlined in this ‘sketch’ is intended to account for processes of making and re-making of meaning within one ‘culture’ and its resources; and, equally, for processes of making meaning in one culture and its resources, and its re-constitution in another culture with its resources.

 

María Mencía (Kingston University London), with Gabriel Gaudette, Manuel Portela and Arnaud Regnauld

'Translating electronic literature: a presentation of multimodal transferral of literary works'

Over a year and a half ago, a group of scholars, programmers, artists and translators started working on a research project focusing on the translation of various works of electronic literature, including María Mencía’s The Poem that Crossed the Atlantic, Luís Lucas Pereira’s Machines of Disquiet and Michael Joyce’s afternoon, a story. In order to identify common and divergent issues depending on the genres, formats and languages of the works under study, they were all examined through the prism of the following concepts: Translinguistic translation (translation between languages), Transcoding (translation between machine-readable codes and between machine-readable codes and human-readable text), Transmedial translation (translation between medial modalities), and Transcreation (translation as a shared creative practice). This workshop will give the members of the project the opportunity to share their observations on their work process, bridging the gap between the practice-based approach and a theoretical perspective on the task of translating electronic literature. In addition to a brief presentation of each work and the specific challenges they raised, the participants will offer key insights into the collective methodologies elaborated throughout the duration of the project. Members of the audience will have an opportunity to explore the works.

 

Helen Julia Minors (Kingston University London)

'Translations between music and dance: analysing the choreomusical gestural interplay in twentieth and twenty-first century dance works'

This paper explores the processes of translation within 20/21st-century music-dance works. It aims to illustrate the transfer of sense between the arts for those who are part of the creative process and for scholars interpreting these works. It questions how meaning is produced within and between these temporal expressive arts. It claims that the interplay of musical movement and danced movement relies on various forms of translation which require cognitive mapping, gestural interpretation, and an awareness of the somatic experience. This paper questions: Can a process of translation be read when choreographers work with pre-composed music? How do composers and choreographers utilise a process of translation and transfer when they collaborate in forming new music-dance works? How do spectators read those emergent meanings between the audio-visual elements? Focusing on narrative balletic works, three case studies illustrate three types of translation:

1. sensory translation is illustrated through a reading of Vaslav Nijinsky’s approach to Debussy’s Prélude à l’après-midi d’un faune and to Igor Stravinsky’s Le sacre du printemps (1912-1913).
2. language translation is illustrated through music visualisation, where syntax of music-dance are read as analogous to language: examples include Béjart’s Bolero (1960) and Morris’s Dido’s Lament (1995).
3. cultural translation is illustrated using the revival of Satie’s La Parade (1917) in cultural translation in the new version, P.A.R.A.D.E., produced by National Dance Company Wales (2017) to celebrate the centenary of the Russian Revolution.

Using translation as a critical framework, I show how sense is transferred between the arts of music and dance, and how meaning is emergent through their interplay. 

 

Tamarin Norwood (University of Oxford)

'Drawing: The Point of Contact'

What is the nature of the point of contact between pencil and paper during the act of drawing? My study has found this point of contact to be especially resistant to being named or wholly analysed; that this resistance is a central aspect of its character; and that a methodology of researching and writing ‘drawingly’ offers a mode of phenomenological enquiry adequate to this resistance. This methodology was arrived at through a process of multimodal translation, pressing an analogy between the moving point of contact in drawing and the path-breaking or not-knowing movement of creative work. It is characterized by (1) moving forward with acute attentiveness and close proximity to the object under scrutiny; (2) paying myopic attention to analogic, anagogic, etymological and anecdotal associations when and where they arise; (3) accepting and thematizing the fact—and the risk—that such a perspective restricts the view of an end-point, of the wider imaginative and critical landscape around, and indeed of a blind spot at the centre of the object; (4) heeding the prospect—and the risk—of ‘surfacing’ from myopia into circumspect analysis. This workshop invites participants to undertake some exercises in half-blind drawing, and to discuss in terms of translation the challenges and strategies encountered from the (a)perspective of the pencil.

 

Carol O’Sullivan (University of Bristol)

'Multimodal Translations: subtitling and image-text relations, 1920-1950'

This chapter analyses interactions between written text and the moving image in films of the first half of the twentieth century, and their implications for translation. It looks at four such kinds of interaction in particular. The first is the use of superimposed text on the image. Used from the 1910s onward, titles on action never became the standard, but functioned as a special effect in silent features including Ben Hur (Fred Niblo, 1925) and Casanova (Alexandre Volkoff, 1927). The second type of multimodal interaction under discussion is the standard subtitle. With the introduction of sound film in the late 1920s, superimposed titles were co-opted to translate foreign-language dialogue. It took some years for titles to settle into a format resembling the current one; in the early years, for instance, subtitles could be placed in different areas of the frame to reflect the position of the speaker, the pattern of light and dark in the frame, etc. The third area I will look at is typography; following Theo van Leeuwen and Michel Chion, I will look at how the typography of silent title cards, subtitles and in-vision text can carry meaning and how this poses challenges for translators. Lastly, I will look at in-vision texts such as telegrams, letters and notices which were frequently reshot in dubbed, or even subtitled films. It will be argued that these multimodal practices posed considerable technical and aesthetic problems for the distribution of films in translation, and that these experiments are mirrored in the proliferating use of written text in the image in film and television today.

 

Luis Pérez González (University of Manchester)

'Subtitling the performance of citizenship in the digital culture: a multimodal perspective'

As the ontological shift from referentiality to deconstruction triggered by the transition from ‘recording technologies’ to ‘synthesizing technologies’ (Kress and van Leeuwen 2006) continues to take hold, ‘the need for conceiving of social semiotic practice in terms of rhetoric and design’ (ibid: 219) has become unanimously acknowledged. This need is particularly acute in the digital media ecology, moulded by variable degrees of convergence between industrial and amateur practices. In those habitats where amateur rhetors have gained greater visibility and influence, the performance of citizenship often involves the deconstruction of representation by exposing the cultural and social make-up of specific semiotic resources within the overall multimodal ensemble.

Unlike critical theory and media sociology, translation studies has been slow to tackle the theoretical challenges that arise as digital media content moves away from the normative logic of linguistic referentiality, which assumes that texts should be faithful representations of the reality they draw on, to enable the emergence of a more deliberative and eclectic public sphere (Chouliaraki 2010). This paper draws on a growing body of research showing that the production and consumption of digital media content allows for playful or ethical forms of multimodal self-expression. It then goes on to examine how various processes of multimodal experimentation undertaken during the subtitling of digital media content, not limited to the reproduction of the verbal component of the subtitled text, contribute to opening up ‘alternative spaces’ for the negotiation of subjectivity and, beyond that, constructing – rather than simply recreating (Baker 2013) – the cultural encounters those texts are embedded in.

 

Matthew Reynolds (University of Oxford)

'Translating a vertical line: Dante, Dionisi, Flaxman, and the post-romantic, anglophone, literary self'

This essay and/or talk will begin from the inherent multimodality of language across sight and sound. I will consider the letter ‘I’, offered by Adam in Dante’s Paradiso as the word for God in his long-lost Edenic language, and of course familiar in English as the first-person subject pronoun (sometimes mistaken for number 1). ‘I’ can become a picture of the self, as in the pillars of text formed by Victorian dramatic monologues such as Browning’s ‘Fra Lippo Lippi’ and Tennyson’s ‘St Simeon Stylites’ (and perhaps also in the paintings of Barnett Newman); because of this it poses a particular and suggestive challenge to translation. I will endeavour to trace the translational ramifications of ‘I’, and possibly of other pictures-that-are-in-words, hoping to elucidate the element of illustration that inheres in translation, and vice versa, as well as the pictorial aspect of all writing. Points of reference beyond those already mentioned are likely to include Charlotte Brontë, Samuel Beckett, Christine Brooke-Rose, and Jorge Méndez Blake.