Web Translation Projects/Translation of multimodal texts on the example of the comics by Andrzej Rysuje
The aim of the project is to present the problem of translating multimodal texts, with a special focus on the system differences between the source text and the target text, e.g. cultural and social aspects. This project covers the introductory section that answers the question of what multimodality is about and what multimodal texts are about. The next step is to describe the characteristics of the comics and the problem of their translation, bearing in mind two concepts of translation: domestication and foreignization. Further sections are devoted to the work of Andrzej Milewski and the analysis of his works as well as their translations.
This project is a part of the 'Translation on the Web' course held at the Jagiellonian University in Cracow, Poland.
To fully understand the definition of multimodality, it is necessary to be familiar with the concepts of modality and modes.
In semiotics ‘modality’ is defined as 'a particular way in which information is to be encoded for presentation to humans, i.e. to the type of sign and the status of reality ascribed to or claimed by a sign, text, or genre’.  Multimodality is a theory of communication and social semiotics. Hence, it can be perceived as a combination of two or more modes, since it requires the recognition of interconnections between them. The mode can be defined as follows; it is a ‘regularised organised set of resources for meaning‐making, including image, gaze, gesture, movement, music, speech and sound‐effect’.  It means that one meaning may be conveyed in many ways at the same time, for example for example by use of graphics and text. When the scholars started using this term, they emphasised the fact that ‘multimodality’ is a matter of ‘division of labour’.  The modes create integrity. The linguistic perspective, which places language at the centre of its scientific analysis, also needs to recognise the relationship between the verbal and the visual in the text, ‘different means of meaning making are not separated but almost always appear together: image with writing, speech with a gesture, math symbolism with writing and so forth’.  The examples of multimodal texts are picture books, comics or posters. Animations or films are also multimodal texts as they constitute a representation of both written and spoken language. There is a visual input in them too. Dance can be also perceived as a multimodal text, because it mixes different modes: gestures and sound.
Because one of the human needs is communication, the interdependence between multimodal elements plays a key role in the perception of a message. Sometimes it is easier to convey a message to a particular person using a sign, icon or any type of graphic, for example for people who have an eidetic memory. A multimodal text allows an active interaction with the addressee using linguistic and visual input, which can be of particular importance, for example, at different stages of language education, especially when language skills are not yet sufficiently developed. It is the image that becomes the stimulus that helps to understand the messages conveyed. Multimodality is an important part of teaching, especially for children. Language learning uses multimodal materials that combine linguistic, visual, spatial or auditory means. Even traditional printed textbooks, which contain texts and static images, can be an excellent example of how visual and verbal elements work together for a coherent and cognitively valuable message, thus supporting the language learning process. Multimodal texts are used not only in language learning but also in general education. From an early age, children are encouraged to use picture books in order to acquire knowledge more easily.
Comic strips are a perfect example of the combination of form, content, image and language.
The comic strip is treated as a type of text, consisting of a series of static images (frames), organised in graphic form, and interrelated iconic and linguistic signs, which provides the viewer with additional information, not necessarily attributable to linguistic categories, such as typeface or font colour. Sequentially arranged frames constitute a coherent narrative and meaningful whole. In the first place, the images are the carrier of narration in comics. They tell the story, introduce the characters, define relationships between them, build up tension and communicate meaning. Linguistic signs, on the other hand, placed in clouds and frames, reflect the statements of the characters. The verbal commentary binds subsequent sequences together, defines their temporality and introduces the pictorial narrative. The binary nature of the comic strip, i.e. the combination of image and text, allows it to be treated as a multimodal text.
In the simplest words, these are different ideas or stories expressed in the form of images, usually organized horizontally. The illustrations form panels that are ‘framed, isolated by empty space (…) and take part in the sequential continuum’.  These 'empty spaces' are commonly known as 'gutters'. However, the most important attribute of the comic strip is the balloon and the shape it takes at a given moment. This depends on the type of expression and the character's emotions (hence the oval and angular balloons). A balloon resembling a smoke sign, on the other hand, refers to the character's unspoken thoughts. Comics has thus become an art that draws on many traditions: besides containing a narrative foundation, it also captures movement and sound. Other key means of expression in comics are the so-called onomatopoeias, i.e. characteristic phrases intended to imitate the sound, for example, explosions, crackling, thundering, rattling, creaking etc. Their intensity can be verbalised or visualised through the size of letters, the intensity of colours, the thickness of lines. The text is conceived as a holistic creation and it is difficult to unambiguously separate the semiotic planes in multimodal texts, as they form the structure of the text in mutual interaction. Comics has thus become an art that draws on many traditions: besides containing a narrative foundation, it also captures movement and sound.
Since ancient times, people have used drawings to represent the world around them, to convey valuable information. The journey through the history of comics should therefore start with the Palaeolithic period when primaeval people left drawings on the walls of the caves in which they lived. Another example of the early use of an image to formulate a message are the first forms of writing. The best example are the Egyptian hieroglyphs. Another important stage for the development of comics was the combination of an image with a specific narrative. It was important to show movement. Ancient Greeks drew stories of mythical heroes, for example on dishes. Picture stories depicting heroic deeds and stories of saints were also known in the Middle Ages. The next element, then developed, was to enrich the comic strip with inscriptions. The best example of this is "the Bayeux Tapestry", a hand-embroidered canvas depicting the conquest of England by William I the Conqueror and the Battle of Hastings.
Early printed materials focused on religious themes. Illustrated versions of the Bible were a widespread medium that combined many images with text to bring Christian doctrines to illiterate people. In the 17th and 18th centuries, began to present different aspects of political and social life, as well as satirical and caricature.
The beginning of the comic strip dates back to 1895. The first comic strip, developed in the USA, was "The Yellow Kid" which appeared in the form of a pull-out supplement in the large print-run newspaper, called “Sunday Funnies”. Then, they were published as daily black-and-white strips. Hence, distributed comic books in the US are considered to be sequential picture stories. As time passed, many new forms of comics appeared, like a comic magazine, comic book etc.  The best known comic books all over the world are those published by huge American corporations, such as DC Comics, Marvel Comics and Dark Horse Comics. Heroes such as Spiderman, Batman, Superman have contributed to the development of comic books in the USA and their widespread distribution throughout the world. The comic book adventures of Disney characters were also hugely popular in the USA.
European comics originated in Belgium, France and Italy.  The first ones were those created by "Wilhelm Busch", his illustrated story 'Max and Moritz' Max and Moritz became the forerunner to the comic strip, and by "Rodolphe Töpffer", who is considered to be the father of the modern comic strip. At a certain point, European comics were no longer just translations of American comics, stories were created by European authors that partly continued American themes and partly introduced new ones. One of the most important periods of comics development in Europe began in the 1960s. It was then that the adventures of Asterix and Obelix began to appear in France.
In Poland, the precursor of comic book stories was Kornel Makuszyński, creator of the character "Koziołek Matołek (Matołek the Billy-Goat)".
The third place for the development of comics was Japan and the birth of manga. The roots go back to the 12th century. Artists depicted choju-giga, sequences of images and text intended to illustrate important events, on scrolls. It was only at the end of the 18th century, thanks to the popularisation of the ukiyo-e technique, that one of the most famous Japanese artists, Hokusai, published his collective sketches under the name 'Hokusai Manga'. Until the war period, short picture stories consisting of only a few frames were published in newspapers. As Japan's borders became friendlier to foreigners, manga began to evolve into the manga we know today. 
Differences throughout the worldEdit
Forms and formats of the comics differ throughout the world. In the United States, comic books were intended to appeal not only to children but also to adults. In Sunday pull-out sections there were texts for the whole family. Yet comic strips had an entertaining character. The opposite situation was in Europe, where comics were mainly aimed at children. Their purpose was not strictly entertainment, but more educational. The aim of the comics was to teach and encourage young people to read. Pictures were not random, they kept the form of comic books but had a special reference to the presented content. Similarly, the inscriptions themselves were not formed as they are nowadays, i.e. superimposed on the picture, but the content itself was included underneath, creating a form of narration, therefore, the pace of reading such a comic is definitely slower.  However, in France, Belgium and Italy, comics began to include stories whose content and treatment of themes was not limited to images aimed at children or young people. 'In Italy in the 1960s, for instance, pocketbooks whose contents were crime, horror and explicit pornography became popular publications, joining classical adventure comic books, especially of the Western genre, on news-stands.'  Japanese comics, known as manga, are even different. Most comics are drawn black and white. What is very important is the fact that they are being read from right to left.  'To 'read' manga is to read images - the rhythm is determined by the sequence of images'.  The biggest difference between the format of American comics and European and Japanese comics is that in the United States they appeared as comic books in thin periodicals, while in Europe and Japan they were mostly published in magazines. 
A very prolific period for comic strips started actually in the last decade, due to the Information Age and the Internet evolution and webcomics began to appear online. These are usually stories involving different social situations. Very common are memes – pictures, photographs with a text, presenting a particular phenomenon or theme, carrying some ideas, very often cultural. It seems as if we came back to what was in the past. Namely, comics in the newspapers (not only) are presented now in the form of one panel or a comic strip. They show a particular issue that is an important topic of discussion in a given country. Very often they are perceived as controversial because they present serious matters in a satirical, ironic and sarcastic way.
Comics in translationEdit
The interaction of image and textEdit
The image has always been appealing to man, and at a time when print was unknown and illiteracy was widespread, the image, next to the spoken word, was the most popular form of communication. For example, paintings hanging in churches conveyed a certain theological knowledge and shaped the imagination. Iconicity in the visual arts has evolved. Aesthetic canons changed and various styles emerged. An image definitely contains more information than its possible verbal description. There are, however, some words that cannot be replaced by images, such as emotions, such as a declaration of love. A comic strip is a medium between words and images. The American cartoonist and comics theorist Scott McCloud believes that comics are the joint work of a writer and a painter. Their work combines a narrative representation, the essence of which is to depict a sequence of consecutive events, with a short textual storyline. These two elements complement each other. However, the painting cannot be realistic, it is greatly simplified. McCloud, treating comics as art, writes: 'In comics at its best, words and pictures are like partners in dance and each one takes turns leading. When both partners try to lead, the competition can subvert the overall goals...though a little playful competition can sometimes product enjoyable results. But when these partners each know their roles-and support each other's strengths-comics can match any of the art forms it draws so much of its strength from.'  In McCloud's chapter of Understanding Comics entitled "Show and Tell," he identifies seven common word-image relationships unique to comics: 
- Word-specific combinations, where pictures are not enough for the reader to fully understand the text and the context
- Picture-specific combinations, where words do not convey any content but sounds
- Duo-specific panels, where both words and images are necessary to understand the content
- Additive combination, where words reinforce the message of an image or vice versa
- Palarell combination, where words and images take two different courses, they do not intersect
- Montage, where words take on the role of images
- Interdependent combination, where the words and pictures work in unison to communicate a complete concept.
In comics, text and illustrations are closely related as in no other literary genre. When translating a comic strip, the image and any relationship between it and the text must be taken into account. This involves the ability to read the visual message and to interpret it correctly. Since comics are a combination of both word and image, ‘the reader is thus required to exercise both visual and verbal interpretive skills. The regimens of art (e.g. perspective, symmetry, brush stroke) and the regimens of literature (e.g. grammar, plot, syntax) become superimposed upon each other’. 
Difficulties involved in translating comicsEdit
Translators face problems when translating a comic strip, including space limitations, cultural references, sounds, proper names, and wordplay. They also must be creative to convey the irony and humour that is often present in comics.
Words written in balloons and frames are not the only element in a comic that is translated. Kaindl (1999) presented a 'taxonomy of aspects of comics' that change during translation. 'These include typographical signs (font type and size, layout, format), pictorial signs(colours, action lines, vignettes, perspective), and linguistic signs (titles, inscriptions, dialogues, onomatopoeias, narration). Each of these aspects may be subjected to strategies of change such as replacement (the standard option for linguistic signs), deletion, addition etc.'  Although the space in the balloon in the original text is usually sufficient for a translated text, often the big limit of translation is precisely the space available. The translator will always have to consider which construction or which word would be more economical, while still having the same meaning and the same intention as in the original text. However, when the translation is too long, a smaller font may be used. If despite all efforts, there is not enough space, it is necessary to decide which information can be dispensed with first. However, nothing that is important for the further development of the plot may be omitted.
Another element is onomatopoeia, very characteristic of comics, as it is often used to imitate sounds. It is known, however, that in different languages sounds "sound" different and therefore requires special attention and hard work in translation. The biggest problem occurs when they are not inside the balloon (as part of the text), but outside it, in the middle of the drawing. This means that the comic will have to be redrawn outside the balloon, or that the text will have to be separated from the drawing. According to Kaindl (2010), there are many strategies to translate onomatopoeias in comics, they 'range from direct borrowing (sometimes with graphemic or phonological adaptation) to literal translations and category changes as well as to new creations of onomatopoeias'. 
The translator of the comic book, apart from having a perfect command of the language of the source text, should also be familiar with pop culture or contexts specific to a given country and its culture. In order to faithfully render what is presented in the original, it is sometimes necessary to dispense with certain references, or to take into account the information conveyed graphically in the drawings and create similar content. Therefore, the comic book translator must also be an experienced comic book reader. The graphical whole must be taken into consideration, because (in the case of comic books) they give rhythm and determine the flow of the narrative. Sometimes, however, cultural references are not adapted to the target culture - these are cases of comics that deal with (or mention) universal themes and characters, i.e. well known to the world because of the symbol they represent or because of their importance in history (e.g. the names of the characters of Asterix and Obelix). Comics dealing exclusively with national characters, facts and situations that are not so familiar in other cultures are mostly adapted (in situations related to politics, social satire, customs, humour or irony, etc.). An example of cultural reference provides Polish translation of Donald Duck's comic strip The master of the Mississippi. When Scrooge decides to quit his job as a ship's captain, he says 'Go west, young duck', referring to John L. Soule, into Polish it was translated: 'Kaczko, ty nad poziomy wylatuj', thus paraphrasing a passage from Mickiewicz's Ode to Youth.
Domestication and foreignizationEdit
A strategy is a plan of proceeding to make a particular translation easier and effortless. It is an approach to the text. There are two translation strategies, which are described in detail in Lawrence Venuti's book The Translator's Invisibility: A History of Translation: domestication and foreignization. He claims that translation strategies ‘involve the basic tasks of choosing the foreign text to be translated and developing a method to translate it. Both of these tasks are determined by various factors: cultural, economic, and political’  This is a very important aspect because when choosing a strategy, the translator decides how to treat the whole text, whether to translate literally or to focus on the target culture. The differences between these two strategies are presented in the table below. 
|Focuses on the target culture||Author-centred and SL culture-oriented|
|Brings the author towards the reader||Brings the reader towards the author|
|Substitutes source text’s lexical units for corresponding elements in target language||Emphasises the cultural aspects of the source text|
|Translator can use the free method of translation||Translator preserves the elements of the source text|
|It is possible to omit or add some elements||It is possible to use notes to additions to make the text more explicit|
|No effort from the reader = lack of educational value||Forces the reader to get knowledge about source language culture|
One of the most known Polish cartoonist and creator of comics is Andrzej Milewski, who works in Poland under the name of Andrzej Rysuje. He graduated from the University of Warsaw in 2009. In the same year he started his own blog, namely andrzejrysuje.pl and he created two accounts on Facebook, both in Polish and English that are Andrzej Rysuje and Polish Doguin. He publishes there his satirical illustrations that refer to the current political and social issues. From 2013 he works with the famous Polish newspaper – Gazeta Wyborcza.  He cooperated with many organizations too, e.g. Amnesty International, The World Wide Fund for Nature, Orange, Rossmann, HBO. Andrzej Rysuje creates also English-language comics under the name of Polish Doguin.
Comics by Andrzej RysujeEdit
Illustrations by Andrzej Rysuje are always drawn in the same style, following a scheme of how the characters look like. His drawings are very simple, they look as if they were drawn by a child. The colours are very intense. They are presented in the form of a single panel, two panels or four panels put together in a shape that resembles a square. The lines are not straight. The simplicity of these illustrations is a key point to present the ideas, they do not have to be well-drawn. His drawings are mostly colourful and usually, he uses speech balloons. Each picture is signed by his name. He believes that 'you have to be controversial to interest people, and I do not avoid difficult topics'. As he said, in one of the interviews, his main inspiration are pieces of information that are provided in the media.  The illustrations are related to current political and social issues happening in the country or in the world. They often convey important content and a message.
Analysis of Andrzej Rysuje's comic strips and their translationsEdit
The illustrations draw attention to the serious problem of plastic bags littering the environment. They cause damage to the environment, they are detrimental not only for plants but also for animals. Because the picture and the message it bears are very strong, this comic became very popular not only in Poland but in the whole of Europe, there is also a translation into Spanish and French. They are used as water pollution posters that send out an alarming but powerful message to their audience. The translation involves only a text in the speech balloon since the pictures are universal and may be used by the audience all over the world. On Polish Doguin's Facebook website this picture is being commented the most frequently. From a linguistic point of view, the form of politeness has also been retained. The Polish "proszę" has been replaced by the English "Sir".
Comic strip in Polish is available 'here'
Pictures are a sarcastic representation of the process of recycling, both in Poland and abroad. Andrzej Rysuje shows how people treat the environment. In the Polish version, people throw their rubbish into forests, even household products. Because it is not how recycling should look like, he "polonizes" the name of it. The English version is slightly different. To translate this comic panel Polish Doguin changed the background. Instead of the forest, there is a sea or an ocean (depends on the audience). This change is since in Poland there is more discussion about littering in forests and about water pollution around the world. The name of recycling is written in inverted commas to keep the same message as in the original image. The translation had to be adjusted to the target audience. Both text and graphic had to be taken into consideration while translating the picture.
Vaccination was and still is, the main subject of discussions in recent times. For many people, the question of whether or not to vaccinate their children is a very difficult one and remains unanswerable. The anti-vaccination movements exist for many years, they are against mandatory childhood vaccination. The issue escalated in 1998 when Andrew Wakefield published his research findings. He proved that vaccines can cause serious health problems and many disorders, for instance, autism. In 2019 this issue was identified as one of the top ten global health threats of 2019 by the World Health Organization. In Poland, the controversy arose in July 2018 when a citizens' bill, named ‘STOP NOP’, was submitted to the Polish Parliament. Andrzej Rysuje presents this issue in his drawings.
The pieces of information provided in the speech balloons are the same, but there is one element that distinguishes between them. On the male character’s t-shirt, there is written a name of the Polish citizens’ bill - ‘STOP NOP’. The English version does not cover this name because it refers only to Poland. Instead, the word ‘STOP’ is used. Though, it does not affect the meaning. As it can be seen, the intended meaning of the presented examples is easily rendered into English. The content is universal and is available for a wide range of people.
Illustrations refer to Terry Pratchett's comic fantasy books series Discworld, a theory that the world is a flat disc and to global warming. It is a deriding depiction of global warming, in a sense. In the Polish version, it looks as if it was a political discussion. The Left in Poland shares the opinion about a global warming influence on the environment. The next point is Greta Thunberg's position in society. In general, she is perceived as a climate change activist, though most people do not consider her as a competent person in Poland. That is why the picture is highly ironic and sarcastic. The translation of this picture is drawn in a slightly different way, taking into consideration a cultural background, political situation and target audience. Hence, the picture presents a turtle (Great A’Tuin in Pratchett’s book) with caricatural Donald Trump’s face. The reason for this change, probably, is because he does not accept global warming, he does not believe it is true and that people cause great damage to the environment. The next element that differs from the original version is that in a speech balloon there are two mistakes that I believe were made on purpose, namely ‘exscess’ and ‘abbys’. Personally, I think that the reason for this may be caused by the fact that his speeches were not substantive enough.
Comics are multimodal texts since they involve using at least two different modes. In the case of the illustrations abovementioned, it was a text in a speech balloon and the picture. As it comes to the format of the comics in all cases is the same. The illustrations are very simple. On the example of these cartoons, the most quintessential was their simplicity but strong message. They are created to raise people’s awareness and knowledge on a given subject matter and make people aware of a particular problem. The issues that Andrzej Rysuje covers are diverse, they concern social and political issues. Each illustration is adjusted to the target audience, and so its culture. There is no need to change the graphic representation in some cases, since the subject matter is universal and is understood by many people and many cultures. The comics that referred to the Polish political or social situation were changed. Translation only of the text would not be easy to understand for the people abroad. The analysis showed that the translation of culture-bound terms requires the use of equivalence or adaptation. Andrzej Rysuje’s works are rather subjective and they are opinion-forming. Considering the translation strategy, it is evident that Andrzej Rysuje opts for domestication in his translations, making the text closer to the target culture.
- Jewitt Carrey, and Kress Gunther. 2003. Multimodal literacy. New York, NY: Peter Lang, p.1
- Jewitt, Carrey; Bezemer, Jeff; O'Halloran, Kay. 2016. Introducing Multimodality. London: Routledge, p.2-3
- Groensteen, Thierry. 2007. The System of Comics. Trans. B. Beaty, N. Nguyen. Univ. Press of Mississippi, p.26
- Zanettin, Federico. 2008. Comics in translation: An overview. Comics in translation, 1-32. Menchester: St. Jerome Publishing, p.1-3
- McCarthy, Helen. 2014. A Brief History of Manga: The Essential Pocket Guide to the Japanese Pop Culture Phenomenon. Hachette UK.
- Johnson-Woods, Toni. (Ed.). 2010. Manga: An anthology of global and cultural perspectives. Bloomsbury Publishing USA, p.270
- Rommens, Aarnoud. 2000. Manga story-telling/showing. Image [&] Narrative, 1(1)
- https://en.wikipedia.org/wiki/Comics, Retrieved 2021-06-10
- McCloud, Scott. 1993. Understanding comics: The invisible art. Northampton, Mass, p.153-156.
- Eisner, Will. 1985. Comics and Sequential Art. New York: Poorhouse Press, p.8
- Zanettin, Federico. 2004. Comics in translation studies. An overview and suggestions for research. Traduction et Interculturalisme. VIIe Séminaire de Traduction Scientifique et Technique en Langua Portugaise, 93-98.
- Kaindl, Klaus. 2010. Comics in translation. Handbook of translation studies, 1, 36-40.
- Venuti, Lawrence. 2001. Strategies of Translation. In M. Baker (ed.), Routledge Encyclopedia of Translation Studies, 240-244. London & New York: Routledge, p.240.
- Venuti, Lawrence. 1995. The Translator’s Invisibility: A History of Translation. London and New York: Routledge.
- https://komiks.blog.polityka.pl/2012/07/30/andrzejrysuje-pl-w-ogniu-pytan-czyli-rozmowa-fac etem-od-psingwina/