The Visual Language of New Media: the book as database

- By Katía Truijen, Eva Valkhoff, Serena Westra and Sasha Wood

What happens when you transform a book into a new media object? Can you visualize a book in a new media way? And what happens to the narrative when you look at a book as a database?

We sought for an answer to these questions with the project ‘The Visual Language of New Media”. The project is a critical reflection of the ‘The Language of New Media’ by Lev Manovich. Manovich is the director of Software Studies Initiative and a new media artist himself, which made it possible to use his own theories and approaches to visualize the book in a new way.

The Book as Database

“After the novel, and subsequently cinema, privileged narrative as the key form of cultural expression of the modern age, the computer age introduces its correlate – the database” (Manovich 2001, p. 218).

In our project, we approached the book ‘The Language of New Media’ as a database. Database is one of his key terms in the book, and he sees this as a cultural form. It represents the world as a list of items without order, which is in contrast to a linear narrative like a novel. In new media, a narrative does not need to be linear and there can be more than one narrative at the same time. By showing the database, a new kind of story can be told. Hence, by showing the book as a database we wanted to show new stories. We created three different interfaces / visualizations with use of Manovich’s own theories and software. In other words, we wanted to transform the book from a traditional form of art into a new media object.

Visualization I: Image Plot
For the first visualization of ‘The Language of New Media, we used a word frequency counter to find out which (relevant) words Manovich uses in his book to explain the language of new media. To this dataset we added the page number on which the word appears first, and an image that represents the word visually.

The Software Studies Initiative released ImagePlot plugin in 2011 for ImageJ software that allows one to explore patterns in large image collections. With use of this software, we made a short visualization of the appearance of the vocabulary of ‘The Language of New Media’ (see video).

Click here to view the embedded video.

The X-axis shows the page number where the word appears for the first time, and the Y-axis shows the frequency of the word used in the book. The graphic shows Manovich’ emphasis on cinema and film as a cultural form, and that most of his vocabulary is introduced in the beginning of the book. The visualization shows a new narrative based on the vocabulary and poses questions like, can the position of a word in the book tell something about its importance? And what do words tell you about the meaning of a story?

Visualization II: Index
For the second visualisation of the book, we were inspired by the prologue of Lev Manovich’ ‘Language of New Media’. In the prologue, he uses the film ‘Man with a Movie Camera’ as a guide to the book’s major ideas. He picked the most important sentences and illustrated them with film stills of the movie.

Our index, INDEX.pdf, is based on the top 100 most frequent words. We selected the sentences in which they appear for the first time to create a new understanding of the book. These sentences where bundled in a new (physical) book. The order of this book is non-linear, like Manovich’s prologue. In fact, some sentences appear more than one time, as can be seen by the original page number on the top of the page.
The index creates a new perspective on the book and raises questions like: can you understand ‘The Language of New Media’ by only reading these 100 sentences? What happens when you organize the content of the book in a different order? The index can function as an extra layer to the top 100 words and provide them with background information.

Visualization III: Language / New / Media
For the third part of The Visual Language of New Media, we were inspired by the visualization that Manovich made in 2010, of the book of Anna Karenina by Tolstoj.

Manovich refers to the common reading practice of underlining important lines and passages in a text using magic markers. To create the visualization, he wrote a program that reads the text from a file and checks whether text lines contain particular word, in this case ‘Anna’ and highlights the found matches.

 Based on this idea, we created our third interface, We used different colors of markers and we highlighted the three words of the title of the book: ‘language’ (yellow), ‘new’ (blue) and ‘media’ (red). By using this simple visualization technique, the pdf of the book can be used as a map of the book to quickly see and skip to the parts where these words are used in the text, and find new patterns. Does Manovich always use ‘new’ in combination with ‘media’? Can the core of the book be found when he writes about ‘language’? For this part of the project we have printed the whole book on A0 paper and it shows beautiful and colorful patterns which allows one to enter the text and concepts in a new, artistic way. For example, you can follow the red route by following the path of the word ‘media’.

In addition, we have created a QR code, which directs one to the online version of this visualization of The Language of New Media. Once there, there is the possibility to scroll down, zoom in/ zoom out and navigate across the book in a new way.

And there is more
So the Visual Language of New Media showed that the book ‘The Language of New Media’ can be seen as a database. By showing the book in this mannery, we invite readers to explore the narrative in a new way. If you want more info, you can read the full document here: The Visual Language of New Media.pdf.

Here you can watch our presentation:

Click here to view the embedded video.