Designing Interactive Newsprint

The possibility of linking paper to digital information is enhanced by recent developments in printed electronics. In this article we report the design and evaluation of a local newspaper augmented with capacitive touch regions and an embedded Bluetooth chip working with an adjunct device. These allowed the interactive playback of associated audio and the registration of manual voting actions on the web. Design conventions inherited from paper and the web were explored by showing four different versions of an interactive newspaper to 16 community residents. The diverse responses of residents are described, outlining the potential of the approach for local journalism and recommendations for the design of interactive newsprint.


Introduction
In the New York Journal of 2nd June 1897, Mark Twain wrote "The report of my death was an exaggeration".The same might be said about printed newspapers themselves, despite falling circulation, as some readers switch to on-line news content consumed on computers, tablets and phones (c.f.Meyer, 2009).This situation is one of the latest manifestations of an old debate about the future of paper and which has variously raged and rumbled on since the earliest presentation of textual information on computer screens.
Comparisons of paper versus screen reading took place in the context of a debate about the paperless office, and the prospects of dedicated reading devices that might eventually replace the printed book (Dillon, 1992).Predictions of new forms of reading in hypertext and hypermedia followed, and eventually came true in the realization of the world-wide-web (Bolter, 1991).New technologies for the presentation and distribution of electronic books, magazines and newspapers were eventually commercialized by companies like Amazon and Apple.And the rise of user-generated content on the web, shifted the balance of power from publishers to individual authors, allowing a wider range of people to create and share information in either screen-based or paper form.
Despite these cataclysmic changes in the authorship and form of media content, paper has proved stubbornly resilient in the face of potential obsolescence.The paperless office never materialized because people simply moved backwards and forwards between paper and screen-based materials to make use of the different affordances of each (Sellen and Harper, 2002).Reading the web simply became a new form of literacy alongside conventional reading (Snyder, 2002).Hence printed materials of all kinds continue to be used with new generation laptops, e-books and tablets for different reading and writing activities, specialized by content.In fact, they are often used simultaneously or serially in paper-and-screen environments such as architect's offices, transport control spaces and news rooms, where information is compared from different sources, and passed between people (Heath and Luff, 2000).
The co-existence and use of both paper and digital content has led several authors to call for the creation of closer links between the two, through technologies of interactive paper (Luff et al., 2004).Interactive paper is the opposite of electronic paper because instead of making existing devices simulate the properties of paper, it "focuses on augmenting regular paper by linking it to supplementary digital information and services" (Signer and Norrie, 2010).A number of different technologies exist for linking paper and digital information together, including 1D and 2D barcodes, RFID and NFC tags, optical page recognition and finger tracking, digitizing tablets, ultrasonic, optical and inductive pens, and capacitive touch regions (Signer, 2008;Steimle, 2012).Perhaps because of this variety of core technologies, the applications of interactive paper are many and varied (see related work below) and principles for the successful design of interactive paper are lacking.In fact Signer and Norrie (2010) conclude their review of the area by listing this as one of the major challenges for the field: "Since the design of interactive paper interfaces is a relatively new domain, there are no established guidelines on how to design an interactive paper interface… Similar to the lack of visual guidelines, there are no rules on how to design the interaction with an application ".
Taking this observation as our starting point, we focus here on a single technology and application domain for interactive paper that we think has great promise, and test out multiple designs in order to extract design principles and recommendations for that domain.The technology comprises capacitive touch points that can sense the position of fingers and thumbs on a printed page through changes in electrical conductivity at the point of contact.We use this interaction technique in the local journalism domain to invoke associated speech or other sounds from printed newspapers and to register subjective responses in the form of 'votes'.In the rest of the paper we briefly explain our reasons for focussing on newspapers, and summarise related work on augmented paper before introducing a new framework for understanding interactive paper experiences at multiple levels.We then describe the interactive newsprint platform and documents, and the way they were tested in a study to elicit reader responses, before summarizing those responses and their design implications.

Why newspapers?
There are two main reasons for considering newsprint as a promising domain for interactive paper, aside from the fact that it had not been fully explored before.The first has to do with the affordances of printed news layouts as a particular kind of paper, while the second has to do with the parallel development of the online news business alongside printed news.
In the first case, newsprint benefits from several interactive properties or affordances of paper, pointed out several years ago by authors such as Sellen and Harper (2002) and Luff et al. (2004).Newspapers can be flicked through to assess their contents quickly, jointly viewed, annotated, and passed around physically without the need for compatibility between devices or formats (see Table 6.2 in Sellen and Harper, 2002).They are therefore easier to share and collaborate around than their e-book counterparts, being not only mobile, but 'micro-mobile' allowing elements to be spread out, re-orientated, juxtaposed and memorized spatially (Luff et al., 2004).The information and graphic design of newspapers in particular, lends itself to this behavior because different physical sections (for News, Sport, Business, etc) are indexed on the front page and often segmented physically into different supplements.Article headings and opening paragraphs summarise the content, which is expanded later as you read on.And many pictures are usually embedded in the layout to illustrate the accompanying text which has a certain shape and style.All these elements make it easy to browse and read a printed newspaper using the affordances above.This structure also lends itself to augmentation with digital content, because sections, headings and pictures can be used to locate interactive regions or buttons from which further multimedia content is available.Digital augmentation could also overcome limitations of space on paper, which leads to very large newspapers in some countries such as the US.
In the second case, there is a business imperative for re-considering the design of newspapers linked to falling newspaper circulations, which have been recorded since the 1970s.This trend has been accelerated by the advent of digital news consumption as readers switch to on-line news content consumed on computers, tablets and phones (c.f.Meyer, 2009).The drivers behind these changing consumption habits, and seeming mass-exodus from print to digital landscape, are complex.Shifting audience demands and instant interactions via digital have seen news and information, or perhaps more accurately the function of news and information to inform a community, migrate not from offline to online newspapers, but from offline to social media, search engines and listings services such as eBay or Craigslist.The result of this audience migration has undermined advertising revenues from printed newspapers (National Union of Journalists, 2016).Advertising revenue is the central revenue stream of many local and national publications across the globe (Holcomb and Mitchell, 2014), and its fall has marked the end of a number of newspapers.According to the American Society of Newspaper Editors, (Barthel, 2015), the number of journalists employed in the US fell by 17,500 in the decade between 2003 and 2013.In the UK, government Labour Force data suggests that, between 2013 and 2015, 6000 fewer people described themselves as journalists, newspaper and periodical editors (Pondsford, 2015).These falling figures represent fewer journalists 'doing journalism'.As such, there are question marks associated with journalism's fourth estate function, defined as its' ability to hold those in power to account and inform the public.
However, the dynamic between print and digital journalism is more complex than a migration from one platform to another, falling circulations, ad revenue and job losses.Many printed newspapers have online editions, providing choice to readers about which platform they can chose for particular articles or situations.Printed articles can point to further information on the web, and web articles can point to books or other printed materials for longer treatment.Studies of the daily news repertoires of individuals show complex media diets involving combinations of print, TV, radio, websites and social media, with some people continuing to favour printed newspapers as their primary source (e.g.Couldry, 2012;Schrøder, 2015).And new business cases are emerging for free newspapers such as the Metro and the London Evening Standard in the UK, which increase circulation by removing the cost to the readers, thereby increasing advertising revenue again.Local journalism is in particular flux as citizen journalists publish local news online, and community groups publish free newsletters and magazines for targeted audiences.This is leading to a 'hyperlocal' news ecosystem of professional and citizen news, both online and in printed form for the same area (Williams et al., 2015).
Given these developments in the design of parallel online and printed news content, and the growing practice of reading between them for particular purposes, we believe that the time is right to try to connect paper to web in this domain.Current attempts involving QR codes have met with modest success, and require a kind of double reading action to scan and then read associated visual content from a mobile phone hovering over the paper.In this study we explore a different paradigm in which web associations are invoked by direct touch of points on the paper, and limited to audio output or voting input.A mobile phone can still make the connection to the web, but remain in the pocket of the reader who handles the newspaper in the usual way.Such interactive newsprint could offer a third way for newspapers to address a number of challenges facing the industry whilst continuing the trend towards greater democratization of news content (e.g.Deuze et al., 2007).For example, the multimedia nature of modern news could be partially recreated across print and digital platforms by linking between them, audio can be overlayed on print content and even updated dynamically, and the ability for readers to respond to articles can be supported by voting on, or even speaking to, paper.Touch sensitive paper might also protect print advertising revenues by providing advertising or purchasing web links, or supplying advertisers with 'paper-analytic' feedback on when a page or item has been read or listened to.

Related work
Experiments on the design of interactive paper are usually said to have started 24 years ago with Wellner's Digital Desk (Wellner, 1991).This explored the use of an overhead camera and projector for enhancing reading activities with paper documents (see also Wellner, 1993).However, earlier experiments include Edison's use of phonographic paper in 1878 to transport 'voice letters' in the physical mail, and Hoshino's 1959 experiments with paper 'synchrosheets' coated in magnetic paint for recording and playing back sound (cited in Frohlich, 2004, pp. 16-19).Today, most experiments of this kind would be classed as augmented reality systems, in which paper artifacts rather than other physical objects or places are enhanced with digital resources.Examples are too numerous to mention exhaustively, but can be summarized in terms of the type of paper documents to which they apply and the technologies they use.
Hence, a number of augmented paper notebooks have been explored, linking handwritten notes to meeting recordings, photographs and video clips.Early systems use PDAs and tablet digitizers with overlaid paper, while later systems use the Anoto pen and paper system (e.g.Mackay et al., 2016;Stifelman, 1996;Yeh et al., 2006).The Livescribe smart pen and paper system is a commercial audio-notebook product in this area based on Anoto technology.Augmented photographs have been created using embedded audio chips, overhead cameras, digitizer/PDA combinations, and Anoto pen and paper (e.g.Frohlich et al., 2000;Frohlich et al., 2004;Kindberg et al., 2004;Piper et al., 2013;West et al., 2007).The primary digital data in these cases has been associated verbal storytelling or non-speech audio.HP Live Photos are an example of a commercial application in this domain, linking printed photographs to associated video clips via a smart phone app.Augmented books of various kinds have been created using electric field sensors, RFID tags, barcodes, PDAs, cameras and head mounted displays.Most examples make extensive use of audio associations to enhance the reading experience (e.g.Back et al., 2001, Delle Monache et al., 2012;Klemmer et al., 2003).Others link animations and video clips to enhance learning or bring characters to life in a story (e.g.Billinghurst et al., 2001, Tallyn et al., 2005).A commercial example in this space was the Leap Pad, which used a double-sided digitizer with booklet overlay to present audio annotations on childrens' books.
There has been very little work on augmented newspapers, aside from promotional weblinks through printed QR codes.The only research explorations we can find is through the AudioCanvas system developed recently for Indian and South African contexts (Robinson et al., 2014) and the RocReadaR system published recently (Fedorovskaya et al., 2016).AudioCanvas allowed users to access audio content on low-end cameraphones via interactive voice services.Users took snapshots of printed pages from maps, leaflets, packaging or newspapers that contained QR reference points and identifiers.Thereafter they interacted with the snapshots to hear associated audio such as spoken texts and explanations.Preliminary evaluations suggested benefits for those with low levels of literacy or difficulty interpreting articles or following instructions.RocReadaR was a similar system designed to trigger digital augmentations to the printed Research Newsletter of Rochester Institute of Technology.Holding a smartphone over pages of this magazine caused hotlinks to be presented on the smartphone screen.These could then be touched to bring up related content on the smartphone itself.The associated information was found to add interest, information and fun to the reading experience and make it easier to remember.However, new design issues were thrown up such as the potential distraction of digital from printed content, the division of labour between digital and print, and how best to cue and control the associations.These preliminary findings suggest a rich design space for interactive newsprint and some potential benefits which we will now explore in our own study.

Aims and design framework
Building on previous work, we set out to understand the potential role of interactive paper in local print journalism, and some of the design constraints and principles governing this.Our starting concept was an interactive newspaper from which readers could seamlessly access associated audio information through handling gestures.Unlike previous attempts to support interactivity through optical page recognition on a smartphone, we set out to explore an alternative paradigm of instrumenting the paper itself, and presenting associations in audio-only form over wireless headphones without the involvement of a second screen device.This avoided visual distraction from a second screen, and was technically challenging; leading to both innovation and design insights on the use of printed electronics in this domain.
Some of the challenges of designing for interactive newsprint are highlighted further in a study of pen-and-paper user interfaces.When designing for these types of interfaces, Steimle (2012) proposes a model which separates semantic from syntactic level activities of interaction between paper and digital resources.For example, conceptual activities of annotating and linking resources are separated from core interactions used to achieve this, such as inking and clicking -both for single and multiple sheet documents.Unfortunately, activities like inking do not apply to touch-based interfaces lacking pen input, and are at a level that overlooks broader factors such as the type of content being presented in different media formats.As Hinckley and Wigdor (2012), even in instances where input technologies focus only on one single modalityin our case a touch-paper interfacewe need to understand the broad limitations of that modality, which may include the specific attributes as 'display' size and form factor, target market, user experience and application design.Hence our first prototypes represent an exploration of a new hybrid form of interactive content that faces similar design challenges to those faced by content creators in newspaper publishing at the outset of desktop publishing, the internet and more recently mobile and tablets.Question marks around what forms of advertising are readers/users prepared to view for access to 'free' content, the length of on-screen articles compared to those of their print siblings and fundamental questions around the user experience and understanding of what content is 'clickable' or 'interactive' have all been the concerns of editorial teams since the emergence of online newspapers in the 1990's.Furthermore, we were intrigued by the idea of maintaining paper-based news publications as a form factor and as a hybrid technology triggering digital content consumption.
For these reasons, we developed our own simple framework for understanding the dynamics of what might be called hand-and-paper interfaces, based on the different levels of design contributed by intersecting design disciplines (see Fig. 1).This is also useful for understanding design constraints and user responses to different elements of interactive paper more generally.For example, at a top level of product design, an interactive document should have a coherent product concept and fall into a recognizable document genre such as book, leaflet, magazine or newspaper.Below this, the document should have a graphical user interface design that identifies which elements of the paper are interactive and interprets manual actions on these elements as requests for certain kinds of content or responses to content.Interaction design specifies the way in which content and responses are controlled through this input over time.Content design is specified separately through a combination of aspects such as information design, editorial design, sound design and journalistic and editorial voice (sub-divisions not shown).This depends on the multimedia capabilities of the adjunct technology being used to present the associated content.
Using this framework we designed and tested a number of 'interactive newsprint' documents involving only capacitive touch sensitive paper.This is an alternative implementation to AudioCanvas and RocReadaR that allows the paper, rather than a snapshot of the paper, to be used as an interface to associated digital content provided by an adjunct mobile device.Although this adds cost and complexity to the paper itself, it provides a more direct form of interaction with printed news content.It also opens up the prospect of making the experience more self-contained in the future, with the printing of further electronics like speakers, storage and CPU.

Interactive newsprint platform
Stepping through the levels of our framework helped us to decide what to implement within this design space.
• Product design: the document will be a kind of printed newspaper.
Our product design is constrained by the limitations of the capacitive paper technology employed, allowing for up to 8 'event triggers' on a single page.
• UI design: graphical buttons (of varying designs) will be used to indicate associated audio content while other icons will be used to collect votes or rankings on content • Interaction design: audio content can be paused by a toggle action on buttons while votes will be registered by single icon presses • Content design: printed journalistic content will be displayed on the newspaper surface while associated audio-only content will be played through wireless headphones.
The resulting Interactive Newsprint platform was implemented by our company partner Novalia who printed capacitive touch regions at selected points on a printed page, together with conductive ink tracks to an embedded Bluetooth chip.This minimized the amount of physical electronics on any page by moving computation to an adjunct device.In fact, the electronics were printed on the back of the interactive front pages of several newspaper versions, as described in Section 6 below.Fig. 2 shows the electronics for the reverse of the Lancashire Evening Post front page (Webpage version in Fig. 3b).Page two was stuck to the reverse of page one to hide these electronics from the reader.The electronics worked as follows, allowing the reader to access associated audio over wireless headphones from certain touch points, or to 'vote' on printed articles by pressing on others.
The system in Fig. 2 consists of four components, a sheet of paper with screen printed conductive ink, a capacitive touch module, battery pack and a Bluetooth module.In the first component, conductive ink is screen printed onto the sheet of paper to form capacitive touch sensors and interconnects to the second component.The screen printed conductive ink has sufficient conductivity to carry electrical charge between the sense area (larger areas of conductive ink) and the capacitive touch control module.The second component (capacitive touch control module) runs software that charges the conductive ink lines connected between the control module and the sense areas sequentially.The module is connected to the conductive ink using a silver conductive epoxy.
The capacitive sensing used in this set up is achieved by charging and discharging lines from the IC (integrated circuit) which are connected to the printed conductive ink tracks that route to the printed sense pads on the paper.The time to discharge a line is measured and stored by the IC.When a line is charged with a voltage the amount of charge stored on the sense pad is greater if a persons' finger is present than when it is not being touched.The finger and the sense pad form two plates of a capacitor.This greater amount of charge is seen as a longer discharge time on the IC pin, ranging from 10 µs to 100 µs, each pin/pad is charged sequentially and the process is complete in less than one microsecond.
This captive touch module is connected via a few wires (SPI connection) to communicate to the Bluetooth module, the Bluetooth module is connected wirelessly to an internet connected device, in this case a laptop.Audio can be played from the internet connected device and data sent to the cloud.This whole system has since been replaced with one control module (1.6 mm thick) with one IC and thin 2016 coin cell, and connects directly to a users' smart phone.
The experimental platform was used here to create four different interactive newsprint documents, developed in collaboration with a community of professional and community reporters.These are described in the next section.Despite the technical complexity of the system, the user experience was surprisingly simple.Users could browse what looked like a regular newspaper laid out on a table, and touch graphical buttons to playback associated audio over wireless headphones or rate printed content.

Document design
The final stage of our design process was to consider possible sources of local news content that would be of interest to residents of a local community.For the study we recruited 16 suburban residents of Preston in Lancashire, UK who were regular readers of either printed or on-line news.The biggest local newspaper was the Lancashire Evening Post (LEP) written by professional journalists, while a popular on-line source was Blog Preston (BP) written by locals themselves.It struck us that interactive newsprint versions of both news sources would lead to two quite different hybrid papers in varying formats.An interactive printed Lancashire Evening Post would result in a conventional newspaper having some of the properties of a web page with voting cues and hyperlinks out to audio clips.Conversely, a printed interactive Blog Preston would have the properties of a printed webpage with voting and audio annotation.This webpage might be at broadsheet newspaper-size or -given the increasing popularity of tablets for reading on-line news -tablet size in format.Graphically then, the content design of these documents would look very different if they simply inherited the conventions of newspapers or the web; different kinds of updateable audio content might be used by professional or community journalists to extend these publications.Our conceptual deliberations went one step further: we wondered what would happen if we switched design conventions for the same content, by printing the Lancashire Evening Post in the style of a webpage and Blog Preston in the style of a traditional printed newspaper.Hence, varying the origin and format of local news across web and print dimensions led to four possible interactive newspapers as follows: • LEP Newspaper -paper origin, newspaper format • LEP printed webpage -paper origin, webpage format • BP newspaper -web origin, newspaper format • BP printed webpage -web origin, webpage format We decided to compare and contrast reactions to all four of these hypothetical documents, as realized in Fig. 3.This is because preferences between them might reveal important design principles to carry over from print and web design, or places where these break down for interactive newsprint.They might also indicate the relative strength of audio in augmenting professional or community journalism, and the value of paper in each domain.Hence a range of audio annotations were designed to supplement a selected issue of the Lancashire Evening Post on Saturday April 21st, 2012 and a selected snapshot of Blog Preston on Thursday 8th November 2012.'Newspaper' versions of both documents were printed at broadsheet scale measuring 29×38 cm, with only the first page being interactive (Figs.2a and 2c).'Webpage' versions of both documents were printed at different scales, with LEP measuring 29×38 cm as above (2b) but BP formatted as a double-sided paper tablet-size measuring 29×19 cm (2d).
Audio and voting touch points were supported only on the front pages of newspapers 3a-3c which thereafter contained a number of non-interactive pages taken from the selected LEP and BP issues.This is because the Bluetooth chip and cell batteries on the back of each  front page only supported up to 8 capacitive touch points, could not currently be printed, and added about 2 mm thickness locally.However the electronics on page 2 could be effectively hidden by sandwiching it to page 3, resulting in a document with localized interactivity which felt like a conventional newspaper.In contrast, both sides of newspaper 3d were interactive because it was created out of a printed webpage folded in half, with the electronics hidden in the middle.This meant that it had no other non-interactive content like the other newspapers.These documents therefore served as user experience prototypes for the purpose of this study, to give users a taste of interactive newsprint in a variety of formats for subjective feedback, in anticipation of being able to print multiple interactive pages in the future.Different audio annotations and voting options for the LEP and BP documents are summarized in Table 1.Audio annotations were drawn from professional and community-created content.For example, on the interactive LEP a full recorded interview with David Cameron was associated with a print article about his visit to Preston; the latest Olly Murs single 'Oh my goodness' was linked to a headline picture; and a BT Infinity radio advert was linked to its printed version.In contrast, items from community radio station Preston FM were linked to articles in the BP documents.These included spoken news items about a miniature model of Preston and a local family event, as well as an interview with the winner of the Preston half marathon.Voting on both documents included opportunities to 'rate' or 'like' a number of articles, and to participate in a straw poll on whether or not a Preston marathon should be an annual event.

Methods
Sixteen participants, were recruited through a snowball sampling method.All of the participants lived within a 14-mile radius of Preston city centre and therefore qualified as the target audience for both the LEP and BP.Participants ranged from 20 to 72 years of age and included ten women and six men.They varied in their experience of reading printed or online news, with approximately one third reading only print, one third reading only online news, and one third reading both.The user trials were all conducted on campus at the University of Central Lancashire in an area with sofa-style seating and coffee tables.Before they were introduced to the prototypes, each participant took part in a short (10 min) audio-recorded interview designed to gather further contextual information about their use of digital/interactive technologies, their consumption of printed and broadcast sources of news, and their use of websites, apps, social media or other services to access news content 'on-screen'.Participants had no prior knowledge of the interactive nature of the prototypes and, following the initial interview, were presented simply with four 'different news publications', one at a time on a coffee table in front of them.The prototypes were presented to participants in varying permutations across the sample in order to prevent a 'learning effect' bias (a latin squares repeated measures design).
Participants were asked to explore the content of each interactive document wearing a pair of wireless headphones.They were encouraged to think aloud as they did so, although most people listened or read silently unless they encountered a problem.When the participants were satisfied that they had fully explored the first document, they were invited to rate the prototypes' 'ease of use', 'informativity, 'enjoyability' and 'usefulness' on a five-point Likert scale (ranging from 'strongly agree' to strongly disagree').This was followed by a short discussion where participants were asked to explain their responses to the four statements.This procedure was repeated for each of the four prototypes.Once the participants had explored all four prototypes they were asked to rank the newspapers in order of preference.To assist in this process, the prototypes were laid out for the participants to view again, and the researcher encouraged participants to explain their decisionmaking.

Findings
The following results are based on observations of 16 user testing sessions recorded on video, a qualitative analysis of transcripts generated from interviews in the testing sessions, and a quantitative analysis of 64 user evaluation forms on which participants were asked to rate the four interactive news platforms on the basis of how useful, informative, easy to use and enjoyable they found them to be.The analysis of transcripts was done by identification of recurrent themes corresponding to each level of design in our framework.The use of these levels ensures that feedback is represented on a variety of aspects of the interaction, from the overall metaphor and value (at the product design level), to the details of interface and content, and the dynamics of the interaction that connects them.Although our main technical intervention involves adding audio and voting interactivity to paper, we do not confine our analysis to these aspects alone.This is because they involve design decisions and responses affecting print layout, content partitioning between print and audio, interface controls, challenges to the relationship between newspapers and radio, and so on; all of which are mixed up in participant responses.Collecting themes by levels in our framework allows us to organize these responses systematically and holistically.Hence, quotes on each theme were assembled into collections and are summarized below.We refer to each version of the newspaper shown in Table 1 and Fig. 3 with the following acronyms: LEPN=Lancashire Evening Post Newspaper (Fig. 3a), LEPW=Lancashire Evening Post printed-webpage (Fig. 3b), BPN=Blog Preston newspaper (Fig. 3c), BPW=Blog Preston printedwebpage.Occasionally the interview analysis led us to review parts of the original video corpus to see the context in which comments were made, especially in relation to the handling of the documents.This led specifically to some findings on grasping and handling the newspapers.The quantitative analysis was done by summarizing the rating scores and rankings, and conducting statistical tests of significance on group differences.We organize these findings now, according to the levels of Participants scored all four interactive newspapers highly on the different dimensions of user experience (see Fig. 4).This is an interesting finding in its own right for the interactive paper prototypes and approach.It suggests that despite their shortcomings in implementation, they were still seen as easy, informative, enjoyable and useful to use.
There were no significant differences between these scores for individual newspaper types on any dimension, according to four nonparametric Kruskal-Wallis tests (p ranging from 0.953 for Useful to 0.414 for Enjoyable).There were also no significant differences in the rank ordering of newspaper types at the end of the testing session, according to a chi-squared test, with ranks varying only between an average of 2.4 and 2.65.These quantitative findings suggest that there is no clear group preference for the format in which an interactive newspaper should be presented, or for the origin of the news content it relates to.Some stronger preferences were expressed by individuals in the interview data, complicating the picture somewhat.But on the whole, the approach can be seen to lend itself to a variety of news content and format types.

Value
When asked whether they would consider buying any of the interactive newspapers introduced during the trial sessions, responses from participants were mixed.Judgments were often made in terms of perceived value for money.These assessments were primarily based on quantity of content and this meant that the LEP was favoured over BP.Although the BP platforms offered an equal amount of interactivity to LEP, the LEP platforms included more printed text, and some participants felt that BPW in particular was 'too short to be a proper newspaper' and was 'likely to be free'.Interestingly, few participants thought that the interactive content of any of the newspapers justified an increase in cost to the consumer.
In addition to value for money, participants made judgements about the usefulness of interactive newspapers -and whether they would buy thembased on their use of Internet and digital technologies and, in particular, portable online devices such as iPhones, tablets and laptops.For example, Darren, 20, said 'I'd never buy this.I can't ever see a time when I would do that for news instead of an iPhone', and Lisa, 41, agreed that 'for me personally it isn't particularly useful because I do use online (news websites) a lot'.However, participants with a keen interest in local and community news were more optimistic about the potential of interactive newspaper technology.Andrea, 20, spoke at length about how the availability of interactive newspapers would encourage her to read more local news: 'It's just great… If you have newspapers like this I would be more willing to buy newspapers… I would (buy them) because I'm involved with Preston FM and… it's very important that I know about local news and I am engaged with community issues'.Like George, Andrea felt that interactive newspapers would be useful for reporting and consuming community-related, hyper-local information.

Potential uses
Whilst many participants felt that they would not buy interactive newspapers for their own personal use, several people felt that the technology would be useful in a variety of public contexts.Participants felt that BPW in particular could be used in health centres, museums, tourist information centres and art galleries.George, 58, envisaged BPW operating as an 'interactive information centre' where users of public services could access information regarding museum exhibits, local 'what's on' information, and details about health clinics and vaccinations, for example.
Other participants thought that interactive newspapers would be beneficial for people with learning difficulties and the partially sighted.The audio news stories were felt to be particularly useful in this respect.Jocelyn, 70, explained, 'with my dyslexia it would be great because I could do the reading whilst listening to it as well', and similarly, Ann, 58, suggested that 'it would be good if you can't read small writing.It… could help people with limited vision possibly, or (people) who are not very good at reading, or people who understand the spoken word rather than the written word'.Whilst these are important insights, it is implied in these statements that the audio content would be identical to the printed content but read aloud, therefore performing the function of a 'talking newspaper'.However, as will be discussed in the following section on content design, most other participants felt that it is important for the audio content of the interactive newspaper to bring 'something extra' or an 'added dimension' to the everyday newspaper experience by providing supplementary or additional information rather than simply reading printed news articles verbatim.
Another important way in which participants felt that interactive newspapers could be used was as a platform for sharing news about local arts and culture activities.Kate, 50, who is a member of a local drama group thought that the use of audio and voting combined with traditional newsprint stories would be an excellent way of raising the profile of local events and engaging with the local population.Similarly, Darren, 20, felt that visual artists and musicians in particular could benefit from the technology.After listening to the Olly Murs song on LEPN he explained, 'that is a really good idea because it's a new platform to promote music'.In addition, the landscape photographs on BPW really captured his imagination.He said 'that I actually find quite interesting because if there's a photographic contest then those (voting) details will go straight onto the site… which will be very interactive.I'd like to participate in something like that because it sounds… really engaging'.Kate and Darren both felt that the combination of audio, print and interactive voting features would be particularly beneficial for the local arts and culture scene.

Audio content
The audio content provided with the newspapers was the main channel by which additional value was created through interactivity, certainly in the short period of testing experienced by participants.Longer term values were also demonstrated by voting mechanisms but never really resulted in benefit to the user within the experimental context.Comments about the overall value and potential of the interactive newspapers in Section 8.1 therefore already relate to audio content to some extent.Here we unpack these perceptions with reference to specific comments about the properties of sound and particular audio associations as shown in Table 1.
Regarding the properties of sound while reading, many participants commented on its element of surprise.Sound is initially hidden from view behind the touch point buttons, creating an element of curiosity and suspense which invites touching.The fact that this is supported from a traditional printed newspaper heightened this feeling, and delayed some people discovering the audio links when presented with the first prototype.As Rebecca said succinctly 'How often can you pick up a newspaper and listen to a song through the newspaper?That's brilliant'.In other cases, people reported pressing a button by accident and being pleasantly surprised by the resulting audio.The downside of hidden content was also mentioned by participants.It was not possible to predict how long an audio clip is or to vary its speed of playback.Participants asked for a printed indication of the duration of sound clips, and controls for both speeding up and slowing down playback in tandem with their reading behaviours.
The multitasking aspects of reading and listening simultaneously were also found to be a benefit of audio.As Patch, 59, put it, 'I like the facility of being able to listen to the story rather than reading it because it gives time to look at other things that are happening on the page'.People reported multitasking with respect to the same article, as with reading the jilted lover article whilst listening to reader comments on it, and across different articles as a way of skimming more content in a given time.This raised comments about the similarity with listening to the radio whilst reading, but in a fashion that can potentially synchronise the content across both platforms.This conjured up a Sunday morning feeling for one participant, Christine, 28, who stressed the fit to relaxing and pottering at home: 'It seems to me to be more like a Sunday paper, in the morning, not going anywhere.Being at home rather than on the go.It might be while you are watching TV or whatever as well.To be sat down especially to listen to some of the articles like the interviews'.Several participants likened this to browsing social media, such as Rebecca, 22, 'It's not like reading a newspaper.Its becoming fully immersed because you can like things as on Facebook, you can read things, you can listen to songs, and you even listen to the stories whilst you're looking through it.To me that's really appealing'.
Although from one perspective the audio associations were seen to provide extra volume of content and therefore better overall value for money from the newspaper (see Section 8.1.2),from another perspective they offered economy of print.Some participants realized that more of the printed story could be shifted from text to speech, as in the Championsheep LIVE article that was longer in audio than print.This was attractive to people who disliked reading dense textual articles and preferred to read headings or pictures with associated spoken stories.This was expressed nicely by Christine, 28, who said 'To be honest if there's something where you can listen to it, I'm not necessarily going to read the text.It's nice to have a balance and seems a bit more modern.Nice pictures and it's a bit more colourful.More like a magazine than a newspaper'.The webpage versions of our newpapers began to give this economic feel to the printed articles, but this could be taken much further in future designs.In general, most participants couldn't see the point of audio associations which simply read aloud the printed article, as with the Superhero article, unless it was an aid for partially sighted or dyslexic readers.But reading aloud an unprinted article from a headline or picture was thought to be a good idea.
The most popular individual audio associations revealed a final property of sound that goes to the heart of the newspaper as journalism.These appeared to be the full speech by David Cameron on the article about his visit to the Far East, the interview with runners in the Preston marathon, and reader comments on the jilted lover article.All these clips introduced an authenticity to the articles given by independent non-journalist voices, which allowed participants to check the truth of the story or understand other perspectives on it.For example, Andrea 20 observed that 'usually in a news article you just get a couple of quotes, whereas if we had interactive newspapers like this we'd be able to have more analysis… and listen to the whole speech ourselves, judging it for ourselves'.Participants paid particular attention to who was speaking, their tone of voice and regional accent.Comments were made about the interesting mix of BBC voices in the 'additional information' clips and the accents of local people represented by the marathon runners and commenting readers.The latter served to give a regional slant on articles which participants liked.Added to this, human voices were said to personalize the printed content, as with the marathon runner comments on the race from an insider's point of view.

Advertisements
Few participants responded positively to the advertisements (both interactive and non-interactive) on the four prototypes.There were two general responses to the adverts: some participants demonstrated a strong negative reaction to them and found them irritating and distracting, and other participants demonstrated a general indifference and a disinclination to listen to them.Among those who demonstrated a strong negative response was Jocelyn, 70, who said 'I can't understand why they have an advert on the top right hand side.I'd prefer to see the headline, to see what it's about rather than advertising… I prefer it to concentrate on its message more than divert your eyes to irrelevant rubbish'.Similarly, when Ann, 58, was asked by the researcher why she had not listened to any of the audio advertisements she responded 'I can hear those on tele(vision) can't I? I haven't pressed them you're absolutely right.Why would I want to press on it?'These participants clearly feel that they are exposed to enough advertising elsewhere, and therefore would not choose to actively engage with advertisements on an interactive newspaper.
Other participants responded with more indifference, however.For example, Rebecca, 22, explained: 'I like the fact that I can choose to listen (to the advert) or completely skim over it if I want to'.Some suggested that although they had not listened to the advertisements, they would be open to listening if they were interested in buying the featured product or were shopping for a special offer or discount.A small number of participants did not notice that the BT (British Telecoms) advert on LEPN and BPN was interactive, perhaps due to the fact that they 'skimmed over' the adverts if they were not interested in the product.When the researcher asked Andrea, 20, why she had not pressed the 'listen' button on the advert she explained 'maybe because it's an advert… I didn't realize it was there'.
Participants' reluctance to listen to the advertisements raises some important questions regarding the usefulness of such advertising and the benefits for placing interactive adverts for the advertisers themselves.Our findings suggest that interactive newspaper readers are unlikely to choose to listen to audio adverts except on the relatively rare occasion when they are shopping for a particular deal or product.

Format
When it came to discussions about the advantages and disadvantages of the conventional newspaper format and the new tablet format, opinion was divided among the participants.There was a division between those who preferred a conventional newspaper format because 'it looks like a newspaper' and 'that's what I'm used to', and participants who felt that an interactive newspaper should be published in a new format in order to highlight its interactivity and because 'it is different'.Having analyzed the quantitative data, there is no clear or conclusive evidence that these opinions correlate with age.Younger participants were just as likely to express a preference for the conventional newspaper format as their older counterparts, and equally, older participants were as likely to favour the tablet format as younger participants.
When asked which of the prototypes would be her first choice suggesting that it is linked to Facebook… you kind of assume it is'.

Grasping and handling the newspapers
Observations from the video data revealed how participants handled the broadsheet formats.In each user trial, the researcher placed the newspapers on a table in front of the participant.When laid flat on the table, users generally used their index finger (or sometimes their middle finger) to press the interactive buttons.This method of handling changed, however, when participants picked the newspapers up and held them above their lap at waist/chest height.Importantly, we observed that when the newspapers are held up, users need to be able to 'pinch' the interactive buttons due to the lack of a hard surface to press against (see Fig. 5).This pinching action means that the interactive buttons are pressed with the thumb while the fingers provide resistance and support.In fact, this is a two-handed action with the other hand steadying the entire newspaper on the opposite page, mid-way up its margin.
This observation has some direct design implications regarding the location of the interactive buttons on these kinds of newspapers.Crucially, the 'pinch region' must not exceed the length of the thumb around the bottom and side edges of the newspaper.The top edge is out of bounds as it is not easy to reach over the top to pinch.Designers should also avoid placing touch/pinch regions in the centre of each side margin, since this is the area users hold to support the newspaper or turn its' pages.
Further data on handling emerged from observations of participants using BPW, perhaps because this two-sided (single-sheet) tabletsized interactive news publication was the most unique product design and participants had not handled anything similar before.When Benjamin, 72, was given BPW to explore, he initially tried to open it, perhaps because he could feel that there was (in our prototype) a single-sheet of paper folded and glued giving the impression of two pages and also because this is how he is used to handling a conventional newspaper -'opening' a paper to reveal more contents inside.Once he realized that the tablet was just two-sided, he turned the tablet horizontally (from left to right) which meant that the content on the back of the tablet was upside down.He then performed a series of quick rotations and turns (both horizontally and vertically) until the content on the back of the tablet was the correct way up.When he decided to go back to the front page, he turned the tablet horizontally again, and went through a similar process of rotations until the content on the front page was correctly oriented.Several other participants performed a similar series of actions.For example, Hayley, 29, picked up BPW with one hand and turned it horizontally so that the text on the back was upside down.She then flipped the tablet around in different directions until the content was the right way up.She suggested that it would be a good idea to 'have the content the other way round', and Ann, 58, agreed that it was easier to turn the tablet horizontally rather than vertically when holding the tablet with one hand.
The BPW prototype had been printed so that the user had to flip the tablet vertically (from top to bottom) for the content on the reverse side to be oriented correctly.However, most participants held BPW with one hand, which meant that by turning the wrist, it was easier to turn the tablet horizontally (from left to right).By turning the tablet this way the text on the reverse side was upside down.A smaller number of participants who held the tablet with two hands (usually on the bottom corners) were more likely to flip it vertically, but as the quotes from Ann and Hayley suggest, orienting the text in the same direction on the front and the back of the tablet makes it easier to use.

Discussion
We began this article by arguing that paper still has valuable properties as a reading medium and might be designed to inter-work more intimately with various digital resources around it, especially audio.Our implementation of a talking newspaper embodied this assertion, and raised issues of how to design hybrid interactions and content using capacitive touch technology in the paper itself.Limiting interactivity to simple voting actions and audio associations, we designed four different documents borrowing design conventions from paper and the web for both professional and citizen news.Responses to these documents were complex, but reveal a number of insights into the design of interactive newsprint and its prospects in local journalism.
We organize the lessons of the study in four parts in the following subsections.The first set of lessons is derived from the design research involved in creating the prototypes, and have to do with the technology and design space involved.The second set is derived from the study findings, and concern the user experience with interactive newsprint.The third set of lessons is recommendations for design, while the fourth set comes from limitations of the study and implications for future research.

Lessons from the prototyping
At the outset of the project we expected to be able to create multipage interactive documents for field trial, partly based on predictions of the printed electronics industry regarding the printing of electronic circuits, batteries and interactive components.While these predictions do indeed relate to cutting edge research in the field, we found that in practice it was difficult to assemble robust working prototypes with off the shelf components, even in the kind of short run volumes needed for field trials.For this reason we were forced to create four experience prototypes with only localized interactivity for lab testing.This is a limitation of the study discussed in Section 9.4 below, which indicates that this technology is not yet ready for the big time in the journalism domain just yet.
One of the challenges is that print newspapers are made out of the thinnest and cheapest forms of paper on the market, making it difficult to print on and providing little scope for attaching additional electronic components.High quality magazines with thicker, stronger paper would be better, or books with spines that could be used for hiding electronic components.As we explored the form factors for our prototypes and considered the 'tablet' size and construction, we realized that innovation is needed at a format as well as a content level, to overcome the limitations of current technology until better printed solutions arrive.The ability to create a thick interactive leaflet in the BPW prototype, where the electronics are hidden inside a folded sheet of paper, is a step in the right direction and a contribution of its own to the design of such documents.However, much more radical form factors for printed news might be explored, to challenge what a newspaper is and might be.

Lessons from the user study
A variety of lessons for the design of interactive newsprint and augmented paper documents more generally, emerged from findings on the user experience with our prototypes.Five in particular are worthy of discussion here.
First, we found that participants came to the documents with multiple expectations about their provenance, meaning and affordances, inherited from prior experiences with printed and on-line news.This was overwhelmingly demonstrated by discussions of the documents' publication title, but also by myriad features of their particular design and behavior.Brand values associated with the Lancashire Evening Post immediately indicated that the two LEP documents were more professional and 'serious' than the Blog Preston documents.Interestingly this was reinforced by the 'proper' newspaper format (LEPN) but not the webpage format (LEPW), which alienated conventional newspaper readers.The very same impressions attracted some participants favouring citizen news and the on-line medium, to the BPN and BPW documents.Subtler expectations could be seen in use of the 'Like' icons and the printed phrase 'More' at the end of some articles.Participants were frustrated at the lack of response to pressing the Like icons, borrowed from Facebook, and confused about the link to Facebook itself.They also tended to press the 'More' labels which were non-interactive, instead of turning over to continue reading the rest of the article.
These expectations govern cultural forms of reading and communication, and cannot be ignored in the design of any new medium of this kind.They show how new media design itself involves a kind of technocultural innovation discussed by Balsamo (2011), in which the imagination of technologists needs to be combined with the cultural production of content creatorsas on the Xerox PARC RED project.A similar point is illustrated in the following quote by Murray (2012) who argues that a medium is both material and cultural: "a stone and chisel only become a medium for writing when society develops practices of marking the stone and interpreting the chisel marks".In our experiment to test out pairwise combinations of print and web news, we seem to have designed some marks which were indecipherable or read incorrectly.In general, most confusion was associated with the LEPW and BPN combinations (Fig. 3b and c), which went against the grain of the news origin and its usual presentation.In contrast, the addition of interactive functionality to a traditional-looking newspaper (Fig. 3a), delighted many readers, as did the notion of printing an online newssheet in tablet form (Fig. 3d).
Second, we found that audio annotations were most powerful when they acted to extend and personalize the printed content of the newspapers with non-journalistic voices.These were most dramatically illustrated by access to spoken interviews with David Cameron or the Preston marathon runners whose comments either validated the journalist's report or extended it with an insider's perspective.The tone and accent of voices appeared to be particularly valuable in conveying emotion or establishing a common regional identity with the readers.Similar effects were reported for readers' comments on the jilted lover article, in a way that is not usually exploited in textual responses to online news.In fact, one implication of the findings of this study is that audio annotation appears underused on the web itself, and might be explored more systematically in relation to online documents of all kinds.
Third, new kinds of synchronized reading-and-listening activities were encountered by participants when accessing audio annotations.They found that they could multi-task not only in listening to audio associated with an article they were reading, but also while reading ahead to other articles.This raised the metaphor of radio listening and the possibility of designing a newspaper to be read with the radio.This is an intriguing idea that deserves further attention, since it reverses the primacy of text and audio explored in this study.Instead of annotating printed text and graphics with audio through interactive newsprint, perhaps we should be annotating radio programmes with the printed word.Furthermore the very notion of annotation is challenged by these findings, which suggest that a looser association of the two media streams might be more attractive and creative.Rather than elaborating a textual or spoken story in the other medium, perhaps it is better to allow different stories to be browsed simultaneously in each medium.This may result in serendipitous connections and associations not built into the stories themselves but emerging out of a more fluid interaction with them both.In this age of media convergence and overload, people appear to be developing new capacities to watch, listen and read to multiple things at the same time.Development of interactive newsprint should attend to this trend and play into it in an appealing way.
Fourth, the value of online voting through the newspaper was immediately understood by participants familiar with similar behaviours on social media sites.However, unlike those sites, our newspapers provided no feedback to the user on voting tallys or identities.This reduced the motivation to vote in our study and might eventually lead to its abandonment in a real life deployment, unless audio feedback could be given or further motivation provided.Given the democratic importance of voting mechanisms for engaging the public in politics and local decision making, greater attention should be paid to feeding back the results and consequences of votes to readers, perhaps through cycles of reporting in newspapers themselves.The possibility of using the newspaper as a web-connected questionnaire holds great potential for local democracy, but more work is required to understand how this should be designed as an information system involving the two-way exchange of information over a cycle of publications and encounters (c.f.Taylor et al., 2012).
The fifth lesson is related to the fourth.Our findings begin to outline broader prospects for the future of the news and information sector in which our research was situated.Although we contrasted responses to separate professional and citizen news, the bigger opportunity appears to be in their combination.This could be seen in the attraction of personalizing professional newsprint with the human voice, voting to raise the profile of local issues, and creating an accessible printed version of an on-line blog and radio show.The fact that hyper-local news is springing up on the web, is often seen as a threat by local publishers.However, interactive newsprint provides a vehicle by which publishers might partner with local community groups to connect professional and grassroots content in new ways, and generate hybrid publications together to the benefit of both (c.f.Gillmor, 2006).Community radio content is a key ingredient in this mix, as is the ability to record verbal responses to the newspaper and refresh published audio content over time.In short, this work suggests more nuanced possibilities for transmedia publishing across print and online platforms, that combine visual and auditory information in new ways, rather than a simple replacement of printed newspapers by the web.

Design recommendations
We believe our design framework has been useful in separating out some of the varying levels of response to interactive documents, and helping us to identify design recommendations at each level.Further research is needed on these before more general design principles can be derived, as discussed in the next subsection.Hence the current recommendations for Product Design, UI design and Interaction Design apply most directly to the kind of physically augmented paper we tested here.Recommendations on Content design, by definition, apply to the design of audio-augmented content and voting on both printed and screen-based news content.
Product design 1. Audio-augmented newspapers should be designed in newspaper form to provide 'bonus' information and the chance to respond.2. Audio-augmented leaflets might be valuable in public settings where information is shared and discussed.3. Audio-augmented paper represents a new opportunity for promoting music through its' packaging.

UI design
4. Develop standardized touch point icons and print conventions to indicate interactive areas and functionalities.5. Clearly indicate the type and length of hidden content.
6. Indicate the destination of voting information.

Interaction design
7. Give immediate audio-feedback on touch actions over the paper.8. Provide in situ audio control for play and pause.9. Place touch points in the margins of pages within thumbs-reach.10.Avoid placing touch points in the same place on both sides of the paper.

Content design
11. Make print and audio content complementary, playing to the strengths of both media.12. Use recorded interview content to increase the trustworthiness of news stories.13.Personalize stories with the spoken voice.14.Provide reader incentives to vote.

Future research
Some limitations of the current study were mentioned in Section 9.1 above when discussing the design of the prototypes.These were suitable only for testing in laboratory conditions, albeit a comfortable laboratory with sofas and coffee tables.The results are therefore based only on short encounters with interactive newsprint, and comparisons within the medium between different versions of the same idea.Two obvious extensions of this are suggested for future research.First, there should be attempts to test the approach in more naturalistic settings over longer periods of time.This implies the need for a field trial in which participants can experience more than one delivery of an augmented newspaper into a home setting where multiple people can read and discuss it over time.This would also provide the opportunity for participants to engage with voting in a more realistic way with feedback on the effect of voting provided in the trial period.Second, there should be comparison across media, to compare this approach with interactive online news sites with similar properties.This would determine whether the positive reactions of participants in this study, were to new functionalities that could be provided just as well online, or to behaviours that they value intrinsically as extensions of printed newspapers they would like to keep reading.
Additional future research suggested by the findings of the research itself has already been mentioned briefly above.It includes attempts to explore novel formats for interactive paper that overcome limitations in the technology itself, innovation in the synchronized combination of radio and newspaper content, and a re-thinking of the roles of professional and community reporters in the assembly of community news.Many of these enterprises stray outside the fields of human computer interaction and media design.However, this only goes to show the relevance of those fields to other walks of life and the need to seriously engage with application domains as we have done in this paper.

Conclusion
This work supports the use of printed electronics to enable a range of audio-paper products within and beyond the news domain.We have shown the technical feasibility of relaying touch information off paper to a nearby mobile device, and the attraction of some kinds of audio to enhance the printed word.This extends a large body of previous work cited above, to a new domain and technology.The additional opportunity to respond to print articles through voting actions met with less enthusiasm, but this may have been an artifact of the experimental setting and the lack of any incentive to vote.The cost and feasibility of printing such documents at scale is clearly a mitigating factor for the audio-paper market, but the same challenges face other forms of augmentation in the internet of things.As advances in nanotechnology continue to enable smaller conductive elements to be laid down on a variety of substrates, we may soon see the emergence of a nextgeneration paper with a whole new lease of digital life.

Fig. 1 .
Fig.1.A framework for understanding and designing interactive paper documents.

Fig. 2 .
Fig. 2. The interactive newsprint electronics for a front page of the Lancashire Evening post.

Fig. 5 .
Fig. 5.The pinching of an interactive region (top right) with the newspaper held off the table.

Table 1
Interactive associations used on Lancashire Evening Post and Blog Preston prototypes.