CitySkin: which color is your city?

This paper discusses an experimental mobile application named CitySkin. The application uses mobile phones for data retrieval and mapping, to design an invisible skin of a city (an output map) following a user journey at a given moment in time. The project explores relations between subjectivity and raw data by combining hard data with visual mapping. Cities and their intrinsic diversity can be compared. Slightly different input variables can present greater changes in a recurrent path. CitySkin records the mood of a specific dérive and prompts consideration of the cultural implications of computing and the design of its ubiquity.


Introduction
According to Weiser and Brown, the concept of ubiquitous computing (U.C.) aims to create "a calm computing" by having computers disappear [1].This is made possible through the integration of computing in the built environment.It is argued that U.C offers a perspective that emphasizes human and social aspects, presenting computation as an open definition [2] challenging its terms, significance and appearance.
U.C. discussion translates into projects that combine research and life.For instance, contributions can be observed given by the progressive miniaturization of sensors and actuators, as to the exploration of smart materials [3] but also by applying natural structures to design as is the case of Neri Oxman [4] as well as Maeda's bits, atoms and crafts research.Maeda began as an engineer, then was inspired by art and design, and latter invested in research, business, and tech [5].These relations inspire hybrids of form and nature.Moreover, some authors [6] argue that we are fully living in a U.C. era, considering that this ubiquity of computation is made real with the use of smartphones.
The CitySkin research project acknowledges this last statement and aims to contribute with a proposal that tackles the hybridization of art with science.
In the last decade, mobile phones have become a common tool for communication in both postindustrialized [7] and developing countries.When analyzing the spread of mobile phones use in a globalized world, a significant factor in their appearance is the emotional tie and social connectivity given by a personal object.But on the other hand, the inequity regarding access to different mobile phone features, and quality network coverage are definitive factors that differentiate audiences.It is relevant to say that smartphones are still a "first world" tool, which translates to contextualize CitySkin in terms of accessibility.
Smartphones are tools that combine computing, mobility, and mapping possibilities.Beyond having an exponential computing power, smartphones commonly have features such as WI-FI, 4G and 5G network connectivity, GPS, accelerometers and high quality cameras,.These particular features offer a realm of possible combinations.
The CitySkin project addresses the consequences of digitalization of information and big data.Tools such as smartphones have created an exponential ability to collect and store massive amounts of information.On one hand, this information generation has highlighted the need to improve its readability.The exponential growth of information has found in graphic visualization a model to simplify the interpretation of data complexity.Data visualization not only improves reading, it creates rich aesthetic experiences, adding new perspectives to visual and cultural discussion while conveying digital information.All these solutions have been made possible by mutual contributions arising from computer science and art.Examples of these prevailing collaborations can be found in current definition of design made by institutions such as New York's MoMA, in the Design and the Elastic Mind [8], Talk To Me [9] and Design and Violence [10] exhibitions commissioned by Antonelli or in the Linz's Ars Electronica [11] a festival where, since the 70's, science and art collaboration is discussed and celebrated.Data Visualization is also a prevailing visual experience distributed in Internet under several categories and by a myriad of authors.
CitySkin's conceptual design is inspired by these approaches, and uses computation to measure and visualize routes inside the city.It references mapping, data visualization, and digital art critique [12] [13] [14], but also makes a contribution to shifting concepts of computing.Cityskin visualizes hard data by tracking a literal and psychogeographic journey, considering human subjectivity, in an implicit invitation for dérive, surprise and improvisation [15].

Related Work
A background to this work in experimental mobile applications is to be found in the context of digital art.Since the beginning of the public internet, terms like "internet art" attributed to extended nomadic networks have emerged and have been explored by digital artists.In this context, the particular designation of "software art", described work that referenced formal outputs given by computational instructions.
Since the 90's, artists like Golan Levin have created interactive software that allowed the manipulation visuals and sounds in real time.Telesymphony is a project that extends Golan's work to nomadic devices.Sound is generated by the ringtones of audience's mobile devices [16].
Mapping is an excellent conceptual tool to understand and cope with urban landscape information.Considering U.C. and Data Visualization strategies, cities that dwell at the interchange of massive quantities of information, are becoming progressively U-Cities [17], meaning, Ubiquitous Cities.MIT SENSEable City Lab [18] has contributed to U-Cities research with several projects.One of these projects, Trash Track, attaches sensors to discarded objects and maps their national and international journeys.The output map gives awareness to the existence of long trails, as opposed to a more efficient proximity systems, indicating waste of resources in an immediate way.Projects like Pedro Cruz's [19] Lisbon blood vessels uses veins in circulatory system as a metaphor to visualize, with aesthetic appeal, Lisbon's traffic flow.This method can also give real-time valuable information to drivers.
A growing number of designers and researchers are using data visualization techniques for artistic expression, but also the particular features on mobile phones as survey machines.GPS, wi-fi, embedded camera, computation, in addition to mobility, are presently used and mixed in different approaches.MobiSpray by Jürgen Scheible, for example, uses mobile phones as an artistic tool to paint digital graffiti.Scheible created a client server application that uses mobiles as gesture-control [20].Mobiles are used as pointing mechanisms drawing on a video-projection, thus, creating digital public art.Large scale drawing using mobile devices is another example of GPS being used for artistic expression, and a concept presented by several artists [21].
Travelling inside a city deals with relations between time and space, i.e. geography, time measurement and a less obvious category, a degree of fun.Enjoyment is addressed by Mark Shepard's experimental mobile application Serendipitor [22] and Atau Tanaka and Petra Gemeinboeck in "Net derives" [23].Serendipitor is part of a broader project, Sentient City, which tackles the design of the city of the future through an i-Phone application that invites the user to explore different paths in a city map.It calculates alternative ways to get to a particular part of the city, with inherent proposals to diverge."Net derive", also follows the concept of the dérive (adapted from the post-war situationist practice), transforming the city in an instrument.Mobile phones, using GPS position, camera and microphone, exchange information between spectators in a gallery and three participants in the streets of a city.Sounds and pictures from the streets become information to visualize and sonificate locations.
Finally, two experimental mobile applications from Japanese company Aircord, show how playfulness can be aesthetically relevant and simple [24].The first virtual free runner is an animated man who reacts to the accelerometer with a tap to jump button, to be used with a projector.Describing time visually has been a concern with clear practical implications in human life since Ancient times.Current time measurements have implicit computations based in sky observations.It is relevant to look into history of science and math.These time divisions originated from computations made in the beginning of civilization, 4000 years ago, in Mesopotamia.Surviving Babylonians' clay tablets records show cumulative data from sky observations that throughout time allowed predicting celestial phenomenon with precision.For instance, it was an ancient civilization that gave 7 days to the week.A copy of the so-called Venus Tablet of Ammisaduqa (detail) The British Museum, retrieved from https://www.britishmuseum.org/research/collection_online/collection_object_details.aspx?objectId=314745&partI

Art Concept
The number seven had a mystical significance to Babylonians.It was associated with the seven heavenly bodies: the sun, Mars, Mercury, Jupiter, Venus and Saturn.As such, the measurements of the week that are still in use today are based in astronomical observations made in ancient past.
Mesopotamians' great mathematics put forward a 60 base system, which allowed defining the sixty minutes in the hour, and the 360 degrees of the circle [25].The sexagesimal system is useful to measure angles, geographic coordinates, and time.Mesopotamian representation of time was created from a circle division, having different attributions to year, month and day.These divisions of time were also based in astronomical observations.For instance, the month division correlates with the observation of the moon.To some extent, and considering some alterations, this system created by ancient mathematicians is still used embedded in our high tech life, and use in the apps in our mobiles phones.
This historical background provides an understanding of the cultural implications of time division, but also indicates a clear relation to direct observation of natural elements.These divisions depend on interpretation and recording of celestial moving objects and therefore have an implicit design (Fig. 1).
CitySkin is also inspired by the convention of time measurement that relies on observation.Using mobile cameras as a metaphor of the eye, and giving direct relation with geographic space, by GPS location recording, CitySkin provides visual maps of travels that connect locations and time.It uses a predefined computational model (based in the number 60 as a direct quote of Babylonian measurements).The result is a graphic map that will give an impression of a particular city at a given time.

Prototyping
The prototyping studies for CitySkin began by physically travelling the city by car, and defining a path.First trial was a crossing over Tagus River, from the South bank, the city of Almada, to Lisbon, in the North Bank (Fig. 2).The journey took 33 minutes.

Fig. 2. Prototyping the journey between Almada and Lisbon
Throughout this journey, one photo was taken roughly each of the 33 minutes using a mobile phone (Fig. 3).These photos were used to output map studies, by stretching, adding filters, and searching for a visual result that was focused on color.Mainly, creating a map of colors predominance within a specific trip (Fig. 4, Fig. 5).

Design
The application design follows the prototyping results.The final graphic map (skin) results from photos taken during an up to 60-minute journey, by foot or in a vehicle.Cityskin takes a photo each minute and has its location recorded.The difference between position A and B will define the velocity and this value determines each photo final length (Fig. 6).A median filter is applied to each photo, in order to emphasize color, and its length is compressed or stretched according to the velocity variables.These images are lined horizontally, and a white space is kept between them.The white space adds readability to each picture, but also becomes an editable input text space.

Fig. 6. Image length computation
GPS coordinates are presented as default text between stripes.The users are allowed to substitute this geographic information by editing their own text labelling in each white line (Fig. 7).The final map results from a representation of hard data and open variables related to movement and color -visual impressions, time, type of transportation and user input.
The final map will be presented as a stripe of colors, showing long stripes when the user is moving faster, and narrower stripes when the user is moving slower.

Technology
The first implementation test was made for iOS.CitySkin output corresponds to a single JPG image file (Fig. 9, Fig. 10), containing the GPS coordinates of each of the images captured, with photos taken each minute.The final image directly represents the path taken by the user, synthesizing the user perspective and the particular variables associated to the travel.
This final skin output can be shared.The jpg format image is aimed to either be published in socials networks or sent via e-mail.

Designing relations: objective derive
Quotidian journeys are often a routine experience where landscapes blur into oblivion.CitySkin can provide insights about different layers of perception, and a singular perspective.The application invites the user to find different maps around the known but also unknown places.
The measurement method is inspired by Babylonian direct observation of celestial phenomena.As such, Cityskin is offering an interpretation on data given from visual cues.CitySkin can visualize and find interesting differences, coincidence or patterns on journey maps.
It becomes possible to compare visuals from different cities, but also to compare the subjective variables given by an individual journey.These changes can be given by time spent in different locations, or even provoked by the user's imagination.
CitySkin was designed considering that identical paths would provide completely different maps accordingly to the use, emphasizing the differences around the experience found in routine.It also contributes to smart cities critic, in particular the case of ruling from hard data, instead of having a citizen centred approach [26].CitySkin presents a graphic perspective of a city dwelling.Results vary according to colors and velocity.As demonstrated in figure 9 and figure 10, it is difficult to repeat an equal image for the same route.This is one of the most impacting results.

Comparing cityskins
CitySkin application allows to ask "which color is your city".The answers show where a driver or walker spends more o less time in a traffic jam, or journey through the city.Each picture length conveys information.Short lines indicate that the driver or walker has stopped.Several lines indicate where and how each minute is spent.One of the main features of the CitySkin application (Fig. 11) is the ability for users to share outputs.This feature is designed to support a ritual of communication, this being, according to Mikko Villi [27] often more important than the photographic quality itself.Also, it has been noticed that photo sharing rituals are followed by text message practices, i.e. often images do not substitute texting.

Sharing images
CitySkin combines image and texting, and opens directions and opportunity for experimentation concerning the combined use of image and texting, as a common way of communication.

Algorithmic flowchart
CitySkin's algorithm is presented by the flowchart (Fig. 12).The start menu allows to access to an information button.The application suggests to answer to the question of "what color is your city?" (Fig. 8).The next screen offers the possibility to choose between "walk" or "drive" (Fig. 13).This allows users to adapt an average velocity, for either scenario.The application will then start taking a photo each minute.During the process it is possible to edit a CitySkin´s GPS position by text, and return back to the picture taking mode.The process can be stopped at any time.At this point, it is possible to share or restart.Sharing allows one to save to the phone gallery, social media, or go back.The button "save to website" is drawn and refers to future work, but is not yet implemented.

Human variables and algorithm
CitySkin's visual variations depend on velocity, which determines each stripe picture's length.This variation, however, brought the necessity to distinguish walking from a vehicle journey, because velocity has implications in the final design (Fig. 13).Accessing two different algorithms solved the issue, with the user input determining velocity, either walking or travelling in a vehicle.location has problems.It is common for mobile phones to lose their signal indoors, making GPS based projects only suitable for outdoors.The lack of accuracy is also observable.
CitySkin invites users to test the application in fair conditions.When that is impossible, the geographic information can be manually inputted as referenced.In the absence of the user labelling, CitySkin will use an average measurement.

Future work
CitySkin is a work-in-progress project.Next steps will include system evaluation aiming to refine the user experience.Also, after the initial iOS prototype, it will be made accessible to other platforms, such as Android and Windows.Also we aim to integrate the distribution of Cityskin's output in the website.

Conclusion
The Cityskin project tests relations between computation and art, acknowledging the incremental step toward computational ubiquity allowed by mobile phones.Ubiquitous Computing is addressed by this application, considering the city's intelligence.In a broad sense the project proposes to test playfulness and a sense of discovery, thus, giving focus on the user experience.The project also aims to challenge the dominant discourse of the "smart city", with its tendency toward quantitative data (traffic flows, phone data, revenues, etc) which arguably leads to an impoverished sense of the city as experienced by its users.Might smart cities instead learn from the post-war practice of the Situationist derive.
CitySkin produces outputs which give visibility to invisible layers present in quotidian life, adding a cultural impression to design and computation.
Each image reflects the point of view of a user along a path.There will be differences in colors and distortion on the output image (skin).Each skin will be unique and will reflect the singular point of view of the user's time, place and playfulness.
CitySkin also provides a means to compare and experiment with different times of the day or year of a specific place, but also between different cities.Furthermore, this application allows recognizing patterns of time.Finally, the information presented in the final map, can give the user a visual and immediate way to evaluate activities that relate to routine and movement.This information presentation benefits from a comparative evaluation, like for instance physical activities or by visualizing traffic jams.Cityskin is a tool to measure the quotidian qualitatively and quantitatively.
Capturing the experienced sense of time, which has periods perceived as blanks or non-places, as opposed to measured time, is ultimately one of the useful contributions of this application.Is this case, the challenge is to re-capture the perception of fleeting time, specifically showing variables that are not obvious to the user.Thus, add a sense of wonder or fun, to an often called draining, empty experience that is commuting, or even register layers of perception while travelling un/familiar places.Finally.CitySkin suggests to slow down, and embrace contemplation.

Fig. 1 .
Fig. 1.Part of a clay tablet, 3 pieces, Neo-Assyrian.A copy of the so-called Venus Tablet of Ammisaduqa (detail) The British Museum, retrieved from