“MORPHING BOX” – AN INTERACTIVE APPLICATION FOR USE IN MUSEUMS

: This paper presents an approach to developing a morphing application for use in museums in order to involve visitors directly in an exhibition and to increase their user experience. The visitors take an active part in generating a unique image from their own portrait and a selected reference picture. Techniques used for the development of this interactive prototype include face detection and morphing algorithms.


INTRODUCTION
As a result of recent developments in the field of information and communication technology, the computer has become a widely used tool to support specific activities in people's everyday lives. At this, the mediation of content occurs with the help of multimedia-based applications that allow active user participation. With the use of new technologies, it is possible to make a visit to a museum more attractive by integrating interactive elements that enhance user experience. According to Klinkhammer and Reiterer [1], the approach of user experience includes several aspects of usability engineering, as well as the factor of fun and entertainment, to create an enjoyable exhibition for visitors. User experience is subjective and can vary between people due to personal experiences and situations [2]. Consequently, it is beneficial to develop an application by means of the iterative user-centred design process [3], in which representatives of the potential target group participate in several pilot studies so that the system can be improved according to their feedback. The prototype "Morphing Box" was specifically produced for use in museums to encourage discussions about different identities. In this context, visitors can generate a unique image that combines their own portrait and a selected reference picture by means of the graphical user interface of this interactive application.

CONCEPTS AND FUNCTIONALITY
According to Heath and vom Lehn [4], the use of new technologies in museums is currently strengthened as a means of enabling possible user interactions and arousing interest in scientific topics. The screen-based prototype "Morphing Box" was developed as a sample application for an exhibition on the subject of identity. Visitors of all ages represent the defined target group. They may create a unique image by morphing their own portrait with a selected reference image provided by the system to generate a final picture. In order to utilise the whole functional range of this application, visitors perform different steps illustrated in the following figure (see Fig. 1).

SYSTEM ARCHITECTURE AND PRO-DUCTION OF REFERENCE POINTS
A computer and a monitor with an integrated webcam are made available to visitors who can interact with a graphical user interface by means of a touchscreen. The following figure shows a detailed overview of the system architecture (see Fig. 2).

Fig. 2 -System architecture
As displayed above, the camera is needed to take an output image, the visitor's portrait, whereas scaling and colour adjustment are automatically carried out by the system. The open source library OpenCV is used for detecting prominent facial characteristics by means of specific algorithms. At this, particular haar cascades were implemented for identifying the facial structure first. Additionally, the eyes, nose and mouth of the visitor are detected as further relevant facial features (see Fig. 3).

Fig. 3 -Face detection
On the basis of these identified characteristics, several reference points are automatically positioned in the output picture. The same facial features are detected in each of the reference images so that corresponding reference points are automatically defined in all preprocessed pictures that are stored in the database. Consequently, the user's portrait can be combined with the selected reference image to create a final image in the later process, by comparing specific reference points of both images with each other. The following paragraphs describe in detail the functional range of this interactive application "Morphing Box".

FUNCTIONAL RANGE
The welcome page allows users to select their gender so that suitable reference images that are stored in a database can be displayed in the subsequent course of the morphing process. The next step is the recording of the visitor's portrait image. To do this, the head needs to be positioned in the predefined frame so that scaling and proportions are in agreement in both the portrait and the reference picture (see Fig. 4).

Fig. 4 -Image recording
Users may then choose one of the four continents Europe, Africa, Asia and South America in order to see themselves in a different identity and to engage in the subject of appearance and culture (see Fig. 5).

Fig. 5 -Selection of identity
Based on the specified gender and the selected continent four different reference images, stored in a database, are displayed (see Fig. 6).

Fig. 6 -Selection of reference image
After choosing one of these reference images, the morphing process starts and the unique, final picture is generated according to specific parameter values. As explained earlier, the final image represents a combination of the user's portrait and the selected reference picture. At this stage, visitors have the option to define and to modify the percentage of facial characteristics and skin colour information with two scales provided by the system (see Fig. 7).

Fig. 7 -Final result
Additionally, users can go back to the start page by clicking the button in the top right corner.

METHODS AND TECHNIQUES
Based on the concept described above, a prototype was developed using the programming language C++ and the framework Qt. This application implements different face detection algorithms provided by the OpenCV library in order to automatically position corresponding reference points in both the user's own portrait and the selected reference image [5,6,7]. Different methods and algorithms used for the development of the interactive prototype "Morphing Box" are explained thoroughly in the following subsections.

CONCEPT OF INTERACTION
According to Sauter [8], interactive information media belong to the standard of each modern museum. Therefore, in relation to the exhibition on the subject of identity, knowledge transfer occurs with the help of multimedia-based systems so that users are allowed to take part in the exposition and to interact with the applications. Consequently, visitors are actively involved in an enjoyable manner and they have the option to exert influence on the final result by generating a unique image that combines their own portrait and a selected reference picture. Active user participation is a significant factor in enhancing user experience and arousing interest in scientific topics. Besides the individual interaction between user and system, the component of social interaction is also very important and needed to be taken into consideration when developing the screen-based prototype. In this case, intensified discussions about different identities are encouraged as a result of information exchange between several visitors debating and analysing the obtained outcomes.

MORPHING PROCESS
The morphing process is controlled by two parameters. Firstly, the Position Morphing Factor P specifies the transposition of the reference points in the output image into the reference picture based on the user's settings. Secondly, the Colour Morphing Factor C describes the colour scheme of the final image that combines both the portrait and the reference picture.
The most important difference in morphing two images compared to the common cross fade is the fact that several reference points need to be identified in both pictures to highlight corresponding image areas. During the morphing process, the output image is transformed into the reference image in relation to colour and position information of the specified reference points. Therefore a distortion of the output image occurs as well as a colour adjustment. In the standard morphing process these two aspects usually take place simultaneously in respect of one factor that specifies the transformation from the output into the reference image.
As a result of setting the factor to 50%, for example, colour and position information of the reference points highlighted in the output image are adjusted to the half of the values in the selected reference image (see Fig. 8).

Fig. 8 -Morphing process -Factor of 50% for colour and position information
In relation to the development of the screenbased prototype, the morphing algorithm described above was modified. As a result of extending the process, the position of the reference points can be changed independently from colour information. Consequently, this aspect is very useful for the accomplishment of face morphing because relevant facial characteristics of the visitor's portrait image are retained (see Fig. 9).

Fig. 9 -Morphing process -Factor of 90% for colour and 10% for position information
As mentioned above, the morphing process is controlled by two parameters, the Position Morphing Factor P and the Colour Morphing Factor C.
There exist n reference points which are highlighted in the output and in the reference image based on significant facial characteristics as explained in section 2.2. Each reference point in the portrait is definitively linked to a corresponding reference point in the reference image.
In the first step, the grid of points of the final image Z is produced on the basis of the grid of points of the output image A using the Morphing Position Factor P. At this stage, the subtraction of each reference point in A (A i ) from the corresponding point in the reference image R (R i ) multiplied by the value of P is added on to A i : Consequently, a grid of points exists for the final image after all reference points have been transformed. So the calculation of colour information of individual pixels for each column (x) and row (y) starts. First, it is necessary to identify the corresponding point for each pixel in the output image. It is not difficult in the case of the reference points Z i because the corresponding reference point is given in A i : For all other pixels, the reference points need to be calculated using interpolation. In this situation the Delaunay triangulation is applied to the grid of reference points in order to identify several triangles. Each point sought (Zi) is assigned to just one triangle in which it is located. A 2x3 matrix M expresses the transformation from A i to Z i so that the following statement can be applied: As in this case the exact position of Z i is known, the formula can be altered in the following way so that A i is the result of applying M to a reference point Z i : In order to calculate A i , it is necessary to search for the coefficients of M. To do this, the reference points of the triangle in which the point Z i is located are used, whereby the position of Z i is being defined by the bijective relation of reference points from A to Z. For these reference points, the transformation from A to Z can also be applied as described in the formula above. Consequently, a system of equations can be set up in which the unknown factors can be identified using Gaussian elimination. If the elements of the transformation matrix M are found, the reference point A i can be calculated. As a result of applying the same procedure between the output and the reference image, the corresponding point concerning A i (R i ) can be calculated. When the position of Z i is known in the output and in the reference image, the colour value of Z i is determined by the colour values of A i and R i as well as the Colour Morphing Factor C.
Face morphing using manual determination of reference points is illustrated with several screenshots in the following figures (see Fig. 10 -13). Figures 10 and 11 represent the use of the system by female and male museum visitors. The user's portrait is placed on the left hand side and the selected reference image is automatically positioned on the right hand side. Users then have the option to alter different parameter values by using two scales located below the pictures in order to determine the percentage of facial characteristics and colour value that should influence the final image (see Fig. 10  and 11).  Figures 12 and 13 illustrate the morphing process implemented in the system. In accordance with the specified parameter values defined by two scales below the pictures, the final image that combines the portrait and the reference picture is generated in the middle of the application. Users have the option to modify the percentage of facial characteristics and colour value at any time by moving the sliders more to the left or more to the right side in order to alter the final image (see Fig. 12 and 13).

MANUAL AND AUTOMATIC POSI-TIONING OF REFERENCE POINTS
In order to obtain good results within the morphing process, it is necessary to set a large amount of reference points that are positioned at prominent facial characteristics. In this context, several problems occur as the field of application includes museums where the public, especially children, need to interact with the system. The positioning of sufficient reference points in adequate time is difficult for the unpractised user. Additionally, the wrong placement of reference points leads to considerable distortions in the final image. In order to solve this problem, a change was implemented during the development process of the interactive application so that prominent facial characteristics are extracted by means of face detection algorithms and reference points are automatically placed on them. For identifying these characteristics, the open source library OpenCV was used. To do this, the interactive application identifies first the shape of the user's face before the eyes are highlighted as an additional feature. Then the nose and the mouth of the museum visitor are detected as further facial characteristics.
On the basis of these identified features, the positioning of reference points is automatically carried out so that the output image can be combined with the selected reference picture to produce a final image in the subsequent process.

CONCLUSIONS AND OUTLOOK
The screen-based prototype "Morphing Box" was developed as a sample application for use in museums considering different methods and techniques in order to make an exhibition, on the subject of identity, more attractive.
A further field of application could be envisaged in the context of schools in order to encourage lively discussions between children and teachers about the specified topic so that they have the opportunity to engage with different identities in class. This aspect will be accomplished within the cooperation project HardMut [9] between the University of Applied Sciences (HTW) Berlin [10] and the Jewish Museum Berlin [11] for the development of a multimediabased mobile museum.
Moreover, further development regarding the content within this research project is also planned. Currently, four reference images of the same gender, stored in a database, are displayed after recording a portrait picture. Consequently, users have the option to see themselves in another identity. In order to extend this aspect, four reference images of the other gender could be displayed in addition so that the final image can also represent the possible appearance of a potential child of the user and the person illustrated in the reference image.
In addition to the contextual advancement, it is also necessary to test the system within future steps in order to improve the application based on the given feedback of the participants. Therefore several pilot studies need to be performed and evaluated.
Furthermore, information gathered during the exhibition visit can be made available in a virtual way afterwards, e.g. by means of a private area on a website, in order to secure the sustainability of content. As a result of having this Web 2.0 platform, social interactions between users could be strengthened by means of the implementation of a discussion board.

ACKNOWLEDGEMENT
The research project HardMut is hosted by the University of Applied Sciences (HTW) Berlin in cooperation with the Jewish Museum Berlin and gratefully funded by the European regional development fund (ERDF) of the European Union.