Stereoscopic Media Art That Changes Based on Gender Classification Using a Depth Sensor

Physical and psychological characteristics of people vary depending on gender and are used in various fields including the arts such as media art. An interface that provides different interaction results based on the gender of the users enhances participant satisfaction. For gender classification in a dark environment such as an exhibition hall, a depth sensor that discerned between the human head and the body and a support vector machine (SVM) that classified internal factors were used. In terms of the stereoscopic media art, factors that influenced the audience were set to be color, depth, and velocity in a certain direction, and a survey was conducted to examine the preferences of men and women. After gender classification, the preference factors of men and women were applied to produce interactive media art that showed different results depending on the interaction. The possibility of the interface based on gender classification was identified through the survey on preferences between a conventional interactive system and an interactive system based on gender classification.


Introduction
Every person has different physical and psychological characteristics.Physical characteristics including the body shape, voice, and walking pattern vary depending on gender and age [1].Emotional and psychological characteristics including concentration, memory, and visual and spatial recognition abilities also differ [2,3].Hence, demographic characteristics were studied for interface design.An interface in which gender characteristics are applied can be effectively used for human-computer interaction, biometrics, web design, learning, demographic collection, and target advertising [4][5][6].As the use of gender-based interaction technology has expanded into the field of arts such as media art beyond industrial fields, immersion and satisfaction of the audience are expected to increase.
Information obtained through interaction with the user is divided into passive information, such as age and gender, which is extracted from the user using an image and an audio sensor, and active information such as the character input and motion.By installing sensors that extract passive information not to be recognized by the user, natural interactions can be performed.While an active input method that requires sensor installation and recognition of instructions shows different competency levels and efficiency levels by user, it can obtain more accurate data than the passive sensor interface [7].Old people are familiar with the conventional interface such as a keyboard and a mouse whereas young people quickly adapt to the natural interface such as touch-based and motionbased interfaces.In particular, a spatial interface such as a motion-based interface can provide comfortable and efficient interactions considering human physical characteristics [8].When physical characteristics of the user combine with passive and active information input through the sensor, user-based interaction can be effectively designed.
For user-based interaction, various sensors and algorithms that automatically classify human characteristics are used.To distinguish gender, vision-based recognition that uses factors including ears, face, fingerprint, hands, and gait and audio-based recognition that uses factors such as voice and speech are used [4,9].Special factors such as DNA, palmar image, and facial and hand thermogram can be used for distinguishing people [10,11].However, the face and the body are frequently used for classifying gender and age in real time.SexNet showed a possibility of classifying gender by using the face of the user [12], and the recognition rate was considerably improved because of the development of algorithms such as the support vector machine (SVM) [13][14][15].Face recognition system improves the sensing coverage area by integrating into a wireless sensor network [16].While the installation location of the sensor for facial measurement is limited, the gait analysis method can classify gender without notifying the user by installing the camera on the side.For the image, a model-based method that analyzes the human structure and uses information on the skeleton depending on a gait cycle is used [17].Meanwhile, an appearance-based method uses the silhouette of the user who is walking in order to classify people by dividing the entire image into images of body parts and analyzing them or analyzing the pattern of the entire image [18,19].
Additional information for gender classification can be obtained by analyzing the image of the body shape.Information on the body shape, hairstyle, clothes, and accessories is used for classifying gender, leading to a decrease in errors [20].Gender classification can be performed by analyzing the face geometry of the 3D depth data [21].In particular, the wide supply of low-price depth cameras such as Microsoft Kinect is expected to be applied in various fields.Low-cost depth cameras exhibit a lower gender recognition rate than the image-based method because of low resolution.However, as the latter is influenced by the surrounding environment, its recognition rate decreases in dark environments such as an exhibition hall because of the considerable influence of lighting and exposure.A multimodal gender classification method that simultaneously uses image data and 3D depth data is also used [22].
Methods of outputting interaction results and emotions of the user vary depending on the purpose of the systems.The visual image and sound are representative output methods and provide more realistic sensations through the application of recent spatial technology.3D audio technology for sound and stereoscopic technology for visual effects are applied to provide spatial sensations and entertainment and promote active audience participation [23].The output interface also shows a different ability to recognize output data and preferences depending on characteristics of the user such as gender and age [24].
Studies on interface design considering the user characteristics beyond the simple interaction method are recently being conducted.The interface based on user characteristics in which the intuitive input interface and the artificial intelligence technology are applied increases user satisfaction [25].The output interface enhances work efficiency by displaying comfortable and effective images or satisfies the user by providing emotional images and sound.Convenience can increase by applying physical characteristics by age and gender in automobile design, or work efficiency can be improved by applying them in the workspace [1].Interface technology has been developed in the fields of education, gaming, and entertainment, and it is being expanded into the field of media art using sensor technology [26].

Human Factors in an Interactive Media Art
2.1.Previous Works.The application of characteristics of gender and age for interface design began in various fields, and the characteristics differ on the basis of the application field.Studies were conducted to identify the influence of user characteristics on interactive media art [26].
As shown in Figure 1, "Garden Party 1" and "Garden Party 2" are stereoscopic interactive media art that have the same themes and images.A touch-based interface was used in "Garden Party 1, " whereas a motion-based interface was used in "Garden Party 2. " The two artworks were installed in the media art exhibition, and a survey was conducted to investigate the preferences of the audience for the input interface and the motion and color of objects shown through stereoscopic images.
The survey was classified on the basis of age (20s, 30s, and 40s-50s) and gender (men: 18; women: 18), and emotions on the input interface, the depth change of the object in the image, and color were examined.In terms of the input interface, men prioritized the motion-based interface; old audiences prioritized the touch-based interface.As shown in Figure 2, preference for the interaction of the stereoscopic image depending on gender showed an insignificant difference.In general, the fast interaction (FI) object was preferred to the slow interaction (SI) and noninteraction (NI) objects, and the large depth (LD) change was preferred to the small depth (SD) change.Men showed a high preference for objects that had FI and LD characteristics, whereas women showed a higher preference for color change (CC) characteristics than men.According to the survey results, preferences by age and gender were not clearly defined.However, preferences were likely to vary depending on interactive factors including changes in the interface, color, and velocity in the media art.To design interactive media art that reacts to the gender of the audience on the basis of previous studies, the study on preference for color, depth, and velocity in a certain direction was conducted.

Adaptive Color.
Color is the factor that has a considerable visual influence on people, and the preference for it differs depending on age and gender.The comfort level varies on the basis of the lighting color, and it affects the ongoing tasks [2].Preferences for color by gender are also used for designing a virtual environment system, and a study on the preferences of both heterosexual and bisexual people was also conducted [27][28][29].Color preference is influenced by the ethnicity and culture of the experimental subjects, and the result varies depending on the experimental conditions.As the background color and the atmosphere of artworks could change the survey result, the survey on preference by gender was conducted by adjusting the color of the artwork to be produced.As depicted in Figure 3, seven colors, namely, green, red, yellow, orange, purple, pink, and blue that both male and female subjects showed different preferences for,       were selected, and the survey was conducted by adjusting the color of the artwork.The survey targeted art students (52 males and 51 females).As shown in Figure 4, men preferred colors in the order of green (1.92%), red (26.92%), yellow (9.62%), orange (3.85%), purple (15.38%), pink (17.31%), and blue (25.0%), while women preferred colors in the order of green (3.92%), red (35.29%), yellow (13.73%), orange (11.76%), purple (11.77%), pink (15.69%), and blue (7.84%).
As a result of analyzing the color preference by gender, we concluded that red generally received the highest preference whereas green had the lowest preference.Women showed higher preference for warm colors such as red, orange, and yellow than men.On the other hand, men were likely to prioritize cool colors such as blue, purple, and pink.

Adaptive Depth Velocity.
Every person has different space perception abilities as well as abilities to recognize stereoscopic images.Accordingly, the study was conducted to analyze the difference by age and gender in the workspace by using stereoscopic images.The abilities differ by age and person in a medical analysis of stereoscopic images, while the difference is not clearly identified in a statistical analysis [30].Men show higher performance on spatial model recognition and stereoscopic images than women, but a clear reason for this has not been found thus far [31].Preference by gender tends to differ on the basis of the stereoscopic image equipment [32].As effects by gender are different depending on the environment using the stereoscopic image, the application field, and tendency of the audience, the preference for stereoscopic motion in the media art work was investigated on the basis of previous studies [33].The three factors to apply adaptive depth velocity that vary depending on gender are shown in Table 1.
As hands in the image move on the basis of the motion of human hands while blowing particles from hands to the flowers inside, an interaction occurs and the depth value significantly changes.When day becomes night, the ground and the background are divided into small meshes that are blown to the audience and then return from the opposite   direction.Petals in fireworks have the complex movement of spirally advancing to the audience.The graph shown in Figure 5 shows the survey result depending on the velocity change of three objects.A higher velocity was prioritized for the hand particle that had a fast interactive factor and a high depth change.The ground and the background that had simple movements did not show a significant difference by gender.Men showed a slightly higher preference for hand particles than women, whereas women showed a higher preference for "petals in fireworks" that exhibited complex and beautiful images without interaction than men.In general, a considerable number of the audiences desired a higher velocity of three objects; the weights of the velocity were set as shown in Table 2.
For the artwork used for the survey, particle velocity was defined as   , mesh velocity of the ground and the  (1)

Gender Classification
Methods that classify gender using the human shape are divided into methods using the image input through the camera and methods using the depth camera such as Kinect.
The human shape can be detected using the image, and color information including hair color and makeup can be used.Human characteristics can be detected using the geometric location of the depth data.While the depth sensor has stable data values beyond external influences such as light, its resolution of 640 × 480 pixels is lower than that of the image sensor.In the general environment, gender classification using images exhibits a higher recognition rate than that using the depth value.To increase the immersion level, dark lighting is maintained in environments such as the media exhibition.As the amount of light is small, it is unlikely to obtain an accurate image by using an image sensor such as a web camera, and it is also difficult to classify gender.The use of the depth value can maintain the same shape data even in a dark environment.
To compare the gender recognition rate between images of the webcam and Kinect in a dark environment, the Biwi Kinect head pose database was used as shown in Figure 6 [34].The dataset consists of poses to watching the various directions.To confirm the gender recognition rate using the image in the dark environment, images with six stages of different brightness (B1∼B6) were created by adjusting the brightness of the images of the dataset as indicated in Figure 7.
For image-based gender classification, shown in Figure 8, face detection was performed using a Haar-like feature in the dataset image [35].The detected face image was normalized to be 32 × 32 pixels.After inputting the face dataset in the classifier SVM by using a radial basis function (RBF) kernel, gender classification was performed.
Further, the SVM was used for gender classification using the 3D depth value.As shown in Figure 9(a), the input data have the combined image of the person and the background.By measuring the distance between the Kinect sensor and the human, we can separate the person from the background in the depth map.After the background and the person are separated, noise from the human shape is removed and  based on feature points is used to efficiently analyze the surface of face.The characteristics such as nose tip, glabella, forehead, eyes, and lips are classified, and the regions of each characteristic are divided as shown in Figure 10.The calculated volume and surface area of the divided region are used as the classifiers.
Men and women can be also classified on the basis of the characteristics of the body.Men have broader shoulders than women, whereas women have more prominent breasts than men.As shown in Figure 11, as the distribution of the height and curvature of breasts varies, the average curvature was used as the classifier.
As such, classification was performed using two classifiers and the SVM.The gender recognition rate using the random images which are watching various directions was applied in dataset images having six-stage brightness, and the same depth value of the gender recognition algorithm using the depth value was used.Figure 12 shows that the recognition rate of the gender classification using image data decreases as the environment becomes darker and the gender recognition rate using depth data was not influenced by environmental changes.The recognition rate was lower when the depth data were used than when the image method was used in a bright environment.However, recognition rate of the depth method was higher than using the image method in a dark environment.

Design of Adaptive Interactive Art
The produced adaptive interactive art is designed through two interactive levels.In the case of the first physical interactive method, flowers were painted in response to the movement of the audience in the stereoscopic image.
The second method provides internal interaction that is not notified to the audience by recognizing the gender of the user and reflecting the gender characteristics to the audience.As indicated in Figure 13, one Kinect sensor was used as the input sensor to recognize the movement of the user and classify gender, while a 3D projector was used as the display equipment to provide the stereoscopic images.
As shown in Figure 14, gender classification is performed on the basis of the 3D depth by using the depth data input through the Kinect sensor.The gender result is transferred to a color generator and a velocity generator and changes the data based on the artwork process.Images of the artwork are shown to the audience through a stereoscopic display while changing their color and velocity.
Until the system classifies the gender of the audience, the color generator provides interactions with a variety of mixed colors as shown in Figure 15(a).When gender classification is completed, flowers are interactively painted using colors according to gender.Red, which received the highest preference, was selected as the common adjusted color.Accordingly, as shown in Figure 15(b), the ratio of orange and yellow flowers increased in the case of women.Meanwhile, as displayed in Figure 15(c), the ratio of blue, pink, and purple flowers increased in the case of men.
As shown in Figure 16, flowers are created according to the average rate of colors of green (2.92%), red (31.11%), yellow (11.68%), orange (7.81%), purple (13.58%), pink (16.5%), and blue (16.42%), in a section in which gender is not recognized.After the gender is classified at  2 by using the color generator, the ratio of preference by gender is created up to  3 and maintained to  4 by adjusting the color of the created flowers in a changing section.
The velocity generator determined  new  ,  new  , and  new  , the velocity values appropriate for men and women in terms of three objects.When the gender of the audience is not classified, the average value is maintained.When gender is classified at  2 , as shown in Figure 17, the velocity becomes linear and the velocity by gender is maintained from  3 to  4 .
A survey was conducted after installing the previously displayed interactive system and the system that simultaneously changed the color and the velocity depending on the gender.According to the survey result shown in Figure 18, interactive media art that changed color and velocity on the basis of gender generally received a high preference.Accordingly, women showed higher preference than men.

Conclusion
The characteristics of the audience towards the artwork in various exhibitions were found to vary.In media art including  technical factors, interface design is crucial.On the basis of a study using stereoscopic image interactive art, a study on media art that changed by gender was conducted as the audience showed different preferences for changes in color and the depth value depending on age and gender.A sensor was used for detecting an object, and technology that identifies the age and gender of a person was developed on the basis of the image.The use of 3D depth image was limited because of the size and cost of the equipment.However, the application of recognition technology using the depth data increased with the sales of low-cost depth cameras.The depth shape of the person was divided into the face and the body, and gender classification was performed by adjusting the classifiers of the SVM.In the general environment, the gender classification method using the image exhibited a higher recognition rate than that using the depth data.Meanwhile, in a special environment such as a dark exhibition hall, the method using the depth data exhibited a higher recognition rate.
The application of interaction technology that provided specialized information for the user by classifying gender and revealed statistical characteristics was expanded into the field of media art.First, a survey on the color, depth, direction, and movement of the stereoscopic image was conducted.
On the basis of the survey result, the color and the object velocity by gender were redetermined, and a system that applied the color and velocity changes on the basis of the gender classification was designed.A survey on interactive artwork that only reacted to human movement and interactive artwork that showed different results in response to the movement depending on gender was conducted.As a result, the system that showed different results according to gender was prioritized.Further studies will be conducted to distinguish age using the depth data and increase the accuracy of age and gender identification by using voice data, such as the voice of the audience.Moreover, studies on a gender-based interface will be continuously conducted by applying a gender classification technology in various interactive systems for stereoscopic media art, games, and education.

Figure 2 :
Figure 2: Survey of input and output interface.

Figure 3 :
Figure 3: Color samples for a new artwork.

Figure 4 :
Figure 4: Color preference for media art.

Figure 5 :
Figure 5: Depth velocity preference of media art.

Figure 9 :Figure 10 :
Figure 9: Face and body detection with depth data.

Figure 11 :Figure 12 :
Figure 11: Curvature and breast height visualization of upper body.
(a) A common artwork with various colors (b) An artwork for men (c) An artwork for women

Table 1 :
Characteristics of depth elements.

Table 2 :
Weights of velocity.