Keywords

1 Introduction

With the increasing maturity of virtual reality shooting and projection technology, more and more immersive video works appear one after another. But how to tell the story clearly has become a problem that immersive video creators have to solve. The purpose of this paper is to explore the possibility of enhancing the narrative ability of immersive images in the field of scene art design. From the perspective of art creators, this paper explores the main elements of scene art design, such as landscape design, landscape distribution, color design, light and shadow design, dynamic design. Authors found potential of visual guidance in these scene design elements. As long as they are carefully arranged and designed, scene art can be used to achieve effective visual guidance. Thus the narrative ability of immersive images can be strengthened.

2 Immersive Video Narration Requires Visual Guidance

Deficiency of shot and scene which are essential elements of traditional video, immersive video cannot convey what director wants to convey to audience. Moving cameras and changing scene are the most common way to use in narrating a story while they are now no more suitable in making immersive video. And as a result we could only search another way to remedy its shortcoming. An obvious characteristic of immersive video is 360° view which means audience has freedom to choose any spot and matter to see in it. However such kind of the freedom for audience can be a big problem for immersive video to tell a whole story cause audience may miss details the director try to transfer to them. On that account, effective visual guidance makes a great difference when creating immersive video. Only in attracting audience’s attention can they clearly comprehend the whole story.

On the audience side, visual guidance is also important. Without an immediate or secondhand visual guidance, audience may not catch the point of the immersive video and they may feel aimless when watching.

The basic rule to determine an effective visual guidance abides by the characteristic of human visual physiology. People may be easily attracted by some kinds of visual art scene, for example colorful matter, prominent proportion of shape or size, contrasty subjects and moving matter are easy to draw attention. After knowing the characteristic of human visual physiology, we can find out effective way to realize visual guidance and apply into creating.

3 Scene Art Design, the Best Way to Realize Visual Guidance in Immersive Video

Scene art design, the biggest visual area of all visual elements in immersive video, takes the responsibility to narrate the story of the visual part. And as a result, when we create the scene art, we should remind ourselves that its function includes beauty expression and visual guidance.

However, a gigantic problem in realize visual guidance is how to catch the audience attention. Cover the whole scene and remain the part in the immersive video you try to convey? No! In this way you will only destroy the artistry of immersive video and affect the audience’s watching feeling, and what is worse, make them disgusted. You can successfully catch audience’s attention and eyesight by scene art design under the basic rules that no destroying the artistry of the video and conveying the whole story of the video. So, to solve the problem in reasonable visual guidance is a problem demanding prompt solution in immersive video scene art design.

Scene art design includes various elements, such as spatial distribution, pattern factory, color, light. They all have the probability to realize the visual guidance in immersive video. In a narrow cave passage, we can easily be attracted to look at the front side or the back side of the passage. In an empty house, we will be sensitive to the doodle text or figure on the white wall. In a dark space capsule, we will absolutely interested in the sound made by laser sword. These simple samples show to us that if we can grab the feature of elements of scene art design, we can easily and successfully realize visual guidance. And in this way, narration of immersive video can also be fulfill.

How to realize effective visual guidance by scene art design in immersive video is the main topic we try to find out in my research. In a short word, narration language is a short slab in immersive video compered to tradition video and we try to use scene art design to be a sally port in attracting audience.

4 Study on the Elements of Scene Art Design in Immersive Video

4.1 Visual Contrast Is a Primary Condition to Achieve Visual Guidance

To achieve visual guidance is to achieve visual attraction, and that is to attract the audience’s vision. After understanding the selective characteristics of visual perception of human beings, it can be known that the “strong contrast” of scene art design can achieve an effective visual guidance effect of immersive image.

The object of visual guidance is human. And human’s understanding of visual object is through visual perception rather than vision. Vision only affects whether a person can see, while visual perception affects what a person can see. The human brain has an understanding of visual objects.

Not only vision, but also other sensory functions, cognitive experience, interests and hobbies will have an impact on people’s visual perception. Gregory, the visual psychologist said: “Visual perception of objects includes many sources of information. These sources of information go beyond the information our eyes receive when we look at an object. It usually includes knowledge of objects resulting from past experience. This experience is not limited to vision, but may include other senses. For example, touch, taste, smell, or temperature or pain.” [1] The famous biochemist Francis crick also said: “Look is a construction process. In this process, the brain responds positively to many different ‘features’ of the scene in parallel, and gives previous experience as a guidance to combine these features into a meaningful whole. Seeing involves some active process in the brain, which leads to a clear, multilevel symbolic interpretation of the scene.” [2] These two scholars have demonstrated that in the internal meaning of the observed thing, besides the meaning itself, it also has the subjective thought and memory of the observer. People’s subjective thought of the observed object is also the element that affects the visual guidance effect.

The formation of visual perception is selective, which is extremely important in immersive video. As the subject of observation and aesthetics, people have the right to independently choose the object of observation, which is an active exploratory movement. For example, when watching a figure, the observer can choose any part of it to observe carefully and pay even more attention in this part while ignore the remaining part of this figure. Such kind of active selective feature of human vision is called visual attention mechanism by Arnhem.

Visual attention mechanism, visual focus in other words, varies from person to person. The reason for diversification is that people have different interests, hobbies, behavior custom, physiology and psychological need. For example, animal lovers prefer to observe small animals in figures. People who are used to paying attention to the facial expressions of actors will pay attention to the performance of the facial expressions of the characters in the film. People who feel hungry will be attracted to the food in the video. All these difference are caused by the different characteristic of the observers.

Even though people may be attracted only by the things they are interested in, there still exist some rules that we can follow to catch their attention. For instance, people will surely find out a red flower among the green leaves, will instantly tell a crane among the chicken, and will easily find out a herd of cattle or sheep when the wind blows the grass low. These three examples are using the principle of color contrast, body comparison and dynamic-static comparison to grab people’s attention. And of course, these principles can be used in creating the scene art design of immersive video to catch audience’s attention to help the director telling the whole story.

In the animation Henry, an immersive video winning the 68th Emmy, the director using the way of strong contrast of light and color to tell his story, such as the lightening dog-shaped balloon, candles and colorful balloon fluttered in the sky in a dark room. All these elements try to work together to catch audience’s attention and achieve the function of visual guidance. In the immersive video and animation of Clash of Clans VR, projectiles, fireballs and flying arrows, which are full of dynamic and striking feelings, continuously streak across the audience’s eyes. The film tries to guide the audience’s perspective with such a strong contrast design, so as to achieve the purpose of supporting the film narrative (Fig. 1).

Fig. 1.
figure 1

Print screen of the immersive video of Henry and Clash of Clans VR

Only can we produce a clear contrast, we could catch audience’s attention immediately. We can generate a strong contrast by various methods, for example shape form, proportion size, location distribution, light and shadow, dynamic and static. In other words, to achieve successful visual guidance in immersive video, we need to make full use of the contrast in the various components of the scene art design.

Around the theme of contrast, we can conduct visual guidance exploration on five elements of the scene art design—distribution, form, color, light and shadow, dynamic and static.

4.2 Study on Visual Guidance on Distribution and Form

Distribution and form of the scene art design are the basic elements in immersive video. Location, density and the distance between every single matter in the immersive video will influence the way the audience pay in it.

Centralization Distribution Experiment

Take a blank sheet for example, if there are many balls of the same size and shape evenly distributed on the drawing paper, there will be no prominent places in the figure. But if all the balls are concentrated in one place in the figure, and the other places are left blank, then these concentrated spheres are prominent places. In this circumstance, when dealing with an entire empty scene in the immersive video, if a mess of the balls are put in one place, they will easily catch audience’s attention in no time.

According to this principle, I try to build up a VR scene by the three-dimensional software MAYA. In this scene, there are a square ground, physical skylight, virtual VR camera and a centrally placed sphere. The plane expansion effect of this immersive virtual scene is shown below (Fig. 2).

Fig. 2.
figure 2

The plane expansion effect and one sight in VR helmet of virtual scene in centralization distribution

As can be seen from the renderings of centralized distributed guidance, the most noticeable point of view is undoubtedly the front perspective, which is full of spheres. Eventually, the viewer will pay attention to the front perspective. That’s to say such way to grab audience’s attention works.

However there is an obvious disadvantage in this centralization distribution method which takes a long time for visual guidance, and viewers need to spend a certain amount of time and attention to find visual objects, which affects the viewing experience of the audience. And unclear instructions are the major taboo of film narration.

In general, the centralized distributed visual guidance method is feasible, but it has the disadvantages of unclear narration and long-time consumption.

Density Distribution Experiment

Density distribution experiment is an improvement on the previous experiment. The improvement method is to place the scene in all directions of the visual horizon, and these scenes will be concentrated in one direction from sparse to dense, forming a gradually dense visual effect.

In this experiment, we try to produce more visible objects in the immersive video to attract the audience’s attention into the object that director tries to show. The specific experimental results is showed as the following Fig. 3.

Fig. 3.
figure 3

The plane expansion effect and one sight in VR helmet of virtual scene in density distribution

It can be seen from the experimental renderings of the visual guidance of the density distribution that clear visual objects can be seen in the four horizontal directions of the front visual angle, the left visual angle, the right visual angle and the back visual angle. Among the four view, we will surely find out the objects in the front view are the most in quantity and prominent in effect while the objects in the back view are the less in quantity and weak in effect.

The trend the four view try to show to us is from minority to majority, from sparse to tight. As a result, audience will change their sight and follow this trend. Therefore, it can be seen that the visual guidance of density distribution is also feasible, and its visual guidance is stronger than the centralized one.

Streamline Guidance Experiment

The first two experiments take the location distribution of scenes as the main visual guidance method. Although they have certain visual guidance ability, they are still weak in the coherence of visual guidance, especially the centralized guidance. To improve this weakness we need to improve the consecutiveness of the object in the scene to better guide the viewer’s perspective.

The shape of the scene has a certain degree of visual coherence. When viewing the scene, audiences tend to comprehensively observe the entire shape of the scene, which can be applied to the visual guidance of immersive images.

This experiment creates a special strip shape that runs through all directions of horizontal vision and has different thickness and density distributions depending on the primary and secondary vision. The specific sculpt and effect is showed as following (Fig. 4):

Fig. 4.
figure 4

The plane expansion effect and one sight in VR helmet of virtual scene in streamline guidance

Evidently, we can catch the audience attention by sculpt and its’ variation. What is important, streamline guidance will be more effective compared to centralization guidance and density guidance.

For visual guidance, distribution and form of the objects in the immersive video can work together to reach the result of visual guidance.

Distribution and Form Experiment

There are many different way to achieve visual guidance by distribution of objects in the scene and form of the matter in the scene art design. The experiments mentioned above are typical case. The feasibility of these visual guidance methods can be further verified by creating cases.

In this distribution and form experiment, I am about to build up a cartoon town in an immersive video atmosphere using the unique distribution and form. As you can see, such imaginary cartoon town with various functional facility is suitable for human being to live in. The specific sculpt and effect is showed as following (Figs. 5 and 6):

Fig. 5.
figure 5

The plane expansion effect and one sight in VR helmet of virtual scene in distribution and form guidance

Fig. 6.
figure 6

Parts of the cartoon town

From the design sketch we can tell that the main cameras are surrounded by floated scenery and gigantic cirrus. The aerial town built on the basis of growing vines is in front of the camera. The overall spatial position relationship is clear, and the audience can easily understand the spatial relationship of this scene. The whole of the town is focused on the main vision, and the vines growing along the top of the cartoon town are constructed in a vertical way, so as to gradually guide the viewers’ eyes to the main characters.

The overall shape of the vine town is almost a cone, with the main structure being the main rhizomes in the middle and the small rhizomes spiraling around the main rhizomes. At different heights of the rhizome, a platform is spread out, gradually becoming smaller and smaller. On the platform are various buildings and furnishings of the town, where cartoon characters live. The overall shape of the vine town is to guide the viewer’s perspective upward. No matter the whole shape of the town which is big on the bottom and small on the top, or the roots of the vines which are getting thicker and thicker and spirally climbing to sky, they all indicate the audience to watch the top of this building. With the attention of the audience, the protagonist standing on the top of the main stem can start his performance and the film narrative can be carried out.

According to the content of the story, the design of the scene that limits the audience’s field of vision is also an effective way of visual guidance. Design sketch as following is another experiment I want to show (Fig. 7).

Fig. 7.
figure 7

The plane expansion effect in visual area restricted by hot air balloon

In this case, half visual are of the observer is restricted by hot air balloon. The top, bottom and back view are half occupied by the hot air balloon. Once the audience find it boring to watch the display in hot air balloon only, they will no doubt pay their attention to the scenery of the town. In this condition, suitable visual guidance is generated.

Making use of the reasonable scenery to restrict the flied of vision is more acceptable for human. On some extreme cases of immersive video, directors did some tough arrangements to control audience vision filed, for example, darken some part of the figure and lighten the middle part so that people will only watch the lighting part. Although such a technique can effectively limit the visual field and guide the vision, it will also seriously affect the audiences’ perception experience.

4.3 Study on Visual Guidance on Color, Light and Shadow

In addition to all the elements of scene art design in immersive video, color, light and shadow can be critical to guide audience’s visual line except for distribution and form of the scene. In the visual guidance method of color, light and shadow, we usually achieve a goal by different kind of contrast, such as hue contrast, saturation contrast, brightness contrast in color, form contrast, hard and soft shadow contrast, and photosynthetic efficiency comparison with special pattern with movement. Samples following are solely made to testify the visual guidance in color, light and shadow. And what’s important, I try to explore the usage of them in the immersive video to effectively realize attention attraction.

Visual Guidance in Color

Color has three nature, hue, saturability and brightness included. These three attributes determine the visual effect of color. If we want to make great use of color to achieve visual guidance, we need to make experiment from every single attribute of color.

Hue Contrast Guidance Experiment

Hue is the appearance of color. If we put various hue together, we will find various visual contrast from them. The degree of hue contrast will determine the visual impact power. That is to say, if we put different color of different hue together we can get different degree of visual impact power. And such visual impact power will be a great element to achieve form visual guidance in color.

Although the different hue collocation of color has countless kinds, the collocation between them is regular. We can distinguish the difference from hue into four degree of hue. The four degrees of hue are same hue contrast, similar hue contrast, contrasting hue contrast and complementary hue contrast.

Firstly, we can see that visual impact from the same hue contrast is the weakest. Same hue contrast means the diversity of hue is in minimum, and the color in the 24 color cycle is within 10°. Secondly, effect of similar hue contrast is a bit stronger than the former contrast. The difference within two hue is within 30°–90° in the 24 color cycle. Thirdly, effect of contrasting hue contrast is stronger than same hue contrast and similar hue contrast. And the hue are within 120°–180° in the 24 color cycle. For instance, red and blue are typical contrasting hue contrast. They have a very strong visual effect when putting together. Lastly, effect of complementary hue contrast is the strongest and its’ visual effect is the biggest. Complementary hue contrast means two color are beyond 180° in the 24 color cycle, for example red and green, yellow and purple, blue and orange are representative complementary color (Fig. 8).

Fig. 8.
figure 8

Color cycle (Color figure online)

According to the characteristic of visual impact of four hue contrast, we can apply it into the scene art design in the visual guidance of immersive video. For example, we can put the hue together from weak to strong on the basis of hue contrast so that we can create visual effect forming a visual impact gradually enhancing effect. The sketch effect of hue contrast in showed as followed (Fig. 9):

Fig. 9.
figure 9

The plane expansion effect and one sight in VR helmet of virtual scene in hue contrast guidance

In horizontal vision, the color contrast of the color belt is different, and the visual impact of the back perspective is the weakest, the left perspective and the right perspective are general, while the visual impact of the front perspective is the strongest, which is easier to get attention.

According to this experiment, we can design a guiding visual model in the aspect of color contrast and realize visual guidance by using the different characteristics of the contrast strength of color and hue.

Saturability Contrast Guidance Experiment

Saturability is a way to express the color brightness. The more colorful the object, the more it will attract people’s attention. And this is determined by the visual physiology of human being. Hence, color saturability contrast is suitable for visual guidance.

According to the characteristics that the higher the degree of color saturation, the stronger the visual attraction, the author carried out an experiment guided by saturability contrast. In this experiment, I put the graphic with the highest saturability at the end of the line that lead the audience’s visual line and put the graphic which getting weaker and weaker in saturability at the horizontal view angle to schedule their line angle. The sketch is showed as followed (Fig. 10).

Fig. 10.
figure 10

The plane expansion effect and one sight in VR helmet of virtual scene in saturability contrast guidance

According to the figure above, the post-viewing angle has the lowest image saturability and the most bland visual effect. Figures from the left and right angle have gradual change in saturability and have the effect of leading people’s sight line. Figure from the front angle has the brightest saturability and the saturability will stronger from both sides to middle.

Using the sliding scale of saturability will help to schedule the sight line of audience and can achieve visual guidance.

Lightness Contrast Guidance Experiment

Except for the hue and saturability in color can schedule audience’s sight line, the lightness contrast of color also has this function. Different degree of the color lightness contrast can produce different result on visual effect. For instance, in the dark tree hole of Star War 8, there is only one bright white line in this pitch-dark figure. As a result, the key object in the story stands out in this scene (Fig. 11).

Fig. 11.
figure 11

A pitch-dark figure from tree hole in Star War 8

Experiment of lightness contrast guidance is similar to the one of hue or saturability contrast guidance. Finally, we can get a sketch as followed (Fig. 12).

Fig. 12.
figure 12

The plane expansion effect and one sight in VR helmet of virtual scene in lightness contrast guidance

According to the sketch effect, we can see the biggest contrast in lightness is in the middle of front angle while the smallest contrast is in the middle of back angle. Front angle has the strongest visual effect. The results from lightness contrast are same with saturability contrast guidance experiment and they can also be applied in the visual guidance design.

Color Design Visual Guidance Comprehensive Experiment

Based on the visual guidance of color saturability, color filed ratio of optesthesia and scenery distribution, we can expand creation in immersive video scene. In this case, I build up a cartoon town. And I try to use a bright yellow car to lead people’s visual line.

In this cartoon town, there are succinct and clear urban architecture. I use roads to divide the city into disparate part and cars to indicate movement area. Of course, I also use color to emphasize objects. Effect are showed as followed (Fig. 13).

Fig. 13.
figure 13

The plane expansion effect and one sight in VR helmet of virtual scene in color design visual guidance comprehensive experiment 1 (Color figure online)

According to the renderings, except for one or two specific buildings, the buildings and the ground in the whole scene are displayed in low saturation colors. Only trees, hot air balloon, street lamp, waste bin and some cars are in high saturability. This design not only creates the artistic style of the film, but also highlights the subject and assists the narration of the film.

The second comprehensive experiment focuses on the visual guidance effect of color contrast and brightness contrast. I try to set up a high hue contrast visual guidance. Design sketch is showed as followed (Fig. 14).

Fig. 14.
figure 14

The plane expansion effect and one sight in VR helmet of virtual scene in color design visual guidance comprehensive experiment 2

According to the effect diagram, although the scene color saturation is high, it seems to be more average. But in fact these scenes in the color hue and brightness of the contrast are different, there are different visual impact between the scenery. The color collocation of scenery can play a certain role in visual attraction.

Analysis on Visual Guidance by Color

Currently, some immersive video has already applied visual guidance by scene art design in color. In the Dear Angelica, for instance, director highlighted hue, saturability and lightness contrast to achieve visual guidance. On this purpose, it helps to set off emotion and mood, and provides powerful narrative (Fig. 15).

Fig. 15.
figure 15

Visual guidance in color of Dear Angelica

Visual Guidance in Shadow Design

In a scene, where there is light, there is shadow, which with different degree of hard and soft, and intensity. So shadow is an ingredient in scene art design. Different shapes and intensity of shadow can produce different visual effects. In special situations, shadows can also serve as visual guides.

Take the simple shadow modeling visual guidance experiment as an example to show the visual guidance effect of the shadow. Design sketch is showed as followed (Fig. 16).

Fig. 16.
figure 16

The plane expansion effect and one sight in VR helmet of virtual scene in visual guidance in shadow design

According to the rendering, figure from back angle can only show the shadow of scenery. This shadow performs a long shape and extend to the other side. Audience will follow this long shadow and finally put their sight at the object from front angle. Extensibility of shadow is a key point in visual guidance of shadow.

In addition, hard or soft shadow margin can produce big contrast and has the potential to lead visual guidance.

Visual Guidance in Lighting Effect

Efficient lighting effect can play a role in setting off atmosphere, displaying artistic style, expressing environmental state or indirect narration. Lighting effect has a high degree on lightness contrast and color saturability contrast so it has a strong visual impact and attraction. It’s an ideal visual guidance element.

The indirect visual guiding effect can be achieved by using the luminous effect of different intensity of object. The experimental effect is shown below (Fig. 17).

Fig. 17.
figure 17

The plane expansion effect and one sight in VR helmet of virtual scene visual guidance in lighting effect

As can be seen in the figure above, the visual impact of objects varies with the intensity of their light emission. If the lighting effect get strong, visual impact will get strong too; vice versa. Especially in the pitch-dark environment, the lighting effect will be even stronger. Under this situation, visual guidance put into effect.

Another common application of special lighting effect visual guidance is image symbolization design. When combining light effect and symbolic shape, it can make the visual guidance more clear and direct. For instance, the arrowhead is illuminated in the dark scene, and the audience can instantly understand the inner meaning of the arrowhead. They will naturally look in the direction indicated by the arrow. Symbolic lighting effect visual guidance is realized.

In addition, Special light effects are not only static, but also can be displayed in a dynamic form in an immersive video. The visual guidance effect changes with the different level of intensity, speed and action of lighting effect changes.

Lighting in the scene design of We Wait is both the beginning of the film narrative, and the important undertaker of film visual guidance. The main light sources in this film are the searchlight of the Marine police and the light of the mobile phone screen of the refugees. Through these two light sources, the main characters are highlighted, and the visual guidance of the film is also realized through these two light sources. It can be said that the lighting design in the scene of We Wait is extremely ingenious, which is not only a necessary stage for narration, but also a tool for visual guidance (Fig. 18).

Fig. 18.
figure 18

Visual guidance of lighting effect in the scene of We Wait

And in Son of Jaguar, the lighting design skillfully combines narrative and visual guidance. An extension of the overhead light in the passage, the concentration of stage lights, character transformation lighting effect expansion have shown strong visual guidance and impact. It is a successful case of visual guidance of lighting and shadow design (Fig. 19).

Fig. 19.
figure 19

Visual guidance of lighting effect in the scene of Son of Jaguar

4.4 Study on Visual Guidance on Dynamic Scene

Visual Guidance Experiment of Scene Dynamic Design

In the scene design, the common dynamic scenes related to the plot are flames, running water, cars, wind and cloud, etc., which not only meet the needs of the plot, but also play the role of visual guidance. Cars and ferris wheels in the following creation experiments are good examples, as well as exaggerated rain and clouds in VR animation Rain or Shine (Fig. 20).

Fig. 20.
figure 20

The plane expansion effect and one sight in VR helmet of virtual scene visual guidance in scene dynamic design

According to the static figure of this case, three-dimensional arrowheads mark the movement of objects. The main moving objects in the scene are ferris wheel and cars on the road. Based on people’s visual preferences, dynamic things get more attention. It is easy to be carried away by a moving car or drawn to a spinning ferris wheel. The audience’s eye will eventually be directed to the main view of the frame.

Dynamic design visual guidance has a priority among objects. When there are multiple moving objects in the video at the same time, the object with more intense movement can get more attention. Of course, it is in the relative case.

Dynamic design visual guidance should be integrated. In coordination with color changes, luminous changes, form changes, and changes in the field of vision, it will make the guiding effect more efficient. The glowing gifts float across the full field of vision and slowly converge to the front view in The Dream Collector. This section is a typical visual guidance of integrated dynamic design (Fig. 21).

Fig. 21.
figure 21

Visual guidance experiment of dynamic luminescence design in The Dream Collector

In Rain of Shine, the clouds rain from the beginning to the end of the film has played an important role in visual guidance. The clouds always follow the leading role. The dark clouds won’t disappear until she takes off her sunglasses. Dark clouds and rain, as the key to the narrative of the film, not only show the fun, but also always attract the attention of the audience to the main character. We can say that the rain and clouds are the key to the narrative of Rain or Shine (Fig. 22).

Fig. 22.
figure 22

Visual guidance experiment of dynamic dark clouds and rain design in Rain or Shine

5 Conclusions

In the creation of immersive video, scene art design can achieve a certain visual guidance. This is an important supplement to the lack of immersive video narration

In the design of immersive video scenes, the distribution and form of the scene, color and shadow and dynamic effect design can form a certain visual guidance effect. Visual guidance design in art design needs to be integrated in many aspects. This paper systematically enumerates the possibility of implementing visual guidance in a small part of scene design to attract the attention of more creators so that they can further explore and study the art design and immersive video narration.