Consumer Subjective Impressions in Virtual Reality Environments: The Role of the Visualization Technique in Product Evaluation

Abstract

The availability and affordability of consumer virtual reality (VR) devices have fueled their adoption during the product design process. High fidelity virtual prototypes can be created more quickly and are more cost-effective than using traditional methods, but certain product features are still difficult to evaluate, resulting in perceptual differences when a product is assessed using different visualization techniques. In this paper, we report two case studies in which a group of participants evaluated different designs of a product typology (i.e., a watering can) as presented in VR, VR with passive haptics (VRPH) and in a real setting (R) for the first case study, and VR and R for the second case study. The semantic differential technique was used for product evaluation, and an inferential statistical method using aligned rank transform (ART) proceedings was applied to determine perceptual differences between groups. Our results showed that product characteristics assessed by touch are the most susceptible to being affected by the environment, while the user background can have an effect in some product features.

1. Introduction

Different key factors can help to ensure the success of a product apart from aesthetics [1], such as a strong market orientation, a clear and compelling product concept, a well-designed development process, and its evaluation, especially during the early stages of the design process [2]. In this context, ensuring a precise perception of a product’s features is crucial to guarantee an accurate evaluation of it [3]. The increasing competitiveness of the market is causing traditional forms of interaction with the product to be replaced by digital means [4], and the way that products are presented is becoming increasingly important for the optimization of the design process. In this regard, it is crucial to understand our perception of virtual prototypes in order to interact efficiently with it [5], since nowadays, the presentation medium is also considered an important factor for the success of a product [6].

With the advancement of virtual reality (VR) devices, virtual prototyping (VP) is gaining popularity in different fields [7,8]. The availability and affordability of VR standalone head-mounted displays (HMD), e.g., Quest 2 or Pico 4, have fueled their adoption in product design and are becoming more prevalent among product designers [9,10]. The metaverse is changing how products are designed, as gaining feedback from potential users during the design process can also be more cost-effective when physical prototypes are not necessarily needed for concept validation (especially in the early stages of the product development process). For example, there are various VR applications that facilitate the design process when individuals are not physically present in the same space. For instance, ShapeXR incorporates features such as eye-tracking and facial tracking to enhance interaction and communication among collaborators [11], or Microsoft Maquette, a general purpose mock-up tool for spatial prototyping within virtual reality [12].

In this context, VP is becoming an affordable and versatile alternative to physical prototyping [13], as it involves less time and money than traditional methods.

Although VR was proposed as a new tool in engineering design many years ago [14], it has gained increasing popularity in recent years as a powerful tool for product development given the technological advances [10], especially during the early stages, when many design variations must be produced [13,15]. This technology allows designers to create immersive virtual environments (VE) to evaluate different design concepts through VP, which provides a more efficient, versatile and cost-effective way to assess new products [13]. Virtual prototypes are easy to modify and share [16], and designers can test different design options in real time without the need for physical prototypes. Concepts can be visualized and tested in a realistic and interactive way, which can lead to more accurate and effective product design as it is positively affected by the user’s confidence and accuracy in the assessment [17]. Additionally, VR can facilitate the evaluation of virtual products in a more natural and intuitive way, as users can interact with them in a similar way to how they would interact with real products [18]. However, some product features are difficult to evaluate [19], as VE may not provide the same level of realism as real-world settings, or they may not accurately reproduce the tactile feedback that users experience when interacting with physical products. This can affect how users perceive and evaluate the products in a VE [20,21].

The introduction of haptic feedback in the VE can be particularly useful in the product design process [16]. Passive haptics provide a sense of touch in VR by synchronizing physical objects to virtual assets [22], significantly reducing costs without the use of intrusive devices, just the user’s hands. This has been shown to increase the sense of presence, improve cognitive mapping of the environment, and improve training performance [23]. In this context, passive haptics can be less expensive and easier to implement than active haptics, as it does not require the use of physical devices [24], which makes it more comfortable. Additionally, the introduction, and recent improvement, of hand-tracking in VR standalone HMD provides a more natural, intuitive and positive virtual experience [25]. With hand-tracking, users can interact with virtual objects using natural hand gestures and movements, rather than using a controller or other external input device. This can help to make the VR experience feel more realistic and allow users to more easily navigate and interact with virtual content.

The use of VR in product design, especially with the incorporation of passive haptic feedback through hand-tracking, has the potential to revolutionize the way products are designed and evaluated; however, assessing products in a VE can be challenging for several reasons. It is assumed that our perceptual and emotional responses to a product that is perceived through a VE are similar to those elicited by the actual product, which is not necessarily the case [26,27], particularly when evaluating product features that rely heavily on our sense of touch [21,28,29]. Overall, it is important to consider the potential perceptual differences when evaluating products in a VE and to use a combination of different evaluation methods (such as physical testing, user testing, and simulations) to obtain a comprehensive understanding of the products being evaluated. Few studies in the literature have examined the impact of hand-tracking and passive haptics on product assessment, so it is necessary to shed some light on this issue.

Studying perceptual variations in VR for product assessment is important for improving the accuracy and effectiveness of product design and evaluation. Further research in this area can help designers and researchers to better understand how different design elements and visualizations impact people’s perceptions and evaluations of products in VR and identify opportunities for improvement and innovation in the field.

In this paper, we report the results of two case studies where a group of participants were asked to evaluate different designs of a watering can using different visualization techniques. Users were also asked to indicate their intended purchasing decision and to rate their level of confidence in their response.

2. Related Work

VR is currently portrayed as a powerful tool for design [30] and product presentation [31,32]. It enables interactions within a VE that are far more effective than those made possible by traditional approaches such as sketches, renderings, or real images of the product [33]. This allows for more natural and accurate evaluations, likely to the perception one would have when viewing the real product. Despite this potential, there are certain limitations, as it may not be able to provide the same level of realism as a real environment, potentially resulting in significant perceptual differences and distorting the opinions of viewers.

Söderman was one of the first authors to make advances in this field [34,35]. Despite the results (no perceptual differences were found between the visualization techniques used), subsequent studies have proven that the visual media can have a significant effect on the user’s subjective impressions, considering the limitations of previous studies [26].

With the emergence of new visualization methods, some authors decided to analyze how immersive VR environments (with the use of an HMD) impact the user’s subjective impressions. For example, Felip et al. discovered that those features of the product evaluated through touch were the most susceptible to being affected by the medium [29]. Generally, these studies presented a significant limitation: the evaluation of a single product, which makes it difficult to extrapolate their conclusions to an entire type of product [18]. In this context, Palacios-Ibáñez et al. carried out similar studies by increasing the number of alternatives [3,36]. The authors corroborate that product features such as size (Large–Small) or weight (Heavy–Light) are susceptible to being affected by the visualization technique, but characteristics associated with the aesthetics of the product can also be influenced by the medium depending on the type of product. In addition, they also noted that the form of evaluation was an important factor when analyzing perceptual differences between sets of objects, as a joint evaluation helps to minimize differences between media.

In this case, several studies have shown that feeling the touch of physical objects in VE can improve global immersion, knowledge about the spatial environment, and users’ sense of presence [23,37]. Passive haptics can help to ensure that those features related to touch are not severely affected during the evaluation of a VP [21]. However, the absence of a virtual model of the hands can complicate the evaluation of the product to some extent, due to the lack of a real-time correspondence between the subject’s real hands and the VE.

More recent studies have introduced the hand-tracking technique during the virtual experience [20] thanks to the development of low-cost autonomous HMDs, demonstrating that the introduction of passive haptics during evaluation in a VE can be an effective tool for the evaluation of virtual prototypes and that the user’s background could influence the evaluation of some product features. However, it would be necessary to observe if this is also true for other types of products.

In this research, a novel type of product in this field of study was selected. On the other hand, the hand-tracking technique was also introduced during the virtual experience to provide a more natural and intuitive experience, similar to that of a real environment.

3. Research Aim and Hypotheses

The purpose of our research was to examine the impact of the visualization technique on product evaluation to determine whether there are variations in users’ subjective impressions when a product is viewed in different settings. Furthermore, we also examined the effect of users’ backgrounds on product evaluation, as this area of study has not received sufficient attention in the product design field [20,38].

In this regard, we conducted two case studies with a specific purpose. The first case study (A1) aimed to investigate the impact of passive haptics in a VE on product evaluation using non-intrusive hand-tracking. A group of participants evaluated four designs of the same product typology through three mediums: a real setting (R), VR, and VR with passive haptics (VRPH). The second case study (A2) aimed to determine if a more advanced virtual experience (increased interaction) would influence product evaluation. In this case, a group of participants from different backgrounds evaluated three designs of the same product typology used in A1 through two visualization techniques: R and VR.

The following hypotheses were postulated: the visualization technique can impact the user’s subjective impressions of the product (H1); the visual media used to display a product can affect the purchase decision (H2); and that the confidence in the response can be affected by the visualization technique (H3). Additionally, for A1, we postulated that the product’s geometry can influence its perception together with the visualization technique (H4), while for A2, we also postulated that the user’s background can influence the evaluation of the product (H5).

4. Materials and Methods

The aim of this research was to examine the impact of physical stimuli during the presentation of a product on the perception of its characteristics and purchase intentions. To achieve this, the two case studies described below were designed with formal approval from the Deontological Commission ethics committee of the Universitat Jaume I (approval number CD/102/2021). For both cases, a range of watering can designs were selected. These designs were chosen due to the participants’ familiarity with the objects, as they are commonly found in domestic settings, and due to their evaluable properties, which can assist in determining the effect of presentation formats.

4.1. Case Study I

For this case study, four watering cans were selected. For the experiment, three visually similar rooms were prepared where the products to be evaluated were presented; in this case, four watering cans were selected from the Ikea catalog (Vattenkrasse, Chilifrukt, Bittergurka, Förenlig), the characteristics of which are described in

Table 1. Physical watering can characteristics used in Room 1 (modeled virtually for Rooms 2–3).

	Product Model
	A (Vattenkrasse)	B (Chilifrukt)	C (Bittergurka)	D (Förenlig)
Material	Metal	Glass	Metal, Wood	Metal
Capacity (L)	0.9	1.2	2	1.5
Weight (g)	240	385	480	660
Height (cm)	16	21	30.5	26
Base diameter (cm)	11.5	-	8.5	8
Base area (cm2)	103.82	126.82	56.72	50.24
Price (€)	15	12	12	9

.

These were presented in three separate rooms (Figure 1), each of which presented the watering cans in a different format: R, VR, and VRPH. In each medium, each participant similarly interacted with the four products, and then evaluated their characteristics, before doing so again in the other media:

1—Room 1 (R1): R. In this room, participants could touch the real surface of the physical watering cans, but they could not lift them off the table. To prevent this from happening, the base of each watering can was glued to a foam cardboard sheet using hot-melt adhesive, which was also fixed to the table. All the physical elements of the room, including colors, textures, distances, and lighting, were used as a reference for the virtual reality modeling of rooms 2 and 3;

2—Room 2 (R2): VR. In this room, a 3D model of the watering cans was used. All the elements of the virtual scene accurately reproduced those used in R1. Participants saw a virtual representation of their hands (avatar), with which they could approach the watering cans to simulate being able to touch them;

3—Room 3 (R3): VRPH. The same scene as in R2 was used, combined with the physical elements used in R1. In this way, the VR representation was enriched because participants could physically touch the watering cans, as they were located on the table in the same coordinates as their virtual counterparts. The scheme in

Table 2. Room features for Case Study I.

	R1 (R)	R2 (VR)	R3 (VRPH)
Visual stimulus	Light beige floor, light gray walls, light gray table, four watering cans
Physical stimulus	Four watering cans	None	Four watering cans
Actions: Touch	Yes	Yes (virtual)	Yes
Avatar	No (real hands)	Yes (virtual hands)	Yes (virtual hands)
HMD	None	Oculus Quest 2	Oculus Quest 2

details the characteristics of each room and the actions that can be performed in them.

4.2. Case Study II

In this case, the participants were required to observe, manipulate, and evaluate three watering-can models. These products were presented at equidistant from each other and placed on a table (160 × 80 × 100 cm) in front of a light gray wall. The order of placement of the products on the table was changed in each experimental condition to balance any possible effects that might appear.

To conduct the experiment, two visually similar rooms were prepared where the products to be evaluated, in this case, three watering cans from the Ikea catalog (Förenlig, Bittergurka, and PS 2002), were presented, the characteristics of which are described in

Table 3. Features of the physical watering cans used in room R1, modeled virtually for room R2.

		Product Model
	A (Förenlig)	B (Bittergurka)	C (PS 2002)
Material	Metal	Metal, Wood	Plastic
Capacity (L)	1.5	2	1.2
Weight (g)	660	480	90
Height (cm)	26	30.5	32
Base diameter (cm)	8	8.5	-
Base area (cm2)	50.24	56.72	54.24
Price (€)	9	12	1.5

.

The watering cans were presented in two separate rooms (Figure 2), each one presenting them in a different format: R and VR. In each medium, each participant interacted with the three products in a similar way before evaluating their characteristics, before doing so again in the other medium.

Table 4. Room features for Case Study II.

		R1 (R)	R2 (VR)
Visual stimulus		Light beige floor, light gray walls, very light gray table, three watering cans.
Physical stimulus		Three watering cans	None
Actions: Touch	Catch	Yes	Yes (virtual)
	Lifting	Yes	Yes (virtual)
	Move	Yes	Yes (virtual)
Avatar		No (real hands)	Yes (virtual hands)
HMD		None	Oculus Quest 2

shows the characteristics of each room and the actions that can be performed in them.

1—Room 1 (R1): R. In this room, participants could pick up the physical products by the handle, lift them up, and move them in space. All physical elements of the room, with their colors, textures, distances, and lighting, were used as a reference for the VR modeling of room R2;

2—Room 2 (R2): VR. In this room, a 3D model of the products was used. All elements of the virtual scene accurately reproduced those used in R1. Participants saw a virtual representation of their hands (avatar), with which they could pick up the watering cans by the handle, lift them up, and move them in space.

4.3. Product Evaluation

The evaluation of the watering cans was carried out using the best–worst scaling (BWS) method [39], specifically Case 1 (object case). This methodology, based on ranking models [40,41,42], requires participants to make choices between various options, determining which is the “best” and which is the “worst” in each case.

Applied to our study, this methodology consisted of comparing which of the different watering cans (best or worst) met each of the properties to be evaluated. This method has advantages over other forms of evaluation, such as the Likert scale and rating scales, because it is easy for humans to identify extremes in a comparison [43], allows for better discrimination between attributes, and the valuation emitted is easier to interpret than the values of a rating scale. In addition, it improves the consistency of responses and can be more easily understood by survey respondents [44].

To determine the properties to be evaluated, the following methodology was followed. First, a Google Forms survey was sent to 76 regular users of this product (48 women and 28 men, aged between 19 and 30, with an average age of 20.56, SD = 1.72) asking them to select the 5 properties that they considered most important when choosing a watering can from a list of 10 (formal aesthetics, weight, resistance, stability, style, capacity, comfort of grip and handling, ease of filling, versatility, irrigation dosage, or lifespan). They were also offered the possibility of writing down another adjective that they considered important and that was not on the list. Then, the eight properties chosen by the highest percentage of users were selected, which would be used to evaluate the product considering the most relevant aspects: grip comfort (92.1%), ease of filling (69.7%), weight (60.5%), capacity (57.9%), irrigation or pouring precision (51.3%), lifespan (43.4%), formal aesthetics (34.2%), and stability (15.8%). Finally, these properties were taken as a reference to define the two adjectives that would determine the “best” and “worst” options for each property. In this way, for example, for the “comfort” property, the adjective “comfortable” would correspond to the best option, while “uncomfortable” would identify the worst (

Table 5. Adjectives used to evaluate the most relevant properties of watering cans using the best–worst scaling method.

	Product Features	Adjective (Best)	Adjective (Worst)
P1	Grip comfort	Comfortable	Uncomfortable
P2	Ease of filling	Easy to fill	Difficult to fill
P3	Weight	Light	Heavy
P4	Filling capacity	Small	Big
P5	Irrigation precision	Accurate	Imprecise
P6	Shelf life	Long-lasting	Perishable
P7	Aesthetics	Beautiful	Ugly
P8	Stability	Stable	Unstable

).

4.4. Materials

The virtual room (R2) was modeled with Unity 2020.2.2f1, using the real scene as a reference, ensuring that the shapes, sizes, colors, textures, distances between elements, and lighting were similar in both (Figure 3). The R2 and R3 scenes were displayed using an Oculus Quest 2 HMD upgraded to version 36.0, a stand-alone immersive VR device with a single fast-switch LCD of 1832 × 1920 pixels per eye and a refresh rate of 72 Hz. This HMD can detect the position of the participants’ hands in real time to generate an avatar that helped them interact with the objects. The passthrough capability was enabled for the calibration of the virtual objects before starting the experiment, and the hand-tracking interaction capability was enabled to provide the non-intrusive virtual models of the user’s hands. Additionally, Oculus Integration SDK version 36.0 was used. The scene used realtime light with hard shadows enabled, and materials were built using a standard shader. The virtual objects were modeled in Autodesk Inventor 2023, and UV mapping was completed in Blender 2.93.0. In R2 for Case Study I, as there was no physical watering can, the hand avatar’s behavior was programmed to stop when it met the surface of the virtual watering can, preventing both volumes from intersecting, thus achieving a realistic visual sensation of collision between volumes equivalent to what occurred in room R1.

4.5. Sample

To estimate the minimum sample size, we performed an a priori power analysis with G*Power 3.1.9.7 [45]. To estimate the sample size for the experimental work, a Wilcoxon signed-rank test (matched pairs) with the following input parameters: effect size: 0.50, α = 0.05, (1-β) = 0.80, and two tails. A sample size of 35 was obtained.

Two case studies were conducted in this experiment, so the samples selected were different for each. For the first case study, different Industrial Design and Product Development Engineering students at the Universitat Jaume I participated in the study. In this case, a total of 53 volunteers took part in the experiment (15 men and 38 women) aged between 19 and 30 years old (mean = 20.70 years; SD = 1.76).

For the second case study, a total of 70 volunteers participated in the experiment, (50 men and 20 women), aged between 19 and 29 years old (mean = 21.67; SD = 2.13). Of these, 34 had an Industrial Design and Product Development Engineering background, while the remaining had an Industrial Engineering background.

4.6. Experimental Protocol

In both case studies, participants arrived at the test location in stages. Before their arrival, they were informed of the protocol to be followed and provided voluntary written consent to participate in the experiment, with formal approval from the Deontological Commission ethics committee of the Universitat Jaume I (approval number CD/102/2021).

The first phase of the experiment consisted of welcoming the participant (2 min) and directing them to proceed to the designated room (R1, R2, or R3 in the first study, or R1 or R2 in the second study as appropriate). The second phase took place within the room (3 min). The researcher noted in the survey the visualization technique used and prepared the order of placement of the watering cans on the table, from left to right. Depending on the case study and room, the participant may perform different actions, which are detailed below:

• First case study:

  •

  R1. The participant entered the room. They were explained that they would see a table with four designs of a watering can, which they should touch with their hands gently (not lift them).

  •

  R2 or R3. Before entering the room, the researcher helped the participant put on the VR headset and helped them enter the room. They were instructed to look and move their hands in front of the viewer to familiarize themselves with their avatar. Similar to R1, the researcher explained that they would see a table with four products, which they should touch with their hands gently and not lift them.

• Second case study:

  •

  R1. The participant entered the room. They were explained that they would see a table and on it there were three watering cans, which they should pick up by the handle, lift them and move them in space.

  •

  R2. Before entering the room, the researcher helped the participant put on the VR headset and helped them enter the room. They were instructed to look and move their hands in front of the viewer to familiarize themselves with their avatar. They were explained that they would see a virtual room with a table and on it there were three watering cans, which they should pick up by the handle, lift them and move them in space.

Once the visualization was ended, the researcher asked each question from the evaluation survey and recorded the participant’s answers. The user left the room once the survey was completed. At this point, the experiment was ended.

5. Results

5.1. Case Study I

First, three participants were discarded from the sample as they were identified as outliers, thus their responses could negatively impact the results. The final sample size considered for this case study was 50. Second, three datasets were obtained: (1) semantic scales; (2) overall evaluation; and (3) purchase decision.

Descriptive statistics for each dataset in our study are shown in

Table 6. Descriptive Statistics for the Semantic Scales.

		A			B			C			D
		VR	VRPH	R	VR	VRPH	R	VR	VRPH	R	VR	VRPH	R
Comfortable– Uncomfortable	M	−0.32	−0.30	−0.30	−0.54	−0.42	−0.52	0.62	0.50	0.66	0.24	0.22	0.16
	Mdn	0.00	0.00	0.00	−1.00	0.00	1.00	1.00	1.00	1.00	0.00	0.00	0.00
	SD	0.59	0.58	0.54	0.58	0.58	0.61	0.49	0.68	0.48	0.52	0.58	0.55
Easy to fill– Difficult to fill	M	−0.96	−0.94	−0.94	0.26	0.12	0.16	0.64	0.72	0.64	0.06	0.10	0.14
	Mdn	−1.00	−1.00	−1.00	0.00	0.00	0.00	1.00	1.00	1.00	0.00	0.00	0.00
	SD	0.20	0.24	0.24	0.49	0.39	0.47	0.53	0.50	0.53	0.24	0.36	0.35
Light– Heavy	M	0.10	0.16	0.30	0.50	0.28	0.08	−0.90	−0.78	−0.76	0.30	0.34	0.38
	Mdn	0.00	0.00	0.00	1.00	0.00	0.00	−1.00	−1.00	−1.00	0.00	0.00	0.00
	SD	0.42	0.55	0.51	0.58	0.64	0.67	0.30	0.47	0.43	0.51	0.52	0.57
Small– Big	M	1.00	0.98	1.00	0.00	−0.02	0.00	−0.86	−0.90	−0.82	−0.14	−0.06	−0.18
	Mdn	1.00	1.00	1.00	0.00	0.00	0.00	−1.00	−1.00	−1.00	0.00	0.00	0.00
	SD	0.00	0.14	0.00	0.00	0.14	0.00	0.35	0.30	0.39	0.35	0.31	0.39
Accurate– Imprecise	M	0.56	0.64	0.60	−0.90	−0.92	−0.86	−0.06	−0.06	−0.14	0.40	0.34	0.40
	Mdn	1.00	1.00	1.00	−1.00	−1.00	−1.00	0.00	0.00	0.00	0.00	0.00	0.00
	SD	0.54	0.49	0.50	0.30	0.27	0.35	0.31	0.31	0.35	0.50	0.48	0.50
Long-lasting– Perishable	M	−0.10	−0.08	0.00	−0.66	−0.66	−0.70	0.00	0.08	−0.02	0.76	0.66	0.72
	Mdn	0.00	0.00	0.00	−1.00	−1.00	−1.00	0.00	0.00	0.00	1.00	1.00	1.00
	SD	0.36	0.44	0.35	0.59	0.59	0.58	0.54	0.53	0.52	0.48	0.56	0.54
Beautiful– Ugly	M	−0.38	−0.40	−0.38	0.00	0.02	0.02	0.36	0.36	0.30	0.02	0.02	0.06
	Mdn	−0.50	−0.50	0.00	0.00	0.00	0.00	0.50	0.00	0.00	0.00	0.00	0.00
	SD	0.70	0.67	0.67	0.70	0.71	0.77	0.72	0.69	0.65	0.52	0.55	0.59
Stable– Unstable	M	0.22	0.22	0.42	0.06	0.18	0.34	−0.90	−0.88	−0.90	0.62	0.48	0.14
	Mdn	0.00	0.00	0.00	0.00	0.00	0.00	−1.00	−1.00	−1.00	1.00	0.50	0.00
	SD	0.42	0.47	0.50	0.51	0.52	0.48	0.30	0.39	0.36	0.49	0.54	0.54

Highest values and corresponding adjective are shown in italics, lowest values and corresponding adjective in bold.

and

Table 7. Descriptive Statistics for the Overall Evaluation and Purchase Decision.

		A			B			C			D
		VR	VRPH	R	VR	VRPH	R	VR	VRPH	R	VR	VRPH	R
Like	M	−0.48	−0.44	−0.38	−0.22	−0.20	−0.16	0.38	0.32	0.24	0.32	0.32	0.30
	Mdn	−0.00	−0.00	−0.50	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	SD	0.61	0.64	0.70	0.65	0.67	0.65	0.64	0.59	0.66	0.55	0.62	0.61
Purchase decision	M	0.06	0.08	0.12	0.06	0.04	0.04	0.32	0.30	0.24	0.34	0.36	0.34
	Mdn	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	SD	0.24	0.27	0.33	0.24	0.20	0.20	0.47	0.46	0.43	0.48	0.49	0.48

Highest values are shown in italics, lowest values are shown in bold.

. The descriptive statistics for the response confidence were M_VR = 0.96, Mdn_VR = 1.00, SD_VR = 0.20 for VR, M_VRPH = 0.98, Mdn_VRPH = 1.00, SD_VRPH = 0.16 for VRPH, and M_R = 0.97, Mdn_R = 1.00, SD_R = 0.17 for R. Stacked bar charts for this case study are shown in Figure 4.

On the other hand, an inferential statistical method was applied to test the hypotheses described, and a normality test was performed on each dataset to select the appropriate statistical test. As the sample size was >50, we used Kolmogórov–Smirnov’s normality test (significance level of 0.05). Results showed that the data did not follow a normal distribution, so parametric tests proved unsuitable. In this regard, we applied the aligned rank transform (ART) procedure [46], as it is known to provide a powerful and robust nonparametric alternative to traditional techniques [47]. It relies on a preprocessing step that “aligns” data before applying averaged ranks. After this step, common ANOVA procedures can be applied [48].

In our study, we performed a series of repeated measures ANOVAs for each of the semantic scales (

Table 8. One Factor ANOVA for the Semantic Scales and the Overall Evaluation.

			A		B		C		D
		df	F	Sig.	F	Sig.	F	Sig.	F	Sig.
Comfortable–Uncomfortable		2	0.081	p = 0.922	1.550	p = 0.218	1.290	p = 0.280	0.649	p = 0.525
Easy to fill–Difficult to fill			0.197	p = 0.822	2.370	p = 0.099	0.940	p = 0.394	1.370	p = 0.258
Light–Heavy			3.470	p = 0.035	7.780	p < 0.001	3.400	p = 0.038	0.600	p = 0.551
Small–Big			1.000	p = 0.372	1.000	p = 0.372	1.090	p = 339	2.330	p = 0.102
Accurate–Imprecise			0.859	p = 0.427	0.696	p = 0.501	1.450	p = 0.241	1.130	p = 0.328
Long-lasting–Perishable			1.940	p = 0.149	0.252	p = 0.777	1.050	p = 0.354	1.110	p = 0.335
Beautiful–Ugly			0.035	p = 0.966	0.045	p = 0.956	0.804	p = 0.450	0.282	p = 0.755
Stable–Unstable			5.750	p = 0.004	9.82	p < 0.001	0.073	p = 0.930	20.700	p < 0.001
Like			1.140	p = 0.325	0.270	p = 0.764	1.830	p = 0.166	0.062	p = 0.940

p-values showing perceptual differences (<0.05) are shown in bold.

) after the ART procedures to obtain detailed results, as well as post hoc tests (Tukey correction was applied) when perceptual differences were found between media to determine the exact groups involved (

Table 9. Post hoc for the Semantic Scales.

		A	B	C	D
Light–Heavy	VR–VRPH	p = 0.664	p = 0.122	p = 0.133
	VR–R	p = 0.006	p < 0.001	p = 0.047
	VRPH–R	p = 0.302	p = 0.15	p = 0.801
Stable–Unstable	VR–VRPH	p = 0.995	p = 0.158		p = 0.144
	VR–R	p = 0.029	p < 0.001		p < 0.001
	VRPH–R	p = 0.009	p = 0.031		p < 0.001

p-values showing perceptual differences (<0.05) are shown in bold.

). For the response confidence and the purchase decision, the Q of Cochran test was applied. No statistically significant differences were found between means for the response confidence (χ²(2) = 5.478, p < 0.065) nor the purchase decision (χ²(2)_A = 3.500, p < 0.174; χ²(2)_B = 1.000, p < 0.607; χ²(2)_C = 3.714, p < 0.156; χ²(2)_D = 0.200, p < 0.905).

Results showed that some semantic scales (Table 8) were affected by the visualization technique. However, the overall evaluation was not impacted by the medium. In this context, the weight (P3) and stability (P8) features were the most influenced by the representation method. Additional post hoc tests (outlined in Table 9) revealed that these differences were primarily observed between VR–R, with stability also showing differences between VRPH–R.

Different two-factor ANOVAs for the semantic scales that showed perceptual differences between the visualization techniques were performed to observe if there was a mixed effect between the medium used to display the product and its design. The results showed that (F(6) = 3.973, p_WEIGHT = <0.001; F(6) = 10.224, p_STABILITY = <0.001).

5.2. Case Study II

Three participants were discarded from the sample for similar reasons as the first case study, so the final sample size considered for this case study was 67. Again, three datasets were obtained: (1) semantic scales; (2) overall evaluation; and (3) purchase decision. Descriptive statistics for each dataset are shown in

Table 10. Descriptive Statistics for the Semantic Scales.

		A		B		C
		VR	R	VR	R	VR	R
Comfortable–Uncomfortable	M	−0.52	−0.55	0.12	0.33	0.40	0.22
	Mdn	−1.00	−1.00	0.00	0.00	1.00	0.00
	SD	0.70	0.76	0.79	0.64	0.68	0.76
Easy to fill–Difficult to fill	M	−0.73	0.01	−0.10	0.10	0.84	−0.12
	Mdn	−1.00	0.00	0.00	0.00	1.00	0.00
	SD	0.57	0.99	0.46	0.39	0.48	0.93
Light–Heavy	M	0.03	−0.61	−0.73	−0.28	0.70	0.90
	Mdn	0.00	−1.00	−1.00	0.00	1.00	1.00
	SD	0.65	0.52	0.54	0.60	0.52	0.35
Small–Big	M	0.97	0.87	−0.25	−0.30	−0.72	−0.57
	Mdn	1.00	1.00	0.00	0.00	−1.00	−1.00
	SD	0.17	0.46	0.50	0.60	0.45	0.53
Accurate–Imprecise	M	0.69	0.60	−0.55	−0.45	−0.13	−0.15
	Mdn	1.00	1.00	−1.00	−1.00	0.00	0.00
	SD	0.50	0.58	0.58	0.68	0.80	0.80
Long-lasting–Perishable	M	0.33	0.55	0.03	−0.07	−0.36	−0.48
	Mdn	0.00	1.00	0.00	0.00	−1.00	−1.00
	SD	0.75	0.58	0.80	0.74	0.77	0.77
Beautiful–Ugly	M	−0.22	−0.10	0.01	0.10	0.21	0.00
	Mdn	0.00	0.00	0.00	0.00	0.00	0.00
	SD	0.67	0.70	0.90	0.86	0.83	0.89
Stable–Unstable	M	0.90	0.97	−0.37	−0.30	−0.52	−0.67
	Mdn	1.00	1.00	0.00	0.00	−1.00	−1.00
	SD	0.39	0.17	0.55	0.52	0.59	0.47

Highest values for the adjectives in bold are shown in bold.

and

Table 11. Descriptive Statistics for the Overall Evaluation and Purchase Decision.

		A		B		C
		VR	R	VR	R	VR	R
Like	M	−0.27	−0.16	−0.15	0.06	0.42	0.09
	Mdn	0.00	0.00	0.00	0.00	1.00	0.00
	SD	0.73	0.75	0.78	0.83	00.78	0.85
Purchase decision	M	0.12	0.18	0.18	0.27	0.54	0.34
	Mdn	0.00	0.00	0.00	0.00	1.00	0.00
	SD	0.33	0.39	0.39	0.45	0.50	0.48

Highest values are shown in bold.

. The descriptive statistics for the response confidence were M_VR = 0.98, Mdn_VR = 1.00, SD_VR = 0.14 for VR, and M_R = 0.99, Mdn_R = 1.00, SD_R = 0.12 for R. The stacked bar charts for the semantic scales are shown in Figure 5.

Similarly, an inferential statistical method was applied to test the hypotheses described. The Kolmogórov–Smirnov normality test (significance level of 0.05) showed that the data did not follow a normal distribution, so parametric tests proved unsuitable again. In this regard, since classic nonparametric statistical tests only allow for the analysis of a single factor, we applied the ART procedure one more time (apart from being a powerful table and robust nonparametric alternative to traditional techniques). In our study, we performed a two-factor repeated measures ANOVA (

Table 12. Two-Factor ANOVA for the Semantic Scales, with Media as the First Factor and Background as the Second Factor.

		df	A		B		C
			F	Sig.	F	Sig.	F	Sig.
Comfortable–Uncomfortable	Media	2	0.040	p = 0.841	2.010	p = 0.161	3.530	p = 0.065
	Background		0.670	p = 0.416	0.243	p = 0.624	3.830	p = 0.055
	Mixed		16.100	p < 0.001	0.223	p = 0.638	11.384	p < 0.001
Easy to fill–Difficult to fill	Media		125.460	p < 0.001	14.500	p < 0.001	213.984	p < 0.001
	Background		223.000	p < 0.001	7.320	p = 0.009	86.600	p < 0.001
	Mixed		207.200	p < 0.001	1.990	p = 0.163	192.990	p < 0.001
Light–Heavy	Media		57.450	p < 0.001	53.183	p < 0.001	7.780	p = 0.007
	Background		53.700	p < 0.001	54.700	p < 0.001	3.100	p = 0.083
	Mixed		3.590	p = 0.063	18.400	p < 0.001	2.840	p = 0.097
Small–Big	Media		2.670	p = 0.107	0.454	p = 0.503	6.990	p = 0.010
	Background		2.600	p = 0.112	1.890	p = 0.174	6.380	p = 0.014
	Mixed		5.910	p = 0.018	0.042	p = 0.838	2.300	p = 0.134
Accurate–Imprecise	Media		1.450	p = 0.234	2.210	p = 0.142	0.003	p = 0.957
	Background		0.010	p = 0.921	1.130	p = 0.291	0.300	p = 0.586
	Mixed		0.520	p = 0.473	4.300	p = 0.042	1.910	p = 0.171
Long-lasting–Perishable	Media		2.630	p = 0.109	1.050	p = 0.310	1.540	p = 0.219
	Background		5.980	p = 0.017	4.080	p = 0.048	0.003	p = 0.954
	Mixed		0.210	p = 0.648	1.360	p = 0.248	1.540	p = 0.219
Beautiful–Ugly	Media		4.290	p = 0.042	2.640	p = 0.109	3.430	p = 0.068
	Background		0.020	p = 0.884	0.181	p = 0.672	0.151	p = 0.699
	Mixed		1.580	p = 0.213	0.003	p = 0.950	0.363	p = 0.549
Stable–Unstable	Media		0.270	p = 0.608	1.880	p = 0.175	4.130	p = 0.046
	Background		0.100	p = 0.753	0.073	p = 0.788	0.501	p = 0.481
	Mixed		0.210	p = 0.650	4.950	p = 0.029	3.660	p = 0.060

p-values showing perceptual differences (<0.05) are shown in bold.

) after the ART procedures. For the response confidence and the purchase decision, the Q of Cochran test was applied. No statistically significant differences were found between means for the response confidence (χ²(2) = 1.059, p = 0.303), while design C showed differences between means for the purchase decision (χ²(2)_A = 1.143, p < 0.285; χ²(2)_B = 3.600, p < 0.058; χ²(2)_C = 9.941, p = 0.002).

Our two-factor ANOVA results showed that, although different characteristics showed statistically significant differences from the influence of both factors, some of these can be highlighted. In this regard, “Easy to fill–Difficult to fill” and “Light–Heavy” were the most-affected features by the change of media (F1), as each of the designs selected was influenced by this factor. On the other hand, the filling capacity was also influenced by the user background (F2) for each of the watering cans. Additionally, design A and B were affected by the participants’ backgrounds for “Light–Heavy” and “Long-lasting–Perishable”. Finally, designs A and C showed a mixed effect between factors for filling capacity and grip comfort.

6. Discussion

In this research, two case studies were carried out to investigate the influence of the visualization technique used to display a product on the subjective impressions of the user. In addition, we studied whether differences in the evaluation of a product depended on the participant’s background (industrial design or industrial engineering). Here, a group of participants observed different watering can designs together and evaluated each of the products using the semantic differential technique. In addition, they made a purchase decision and gave an evaluation of their confidence in the response.

First, the authors postulated that the visualization technique could have an impact on the user’s subjective impressions of the product (H1). In this context, the results of the statistical analysis showed that some product features can be affected by the medium, which is in line with findings obtained by other authors [26,29].

In A1, the semantic scales “Light–Heavy” and “Stable–Unstable” were influenced by the visual display for three of the four watering can models (Table 8), while in A2, the two-factor ANOVA (Table 12) showed a combined effect between product design and the medium used to present it, which may explain why some designs did not show perceptual differences for those features, according to Palacios-Ibáñez et al. [3]. Additionally, in the case of weight, brightness may have affected size perception, as brighter objects often appear larger and closer to the user [49]. In this case, a larger product size may be confused with a larger product weight. Although attempts were made to maintain the same lighting conditions for the different experimental conditions, slight differences may have influenced the results. Descriptive statistics showed that the product was perceived as heavier in the VE, so the products may have appeared brighter due to the lights, or the materials used. Pairwise comparisons showed that differences were mostly found between the VR–R media, where interaction differences were also present (touch was not available in VR). Even if tactile feedback was available in the VRPH medium, the virtual media had the same visual conditions, so statistically significant differences between the virtual media should not necessarily be present. In the case of stability, these differences were also present between VRPH–R.

These characteristics are evaluated mainly through the sense of touch and sight. In the case of weight, the absence of touch could have hindered the evaluation of this feature, resulting in statistically significant differences between means [20,21]. In the second case, the visual differences between the VE and R could have introduced variability in the user’s response. Some authors made a classification of the product features according to Jordan’s pleasure categories [50]. Their results [36] showed that the physical pleasure category is the most affected by the medium, which is in line with our results.

The remaining features did not show statistically significant differences between mediums (Table 8), which can be attributed to the design of the virtual experience. In this sense, the ability to physically pick up and move the product to simulate its use may justify the differences between the visualization technique for the ease of filling observed in A2 (Table 12) but not in A1. On the other hand, technical limitations prevented the simulation of the full functionality of the product, therefore users were unable to fill them with water and use them realistically, which may explain the absence of differences in features such as comfort, or precision. In this regard, participants primarily relied on the product’s geometry, which was consistent across all mediums (which can explain the absence of a generalizable outcome regarding the existing differences in aesthetics or size). In future studies, it is advisable to make the virtual experience as similar as possible to the real one to ensure more robust results.

Similar results were obtained for A2. Although the overall evaluation showed no differences between the experimental conditions, the results of the statistical analysis (Table 12) showed that the adjectives “Easy to fill–Difficult to fill” and “Light–Heavy” were the most affected by the medium, as statistically significant differences were observed for each watering can design. Similarly, brightness may have also affected the results [39]. Furthermore, although several bipolar pairs of adjectives were influenced by the medium, not all categories were affected in the same manner, as confirmed by Galán et al. [28]. In this context, the results agree with previous studies in that adjectives that require sensory interaction, such as touch, are more sensitive to the change of display medium [21,24,28]. For this case, H1 was confirmed.

In the second hypothesis, the authors speculated that the visual media used to display a product could affect the purchase decision (H2). The only design that showed differences between means for this dataset was C in A2, thus H2 was not confirmed. There are two possible explanations for this: first, although access to information is critical when making a purchase decision [51], the medium used to present the information may not have been a determining factor in our case, since the selected media may have displayed the information necessary for the purchase intention to occur or not (which agrees with [52]); second, the selected product may not have elicited a positive purchase decision given the limited sample used (Table 7 and Table 11 showed low values for this dataset). Additionally, if we look at the descriptive statistics for this dataset in A1, we see that the highest scores tend to be found in the virtual media; therefore, showing this product using a virtual environment may have favored the purchase decision.

Volunteers also gave a rating on the response confidence during the evaluation. Although the descriptive statistics showed that participants felt more confident in their responses in the VRPH or R medium, and more insecure in VR, the statistical analysis showed no statistically significant differences in either of the two case studies (A1 and A2), thus H3 was rejected. The results of the descriptive statistics were not sufficient to conclude that the medium significantly affects confidence in the response. The introduction of tactile feedback during the experience explains this result, which appears to increase confidence in response but not to a significant extent compared to a fully synthetic environment.

For A1, we specifically questioned that the geometry of the product could influence its perception along with the visualization technique (H4). Many authors have examined the relationship between the shape of a product and the emotions it elicits since aesthetics is one of the first channels through which designers communicate with consumers [53,54]. The two-factor ANOVA showed that there is a combined factor between design and medium for two of the characteristics: weight and stability. Our results are consistent with those obtained by other authors, who suggest that geometric product features (i.e., product aesthetics) may influence users’ subjective impressions [55,56]. In this case, H4 was confirmed.

Finally, for A2, we also postulated that the users’ background may influence product evaluation (H5). The results of the statistical analysis (Table 11) showed that the ease of filling, weight, and duration of the product can be influenced by the subject’s background. In addition, for “Easy to fill–Difficult to fill”, there was a combined factor between medium and background. This can be explained by the training given to the different volunteers in the study, who can, for better or worse, intuit the behavior of the materials (related to the weight and durability of the object) or the usability of the product (related to the ease of filling). This is in line with the results obtained by [20]. For the case of the watering cans, H5 was confirmed.

Our results show that the visual medium used to present a product may significantly affect how the product is perceived. However, other factors, such as geometry, can also influence the user’s subjective impressions of a product. Therefore, not all products will yield the same response when the presentation medium is changed. For certain product features, technologies such as VRPH can be effective tools, but it is important to recognize how a particular medium relates to a specific product typology.

7. Conclusions

Our study demonstrates that the visual medium used to present a product can influence the user’s subjective impressions. Product evaluation is an essential task during the development of a new concept, so it is important to keep in mind that the use of a virtual prototype can, to some extent, distort the perception of some product features. In this context, the use of a virtual prototype may introduce some errors during product assessment. Some features are more difficult to evaluate using this prototyping technique, especially those that have to do with touch. In this regard, the use of passive haptics can help to minimize these differences, so VRPH can be an effective tool during the design process. We have shown that a VP cannot definitively replace physical prototypes, but both evaluation methods can complement each other to make the design process as fast and cost-effective as possible.

On the other hand, we also showed that an individual’s background can also influence product perception of certain product features. This shows that it is important that the evaluations produced during the development of a product are conducted with end users. In this way, the evaluation will be as accurate as possible.

The authors are aware of the limitations of the study. On the one hand, the age range of the participants was very narrow, which may have negatively influenced some of the results (such as the purchase decision). On the other hand, the small light differences between the experimental conditions could also have influenced the assessment of some semantic scales.

Our work contributes to the fields of product design and human–computer interaction by empirically assessing the reliability of VR and VRPH as tools for product evaluation in the early stages of product development, highlighting the importance of understanding how new technologies affect user perception in the design process. In future studies, we plan to use physiological measures, such as eye-tracking technologies, to analyze user behavior more accurately and objectively during product evaluation activities.