Stimulus range bias leads to different settings when using luminance adjustment to evaluate discomfort due to glare

Luminance adjustment is widely used to evaluate discomfort due to glare. This paper reports an experiment conducted to investigate two factors of the luminance adjustment procedure, stimulus range bias and direct vs indirect control. Stimulus range bias describes the influence on subjective evaluations of the range of stimuli available to the test observer, with range being the minimum and maximum available glare source luminance in the current context. For the glare source, an artificial window, there were three ranges, having maximum luminances of 5 106, 7 288 and 9 469 cd/m 2 . The results suggest that luminance range had a significant effect on settings made, sufficient to change settings by an amount equivalent to one step of a Hopkinson-like discomfort sensation scale. The mean luminance associated with just intolerable discomfort with the low range was less than that associated with just uncomfortable with the high range. Past experiments have used direct control, where the observer makes the adjustment directly, and indirect control, where the observer instructs the experimenter to make the adjustment actions. Both methods were used in the current experiment. It was found that range bias was larger when using direct control than with indirect control. These findings contribute to an understanding of why different studies of discomfort glare have reported different results and hence proposed different discomfort models.


Introduction
Although it is generally accepted that well daylit conditions provide comfortable and healthy environments [1], too much daylight in the form of glare can be a problem [2,3]. One reason for this is we are, as yet, unable to confidently predict the degree of discomfort due to glare. This arises partly because past studies have given insufficient consideration to the experimental methodologies that were used. Changes in experimental design can significantly affect the ensuring glare thresholds as shown in recent work [4][5][6][7], resulting in different studies proposing different thresholds due to differences in the experimental procedure. To establish more robust design criteria for minimising the influence of daylight glare in buildings, further work needs to give more consideration to experimental design.
Luminance adjustment is a procedure for evaluating the subjective degree of discomfort due to glare. In this procedure, the brightness of the glare source (or the background visual scene) is varied to meet one or more predefined sensations of visual discomfort. Following early use by Hopkinson [8] many studies have used luminance adjustment to evaluate discomfort due to glare, e.g., [4][5][6][7][9][10][11][12][13][14][15][16].
Stimulus range bias describes the influence on subjective evaluations of the range of stimuli available to the test observer [17]. Range effects have been found to affect many sensory responses when using the adjustment procedure to meet a given subjective sensation, including preferred colour [18,19], preferred brightness levels [18,20], and perceived loudness [21,22]. Table 1 shows past studies that have used the luminance adjustment procedure with different stimulus ranges to evaluate discomfort due to glare. Common to all experimental procedures, observers were asked to set glare source luminances to represent each of four degrees of discomfort sensation ( Table 2). The four studies are presented by ascending order of the maximum luminance available: it can be seen that, for each discomfort sensation, higher settings were made when there was a higher maximum luminance available. This demonstrates that the available stimulus range influenced the settings made. Note for example that the luminance (126 cd/m 2 ) associated with disturbing glare in one study (Osterhaus and Bailey) is similar to the luminance (134 cd/m 2 ) associated with just perceptible glare in another study (Hopkinson and Bailey), a discomfort sensation two steps lower.
We do not suggest that stimulus range bias is the sole cause of the observed differences between these studies because there were other variations in experimental design, but we do propose that it may be a significant contribution. To better isolate the influence of stimulus range on luminance settings requires that an experiment is conducted using different stimulus ranges but without other purposeful change in the experimental design.  We explore also a second issue when using adjustment to vary the luminance of the visual scene. In some past studies, the observer was required to directly vary the luminance, such as by using a control dial e.g. [8,11,14]. With this approach, the observer has direct control over the variable stimulus and is free to adjust the variable stimulus in any manner they choose until they reach the final setting. In other studies this control is indirect, with the experimenter making the adjustments according to the vocal instructions from the test observer e.g. [10,15,16]. Following Glass et al. [23] we suggest two reasons why this may make a difference. First is the perception of personal control, the degree to which participants believe they have control over their environment as opposed to the degree of control they actually have. The perceived level of personal control over an environment (i.e., lighting, acoustics, temperature, air quality etc.) plays a large role on occupant performance, satisfaction and behaviour [24]. For example, when test observers had been assigned to their preferred lit condition and reported higher levels of perceived control, performed slower in comparison to a condition to which less control was given [25]. Given that control may influence the outcome of the result, this aspect should also be evaluated when adjustments are performed. Second, the observer may employ a different level of precision when giving instructions to a second person rather than having direct control. For example, they may accept an otherwise imperfect setting simply to reduce the need to further increase or decrease the brightness of the glare source.
An experiment was conducted to investigate two issues related to evaluation of the discomfort due to glare using a luminance adjustment procedure. First, whether different stimulus ranges lead to different settings. Specifically, it was hypothesised that increasing the maximum luminance available would increase the luminance set for a given discomfort sensation. Second, whether direct and indirect control over the variable stimulus influences the settings made. Specifically, whether this difference would affect the influence of stimulus range bias.

Experimental setting
The experiment was carried out in the SinBerBEST test room at the Berkeley Education Alliance for Research in Singapore (BEARS). The test room is of dimensions 4.25 x 5.5 x 3.6 m ( Figure 1) and has an artificial window, used as the glare source in the current experiment. The window is full height and near full width of one wall. It has overall dimensions of 3.63 x 3.6 m and is divided vertically into three panes. It is backlit by an array of light emitting diodes (LEDs) projected directly on to a membrane located behind the glass panes to promote an even diffusion of light across the surface of the window.
Surfaces in the test room had luminous reflectances of approximately: ρwall= 0.56, ρfloor= 0.72 and ρceiling= 0.72 (as estimated by matching to samples in the Munsell system [26]. An office-like workstation (i.e., chair, desk and laptop computer) was placed inside the room at a position that was parallel to and facing the artificial window. The desk top had an estimated surface reflectance of ρ= 0.56, dimensions of 1.80 x 0.75 m, and a height of 0.74 m from the test room floor.
The experimental arrangement of the apparatus in this investigation followed previous work using an artificial window and the luminance adjustment procedure to evaluate discomfort due to glare [7,9,13]. A flat screen 23" Hewlett Packard EliteDisplay E231 liquid crystal display computer screen (mean self-luminance of 196 cd/m 2 ) was placed in front of the participant. For those trials where adjustment was conducted by the experimenter (indirect control) the screen showed a simple visual target (a small circle [6]) to maintain fixation. For those trials where adjustment was conducted by the participant (direct control) the screen displayed the control commands (increase; decrease; pause). A chin rest mounted on the desk was used to hold participants' heads at a constant position, a height of 1.2 m from the floor, facing the computer screen. While a fixed viewing position is not essential for discomfort evaluations, and does not resemble the free viewing of typical natural situations, it enables precise characterisation of the size and location of the glare source which are key parameters of discomfort models.

Photometric measurements
The luminance of the artificial window was controlled using a Digital Multiplex (DMX) controller: the DMX was operated by one of two laptop computers, one used by the participant and the other used by the experimenter. The on-screen control adjustment had a linear relationship with average window luminance. This enabled the control settings made during trials to be later translated to window luminances.
The LED array allowed average window luminance to be varied from 441 to 9 469 cd/m 2 . Average window luminance is here defined as the mean luminance as measured at 27 locations across the whole window, this comprising a 3 x 3 grid of measurements on each of the three panes. The standard deviation from the 27 points varied from 41 cd/m 2 for the lowest average luminance (441 cd/m 2 ) to 709 cd/m 2 for the maximum average luminance (9 469 cd/m 2 ).
During trials, the room was lit only by the artificial window and the two laptop display screens. The luminances of surfaces in the participant's field of view surrounding the window were measured at 12 locations, equally spaced across the desk top, the walls, and the ceiling. The average of these 12 luminances ranged from 58 cd/m 2 (window set to lowest luminance) to 1 299 cd/m 2 (window set to maximum luminance). In parallel, the vertical illuminance facing the window at the viewing position of the test participant ranged from 434 to 9 253 lux. Across this range of window luminances, the correlated colour temperature (CCT) remained relatively constant at approximately 5 000 K. Light levels were measured using a Minolta LS-100 luminance meter (manufacturer's reported accuracy ±2% cd/m 2 ) and a Minolta CL-200a illuminance chromameter (accuracy ±2% lux).

Variation of stimulus range
This experiment used an adjustment procedure to set luminances associated with a specified degree of discomfort due to glare. An aim of the experiment was to investigate whether those settings are affected by the range of luminances available. Three ranges were used, herein labelled as low, middle and high (Table 3). In past studies, any basis for the choice of stimulus range is explained rarely, if at all. For the current purpose of demonstrating range bias, rather than establishing an absolute threshold then the luminance range is arbitrary and was chosen to fall within the ranges used in past studies ( Table 1). The minimum limit was constant for all three ranges (441 cd/m 2 ) and only the maximum limit was varied. This maximum limit was varied by the experimenter without informing the test participant.
Luminance adjustment during trials was achieved by clicking on-screen commands (increase, decrease, pause). The DMX control scale ranged from 10 to 250, with adjustment intervals of 4 units, each interval increasing the average luminance by approximately 140 cd/m 2 . An increase by a fixed luminance rather than a proportion of the overall range was employed because it meant that the increase was the same for all three ranges: adjustment by a fixed proportion would have resulted in a greater, and possibly more noticeable, variation in the high range than the lower ranges, and this may have influenced the settings made. One disadvantage of this approach was that there were fewer intervals on the lower ranges than on the higher ranges.

Degree of discomfort due to glare
During the experiment, participants were required to adjust the window luminance so that it represented a particular degree of visual discomfort. There were four levels of discomfort, here labelled A, B, C and D, with A representing a low degree of discomfort and D a high degree of discomfort. Figure 2 shows the definitions of these discomfort levels as shown to test participants.  The following definition of discomfort due to glare was given to all test participants at the start of the test session [27][28][29]: "Discomfort due to glare is a subjective sensation that is based on your visual impression of the conditions present inside the room. It is often caused when you focussing on a visual task (i.e., reading, writing or typing) and something in the background of your peripheral vision is excessively bright. Because it is excessively bright, the brightness or contrast in brightness it causes may cause mild discomfort or annoyance. The sensation should not be mistaken with conditions, whereby the source of brightness impairs your vision or starts to reduce your ability to see the visual task. The source of glare could originate from artificial lights or from reflective surfaces but in this experiment, it is the artificial window".

Test Participants
Forty-two participants were recruited for this experiment. Of these, 19 were female and 23 were male and the mean age was 27 years (SD= 6 years). Thirty wore their normal glasses or contact lenses, and all test participants self-certified as having no other eye problems. All test participants spoke fluent English, the language used for the test instructions. Test participants were paid for their participation to the experimental study. The UC Berkeley Committee for Protection of Human Subjects approved the research protocol (CPHS #2018-05-11072) and all participants signed an informed consent form before taking part to the study.

Procedure
At the beginning of the experiment test participants were seated at the desk and placed their head on the chin rest. The experimenters provided a set of instructions, including a definition of discomfort due to glare, the glare scale, and a description of how the experiment would be performed. Practise trials were conducted before the experiment commenced (see below).
Participants were instructed to set the window brightness to a level representing one of the four degrees of discomfort ( Figure 2). These four settings were made, in a random order, within one block of trials. At the start of each block the window luminance was set to the mid-point of the particular range (Table 3). Logadóttir et al. [18]  anchor point at the 50% point of the range, this produced similar results to an average of the settings that had made when the initial starting luminance commenced with high and low anchors. For the remaining settings made within each block, the initial luminance was the setting made on the immediately preceding trial.
To investigate range bias, there were three blocks of trials, with each block using a different stimulus range (Table 3). To investigate the difference between participants' direct adjustment of luminance and indirect adjustment (i.e. instructing the experimenter to make adjustments) these three blocks of trials were repeated. Considering both the method of adjustment (i.e., direct and indirect) and the three ranges of window luminance (i.e., low, middle and high) required six blocks, and these six blocks were fully randomised. Each block comprised four glare settings and thus each participant performed 24 trials. Within each block, the four glare settings were made in a randomised order. This was done to counter the order effect shown in previous work [4]. Each test session (all six blocks) lasted approximately one hour, which included a rest period of approximately 5 minutes before each block of four trials.
As recommended [17,30] a demonstration of luminance range was demonstrated before each of the six blocks of trials. The demonstration started with the luminance set to the 50% position for that range, it increased to the range maximum, decreased to the range minimum, and then returned to the 50% position. The maximum to minimum sequence was reversed for half of the demonstrations.
Test participants adjusted luminance by direct and indirect control. Direct means that they physically carried out the adjustment action, a mouse click on a screen command; indirect means they instructed (by voice) the experimenter who carried out the physical action. During indirect adjustment trials, participants were instructed to maintain their visual fixation towards a small circle (12 mm diameter, subtending an angle at the eye of 1.7 o ) displayed at the centre of the computer screen as used in previous work [6]. During direct adjustment trials, participants' visual fixation was directed onto the software interface displayed on the computer screen.
There were two practise trials before the six blocks commenced, these requiring the participant to adjust the luminance to discomfort setting B from the low initial setting and the low range (Table 3).

Statistical analyses
Parametric tests that relied on the assumptions of normality were used to analyse the data. Graphical (Quantile plots) and statistical (Shapiro-Wilks) tests were used to examine the normality of data distributions about the mean parameter. For comparisons that used repeated-measures analyses, tests were applied to determine whether the differences were normal about the mean parameter; alternatively, normality of each individual conditional group was tested when the independent variable was examined with between-subjects tests [31]. To test the assumption of sphericity, the Maulchly's test was used to determine whether https://doi.org/10.1016/j.buildenv.2018.12.061 https://escholarship.org/uc/item/0zp9j294 the variance of the differences between all comparisons made with the within-subject variable (i.e., the three ranges) were equal [32]. To test the assumption of homogeneity of variance, the Levene's test was used to test whether variances across independent groups were not statistically different [31].
Null hypothesis significance testing (NHST) was used to determine whether the differences in the average mean window luminances were significantly different across the three range conditions. Emphasis was placed on the effect size, defined as a measure of the magnitude of the differences examined [33], and not only on the statistical significance (which, in cases can confound in cases of small, large or uneven sample sizes) [34]. The effect size was estimated by making use of the effect size, omega squared (ω 2 ). The interpretation of the outcome was derived from the benchmarks proposed by Ferguson [35], in which values have been provided for small, moderate, and strong effect sizes (ω 2 ≥0.04, 0.25, 0.64 and r≥0.20, 0.50, 0.80, respectively). Values that were lower than the recommended minimum effect size (ω 2 <0.04 and r<0.20) do not represent a practically significant influence. Figure 3 shows the results of the experiment, with mean window luminances plotted for each combination of the four discomfort sensations and three luminance ranges. The first graph (a) shows these data for those trials with indirect control over luminance adjustment, and the second (b) those trials with direct control over adjustment. Both graphs exhibit similar trends: mean luminances increase for higher degrees of discomfort and with the higher stimulus range. The error bars show the standard deviations about the means.  Table 4 presents, for both direct and indirect control, the four discomfort sensations, the test statistic (Mauchly's W) and statistical significance (p-value) for the Mauchly's test of sphericity, the test statistic (F), statistical significance (p-value), and effect size (ω 2 ) for the repeated-measures analysis of variance (RM-ANOVA) applied when considering data from all trials. The assumption of sphericity was not met in one case (criterion D) for indirect control and two cases (discomfort sensations A and B) for direct control. Therefore, since the assumption of sphericity had been violated, Greenhouse-Geisser corrections were used to adjust the degrees of freedom (df) and calculate a conservative test statistic (F) protected against the Type I error [36]. The results of the RM-ANOVA considering all trials show that, the differences across the three ranges are all statistically significant for each reported discomfort sensation and under both indirect and direct control. Also, the differences are all of substantive magnitude as indicated by their corresponding effect sizes, ranging from strong (ω 2 ≥0.64 for direct control: sensation D), moderate (0.25≤ω 2 <0.64 for indirect control: sensations B, C and D and direct control: sensations A, B and C), and small (0.04≤ω 2 <0.25 for indirect control: sensation A). The effect sizes for each level of discomfort sensation are larger for direct control than indirect control, which suggests that the effect of range bias is larger. Under both indirect and direct control, the effect sizes increase when adjustments were made to higher sensations of discomfort due to glare. That is, the differences in the window luminance across the three different ranges under the same sensation of glare sensation are larger.

First block only
Following recommended practise, participants experienced the three different stimulus ranges (i.e. low, middle and high) in a randomised order. Therefore, when evaluating the second and subsequent blocks of trials they had already carried out evaluations with at least one of the three ranges. To counter this we reanalysed the data by considering results from only the first block of trials that participants had performed. While this may give a better representation of range bias, it is done at the cost of smaller sample sizes. Of the 42 test participants, 16 made their first glare settings with the low range, 14 with the middle range, and 12 with the high range, the inequality arising because block orders were randomised. Figure 4 presents results from only the first block of trials. Mean luminances tended to increase for higher degrees of discomfort sensation and with the higher stimulus range, similar to the results found when considering all trials (Figure 3). This trend is not apparent, for indirect control for discomfort sensations A and B, but this may be noise due to the small sample sizes available for the first block comparisons.  Table 5 presents, the test statistic (F) and degrees of freedom (df) and statistical significance (p-value) for the Levene's test, the test statistic (F), statistical significance (pvalue), and effect size (ω 2 ) for the Welch's ANOVA test. Since the results of the Levene's test show that the differences in variance across the independent variable are statistically significant in two cases (sensations C and D, indirect control) and in two cases (sensations B and D, direct control), this shows that the assumption of homogeneity of variance had not been met. Therefore, we used the Welch's (unequal variance) ANOVA to compare the means from multiple groups when the variances are not equal [37]. The results of the Welch's ANOVA show that the differences across the ranges are statistically significant in seven out of the eight cases that were examined, with an exception of indirect control discomfort sensation A, this showed that the differences were not statistically significant. The differences examined are mostly of substantive effect sizes ranging from, moderate (0.25≤ω 2 <0.64 for direct control: sensations B, C and D), small (0.04≤ω 2 <0.25 for indirect control: sensations B, C and D and direct control: sensation A), and negligible (ω 2 <0.04 for indirect control: sensation A). Similar to the findings in the previous analysis (i.e., all data), the effect sizes under the direct control condition were consistently larger than with indirect control for each of the four glare sensations. Also, the effect sizes appear to increase when considering a high level of glare sensation.

Discussion
Inferential analysis of the data confirmed that stimulus range had a statistically significant and practically relevant influence on the final settings made when using the luminance adjustment procedure to evaluate discomfort due to glare. Specifically, when the available range of luminances was larger, participants tended to set a higher luminance for the same discomfort sensation. The effect of the range bias appears to be larger when the adjustments were performed to higher levels of discomfort sensation and when test participants had direct control over the brightness of the artificial window. Although these data do not reveal which adjustment method (indirect or direct) gave more relevant glare settings, the data did reveal that the choice of control strategy does influence the settings evaluations made by participants and subsequent analysis. In this case, indirect control reduced the size of range bias, although it still exerts a practically significant effect on glare settings made to the same discomfort sensations. Table 6 compares the effects on discomfort evaluations of different aspects of experimental design. The comparisons are made here using the effect size. Effect sizes provide a standardised measure of the experimental effect and are comparable across experiments [31], and therefore enable a direct comparison of the influence in each parameter on the luminance adjustment procedure. In Table 6 effect sizes are reported using Pearson's r coefficient (rather than ω 2 as used in Table 4). For three of the four discomfort sensations (sensations 2, 3 and 4) the effect sizes are the largest when measuring the stimulus range effect with direct control, although the effect of anchoring is only of slightly lesser magnitude. The effects sizes of these two factors are affected by the parameters chosen when conducting those experiments; an alternative method of comparison would be to identify what stimulus ranges led to the same effect size as does anchoring (or other experimental design choice). Note also that direct control in the current study produced larger effect sizes than indirect control for each discomfort sensation. This suggests that the effect sizes of other methodology parameters (Table 6) may be conservative estimates since indirect control was used in each of those investigations. To illustrate practical implications, consider the luminances set using direct control. For discomfort sensation D, the highest degree of discomfort in Figure 3, the mean luminance set with the low stimulus range was 4 112 cd/m 2 , and this is smaller than the mean luminance (4 475 cd/m 2 ) set for a lower discomfort sensation (C in Figure 3) but with the high stimulus range. In other words, a change in stimulus range caused a change in luminance settings equal to one whole criterion step on the glare scale. This would, in fact, completely change the interpretation the experimenter would make depending on which of the three ranges had been selected to make the luminance settings. As shown in Table 1, there is a wide variation in the luminance ranges used in past studies.
These findings lead to important practical considerations. First, stimulus range bias provides an explanation as to why different glare studies have revealed different results. Therefore, there is a need to carefully review the experimental procedures used in the literature in order to bridge the inconsistencies commonly associated with discomfort glare studies. Second, given that stimulus range bias does matter, there is a need to consider how to overcome the problem when evaluating discomfort due to glare. Within the literature there are two proposals. (1) To analyse only the first series of settings made when using the first range, since range response effects are easy to remember and will influence subsequent evaluations made when using different ranges [17]; (2) Take the average of multiple ranges when evaluating the same sensation [38]. Nevertheless, if a different set of multiple ranges were used this would result in different settings made to the same subjective discomfort sensation. This may falsely give an impression that the resultant setting from multiple ranges gives a correct result. However, this would imply that when performing the luminance adjustment procedure, the correct response range needs to be selected (i.e., luminance values that observers are typically exposed to in buildings). If a limited range of responses are provided, any proposed glare model will provide unreliable predictions of the perceived degree of discomfort due to glare. An alternative proposal is to accept that adjustment reveals relative affects and should not be used to establish absolute thresholds. For practical application, these data show that existing models of discomfort and threshold criteria are likely to be erroneous by a significant margin, leading to either unexpected discomfort or to an overly-conservative daylighting design.
Although the range bias appears to present a substantive finding in this study, some limitations of the current work need to be considered before practical application can be proposed. One limitation is that the artificial window represented a glare source with no visual information (i.e., with no view to the outdoor environment): it has been shown that observers are more tolerant to discomfort due to glare when the glare source contains pleasant visual information [16,39,40]. A second limitation is the degree to which the glare source in this work resembled glare sources experimented in natural situations. The artificial window was large, uniform and of a high luminance. Daylit windows have a variety of sizes, luminances and luminance uniformities. Waters et al. [41] showed that observers perceive uniform and non-uniform glare sources differently. While these limitations may affect any attempt to establish an absolute threshold for discomfort they do not affect the current purpose of demonstrating an influence of range bias.

Conclusion
This paper describes an experiment investigating discomfort due to glare from an artificial window. 42 test participants were instructed set the luminance associated with a particular discomfort sensation using a luminance adjustment procedure. The results showed a substantive influence of the stimulus range: Compared with the high luminance range, the low and middle luminance ranges led to significantly lower window luminances for all four discomfort sensations. The results also show that this effect is larger when observers make the adjustment directly rather than indirectly, although it is not clear whether direct or indirect control gives more relevant settings. In further work using laboratory trials to explore discomfort due to glare the current results suggest indirect control may be preferable as it reduces the influence of range bias.
The luminance adjustment procedure has been used a fundamental reference for the development of current glare models including unified glare rating and visual comfort probability [42][43][44]. While the two original studies [11,14] used to derive these glare models both used the adjustment procedure, different formulae were proposed. One possible reason for this inconsistency is that they did not consider the influence of luminance range when conducting their work. These models therefore require validation using an experimental procedure which recognises the influence of stimulus range bias and other problems.