Scaling depth from shadow offset

When an object casts a shadow on a background surface, both the offset of the shadow and the blur of its penumbra are potential cues to the distance between the object and the background. However, the shadow offset and blur are also affected by the direction and angular extent of the light source and these are often unknown. This means that the observer must make some assumptions about the illumination, the expected distribution of depth, or the relation between offset and depth in order to use shadows to make distance judgments. Here, we measure human judgments of perceived depth over a range of shadow offsets, blurs, and lighting directions to gain insight into this internal model. We find that distance judgments are relatively unaffected by blur or light direction, whereas the shadow offset has a strong and linear effect. The data are consistent with two models, a generic shadow-to-depth model and a Bayesian model.

of the relief r, we have that (3) Fig. S2 shows the log likelihood log ( | ). Note that for tight shadows, the likelihood is strongly peaked near zero relief, whereas for loose shadows, larger relief becomes more probable.  < l a t e x i t s h a 1 _ b a s e 6 4 = " y z H 4 S W U f b k z P g h R S g T 6 2 h R Z / X 6 g = " > A A A C N H i c b V D L S g M x F M 3 4 t r 6 q L t 0 E i 6 A I Z a a C u h G K b k Q 3 C l a F p p Z M m m l D M 5 m Q 3 B H K 2 I 9 y 4 4 e 4 E c G F I m 7 9 B t N O F 7 4 u J B z O g + S e U E t h w f e f v b H x i c m p 6 Z n Z w t z 8 w u J S c X n l 0 i a p Y b z G E p m Y 6 5 B a L o X i N R A g + b U 2 n M a h 5 F d h 9 2 i g X 9 1 y Y 0 W i L q C n e S O m b S U i w S g 4 q l k 8 1 U T y C D a J 7 o g 7 Q 4 x o d 2 A L H 2 A S G c q y i u l n R I v c Y m 4 q h C X W 3 c 6 7 T a x Q O R y F + o V m s e S X / e H g v y A Y g V J 1 5 2 T / 6 E C a s 2 b x k b Q S l s Z c A Z P U 2 n r g a 2 h k 1 I B g k v c L J L V c U 9 a l b V 5 3 U N G Y 2 0 Y 2 X L q P N x z T w l F i 3 F G A h + z 3 R E Z j a 3 t x 6 J w x h Y 7 9 r Q 3 I / 7 R 6 C t F + I x N K p 8 A V y x + K U o k h w Y M G c U s Y z k D 2 H K D M C P d X z D r U 1 Q W u 5 0 E J w e + V / 4 L L S j n Y L e + c B 6 X q I c p n B q 2 h d b S J A r S H q u g Y n a E a Y u g e P a F X 9 O Y 9 e C / e u / e R W 8 e 8 U W Y V / R j v 8 w u f T a 2 m < / l a t e x i t > p ( |r) = 2r ⇡(r 2 cos 2 +sin 2 ) < l a t e x i t s h a 1 _ b a s e 6 4 = " 6 T 9 b o V x v u a N 1 m e X Y 1 H F + 8 H e s U / Y = " > A A A C P X i c b V B L S y s x G M 2 o 9 1 r r 9 V p 1 6 S Z Y L l S E M l N B 3 Q j F b s S V g l W h q S W T Z t p g J g n J N 0 I Z + 8 f c + B / c u X P j Q h G 3 b k 0 f C x / 3 g 4 T D e Z B 8 J z Z S O A j D h 2 B m d u 7 X 7 / n C Q n H x z 9 L f 5 d L K 6 p n T m W W 8 y b T U 9 i K m j k u h e B M E S H 5 h L K d p L P l 5 f N U Y 6 e f X 3 D q h 1 S k M D G + n t K d E I h g F T 3 V K p 4 Z I n k D F 3 h D T F 8 S K X h 8 2 M T F W G 9 C Y J J a y v G a H O T F i a r y s E a a d v 7 1 / i z i h J n A a H Z q K 3 e y U y m E 1 H A / + C a I p K N e 3 j / Y a + 9 I e d 0 r 3 p K t Z l n I F T F L n W l F o o J 1 T C 4 J J P i y S z H F D 2 R X t 8 Z a H i q b c t f P x 9 k P 8 z z N d n G j r j w I 8 Z j 8 n c p o 6 N 0 h j 7 0 w p 9 N 1 3 b U T + T 2 t l k O y 1 c 6 F M B l y x y U N J J r E v Z l Q l 7 g r L G c i B B 5 R Z 4 f + K W Z / 6 x s A X X v Q l R N 9 X / g n O a t V o p 7 p 9 E p X r B 2 g y B b S O N l A F R W g X 1 d E h O k Z N x N A t e k T P 6 C W 4 C 5 6 C 1 + B t Y p 0 J p p k 1 9 G W C 9 w 8 M m b H s < / l a t e x i t > p (r| ) / 2r ⇡(r 2 cos 2 +sin 2 ) p(r) Figure S3. Log prior ( ) relief. These results indicate that a Bayesian observer should report low relief for tight shadows and increasing relief as the shadow offset increases. Fig. S6 shows the depth estimates predicted by this Bayesian model in comparison to the GLD model. While the Bayesian model fails to match the human data exactly, predicting a compressive scaling rather than the roughly linear scaling observed, it does roughly match the range of human judgements.
Note that the bias toward lower relief encoded in the empirical prior is critical for generating estimates in this range. Although the likelihood peaks for small reliefs r, it is far from symmetric: for small r, it is proportional to r, but for large r it is proportional to 1/r. Thus, while the peak occurs at relatively   small values of r, the distribution has a heavy positive tail. If a flat prior over r is employed, the MMSE model will generate relatively large estimates of r to avoid the heavy penalty that would otherwise be paid if the true relief turns out to be in that tail. The empirical prior prevents this by down-weighting the probability of those large reliefs.
We also considered the possibility of a non-uniform probability distribution over the slant of the light source. In particular, we modeled the light source slant as a two-parameter beta distribution transformed to [0, 90] degree, and identified the parameters that minimize the weighted squared deviation between the human depth judgements and the expected depth. (Weighting by the precision of the human judgements for each shadow offset corrects for the heterogeneity in the human data apparent in Fig. 6 of the main text). We evaluated a model that combines this light prior with a flat depth prior and a second model that combines the light prior with our empirical depth prior. Table S1 shows the resulting optimal estimates for the two parameters of the beta distribution, and the mean light source direction for the two models, and Fig. S7 shows the predictions of the resulting Bayesian model with flat and empirical depth priors. This exercise reveals that a broad but nonuniform prior over the light source can account for the human depth judgements just as well as the generic light source model.  To compare model fits to the human depth judgements we measured fit using the squared deviation, weighted by inverse variance, and performed a leave-one-out cross-validation across our 25 observers, which controls for overfitting and differences in model complexity. Fig. S8 shows the resulting RMS errors for the models considered here. We found that the GLD and Bayesian models with both depth and light priors did not differ significantly in their agreement with the human data (p = 0.47), but were both statistically superior to a maximum likelihood (ML) and a Bayesian model with empirical depth prior and flat light prior (p < .05).
Note that both the depth and light prior are found to be important. Assuming a flat depth prior and fitting a beta light prior results in a significantly poorer fit (p = 0.0047) to the human data and also an implausible light prior that is heavily biased toward extremely oblique light sources just grazing the surface (slants near 90 degree - Fig. S9). Incorporating the empirical depth prior leads to a more reasonable, roughly symmetric light prior peaking at 37.4 degree slant.

RMS error
Such a light prior reflects a bias away from light sources with very small or very large slants. Could there be an empirical basis for such a prior? Note that a small light source slant would entail a light source that was either in front of the observer's eyes, which would make it a visible occluder, or behind the head, which should cast a visible shadow of the head. Absence of these visual cues should therefore bias the observer away from this solution.
What about the bias away from very oblique light sources (large slants)? For the shadows cast below the square one possible explanation could relate to our experience with the sun. If the sun is very oblique to the view vector, its elevation must be close to 45 degree for it to cast a 45 degree shadow, and in Toronto, where the experiments were conducted, the sun reaches this elevation only between the months of March and September. Lower solar elevations are much more common and this should bias the observer away from solutions entailing very oblique light sources.
For indoor lighting, very oblique light sources would have to be in front of the observer to be below the ceiling or above the floor. In either case, the light source would often be visible to the observer were it present. Thus absence of a visible light source should bias the observer away from this solution.  Figure S9. Estimated priors for the illumination slant when assuming a flat depth prior (left) and when using our empirical depth prior (right).