Integration of disparity and velocity information for haptic and perceptual judgments of object depth

Abstract

Do reach-to-grasp (prehension) movements require a metric representation of three-dimensional (3D) layouts and objects? We propose a model relying only on direct sensory information to account for the planning and execution of prehension movements in the absence of haptic feedback and when the hand is not visible. In the present investigation, we isolate relative motion and binocular disparity information from other depth cues and we study their efficacy for reach-to-grasp movements and visual judgments. We show that (i) the amplitude of the grasp increases when relative motion is added to binocular disparity information, even if depth from disparity information is already veridical, and (ii) similar distortions of derived depth are found for haptic tasks and perceptual judgments. With a quantitative test, we demonstrate that our results are consistent with the Intrinsic Constraint model and do not require 3D metric inferences (Domini, Caudek, & Tassinari, 2006). By contrast, the linear cue integration model (Landy, Maloney, Johnston, & Young, 1995) cannot explain the present results, even if the flatness cues are taken into account.

Introduction

It is commonly believed that visually guided behavior relies on a three-dimensional (3D) metric representation of the environment and the objects in it (Glover, 2004, Greenwald and Knill, 2009). It is also believed that this 3D depth map is found by reversing the physics of image generation to infer the outside world from sensory data (Helmholtz, 1867, Landy et al., 1995, Landy et al., in press, Poggio et al., 1985). The solution of the so-called “inverse-optics” problem by a biological system, however, is extremely difficult because of the underdetermination of the required information. Horizontal binocular disparities, for instance, are not sufficient to recover an object's depth unless the viewing distance is known (Mayhew & Longuet-Higgins, 1982; Fantoni, 2008). Similarly, optic flow is not sufficient to recover surface slant unless additional parameters are known (i.e., the angular displacement between the observer and the surface and the amount of surface rotation) — see Fantoni, Caudek, and Domini (2010). Moreover, even sufficient constraints provided by multiple cues do not guarantee unique percepts (Todd, 2004).

For these reasons, some researchers have questioned the assumption that visuomotor processes rely on metric representations of target distances. Instead, they have hypothesized that (1) the brain relies mainly on image measurements that specify 3D properties directly, without building an explicit metric representation of the environment, and (2) appropriate body–environment interactions emerge as a consequence of adaptive mechanisms, not as the solution of the “inverse-optics” problem (Braunstein, 1994, Domini and Caudek, 2003, Robert et al., 1997, Thaler and Goodale, 2010, Todd, 2004). In prehension movements aimed at reaching and grasping visual objects, for instance, the (haptic and/or visual) feedback resulting from the contact between the hand and the target provides an error signal for calibration that improves the accuracy of subsequent reaches (e.g., Mon-Williams & Bingham, 2007). Thus, visuomotor actions (such as prehension and pointing) may not require the recovery of the full 3D metric depth map, but instead be based on simpler mechanisms of conditional associative learning. If this is true, we should expect that perceptual metric judgments and motor actions in novel stimulus situations with no haptic feedback should be systematically distorted, which indeed has been found to be the case (e.g., Cuijpers, Brenner, & Smeets, 2008).

In the current investigation, we carried out a cue combination experiment in which human performance was measured in three stimulus conditions: with disparity-only information, motion-only information, or both (see also Tittle, Norman, Perotti, & Phillips, 1998). In different blocks of trials, participants either performed a grasping task or provided a perceptual judgment.

Two models of cue integration are considered here. In the first model, image measurements, diagnostic of 3D depth, but insufficient for metric reconstruction, are utilized (intrinsic constraints). The second model, instead, is based on the assumption that the brain uses metric structure (i.e., distance and direction) to represent locations (linear cue integration). In the next sections, we will describe the two models and show how it is possible to empirically validate their predictions by using the results of the present experiments.