CNN-based Visual Servoing for Simultaneous Positioning and Flattening of
Soft Fabric Parts
Abstract
This paper proposes CNN-based visual servoing for simultaneous
positioning and flattening of a soft fabric part placed on a table by a
dual manipulator system. We propose a network for multimodal data
processing of grayscale images captured by a camera and force/torque
applied to force sensors. The training dataset is collected by moving
the real manipulators, which enables the network to map the captured
images and force/torque to the manipulator’s motion in Cartesian space.
We apply structured lighting to emphasize the features of the surface of
the fabric part since the surface shape of the non-textured fabric part
is difficult to recognize by a single grayscale image. Through
experiments, we show that the fabric part with unseen wrinkles can be
positioned and flattened by the proposed visual servoing scheme.