Mumford-Shah for Segmentation and Stereo

Abstract

We begin by presenting an active contour model which utilizes the Mumford-Shah energy functional for the purpose of piecewise smooth image segmentation. We then show how the use of simultaneous piecewise smooth image segmentation on a set of calibrated 2D images of a common 3D scene may be utilized for reconstructing the unknown shapes and radiances of scene objects. To do so, we must lift the the support of the unknown smooth functions in the traditional Mumford-Shah framework, which will now represent the unknown radiance of scene objects in this application, onto a manifold which will represent the unknown shape of scene objects. This constitutes a significant mathematical departure from the traditional Mumford-Shah model since the unknown functions now live directly on the unknown geometric surfaces rather than their surrounding ambient space. The intution, however, follows from the original model in that the estimates must closely match the observed data while maintaining a high degree of smoothness both in the estimated functions as well as the estimated geometry.