This paper proposes a new type of stereo matching algorithm called “Stereo Plane Matching”. Stereo plane matching adopts least square matching (LSM) method under the constraint that all points in an area specified by a polygon should lie on a same plane. This algorithm stands on the fact that correspondence between stereo images becomes 2-D projective transformation (homography) within an area where a common plane is projected. Three corresponding point pairs on stereo image pair are used for parameterization of geometry of a target plane, which allows computation of homography within the polygon. The most powerful feature of this algorithm is that it can impose geometrical constraint in planar direction and position. For example, the target plane can be fixed in horizontal or vertical direction, as well as in other specified direction in 3-D space. Another feature is that it can be applied to unrectified stereo pair, so far as its orientation parameters are known. Experimental results show that this algorithm can measure oblique roof or vertical wall of a building.