Sudipta N. Sinha
Richard Szeliski
Marc Pollefeys
Abstract
We present an interactive system for generating photorealistic, textured, piecewise-planar 3D models of architectural structures and urban scenes from unordered sets of photographs. To reconstruct 3D geometry in our system, the user draws outlines overlaid on 2D photographs. The 3D structure is then automatically computed by combining the 2D interaction with the multi-view geometric information recovered by performing structure from motion analysis on the input photographs. We utilize vanishing point constraints at multiple stages during the reconstruction, which is particularly useful for architectural scenes where parallel lines are abundant. Our approach enables us to accurately model polygonal faces from 2D interactions in a single image. Our system also supports useful operations such as edge snapping and extrusions.
Seamless texture maps are automatically generated by combining multiple input photographs using graph cut optimization and Poisson blending. The user can add brush strokes as hints during the texture generation stage to remove artifacts caused by unmodeled geometric structures. We build models for a variety of architectural scenes from collections of up to about a hundred photographs.
Supplemental Material
- Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. In ACM Trans. on Graphics (SIGGRAPH'04), 294--302. Google Scholar
- Aliaga, D. G., Rosen, P. A., and Bekins, D. R. 2007. Style grammars for interactive visualization of architecture. IEEE Trans. on Visualization and Computer Graphics 13, 4, 786--797. Google ScholarDigital Library
- Baillard, C., and Zisserman, A. 2001. Automatic reconstruction of piecewise planar models from multiple views. In CVPR, 559--565.Google Scholar
- Brown, M., and Lowe, D. G. 2005. Unsupervised 3d object recognition and reconstruction in unordered datasets. In 3DIM '05, IEEE Computer Society, Washington, DC, USA, 56--63. Google Scholar
- Cipolla, R., and Robertson, D. 1999. 3d models of architectural scenes from uncalibrated images and vanishing points. ICIAP 00. Google Scholar
- Criminisi, A., Reid, I. D., and Zisserman, A. 2000. Single view metrology. Int. J. of Computer Vision 40, 2, 123--148. Google ScholarCross Ref
- Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In SIGGRAPH 1996, Computer Graphics Proceedings, 11--20. Google Scholar
- Dick, A. R., Torr, P. H. S., and Cipolla, R. 2004. Modelling and interpretation of architecture from several images. Int. J. Comput. Vision 60, 2, 111--134. Google ScholarCross Ref
- El-Hakim, S., Whiting, E., and Gonzo, L. 2005. 3d modeling with reusable and integrated building blocks. The 7th Conference on Optical 3-D Measurement Techniques.Google Scholar
- Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381--395. Google ScholarDigital Library
- Gibson, S., Hubbold, R., Cook, J., and Howard, T. 2003. Interactive reconstruction of virtual environments from video sequences. Computers and Graphics 27, 2.Google ScholarCross Ref
- Goesele, M., Snavely, N., Curless, B., Hoppe, H., and Seitz, S. M. 2007. Multi-view stereo for community photo collections. In ICCV.Google Scholar
- Hoiem, D., Efros, A. A., and Hebert, M. 2005. Automatic photo pop-up. In SIGGRAPH 2005, Computer Graphics Proceedings. Google Scholar
- Igarashi, T., and Hughes, J. F. 2001. A suggestive interface for 3d drawing. In UIST, 173--181. Google Scholar
- Kim, S. J., and Pollefeys, M. 2008. Robust radiometric calibration and vignetting correction. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 4, 562--576. Google ScholarDigital Library
- Lempitsky, V. S., and Ivanov, D. V. 2007. Seamless mosaicing of image-based texture maps. In CVPR.Google Scholar
- Mueller, P., Zeng, G., Wonka, P., and Gool, L. V. 2007. Image-based procedural modeling of facades. ACM Trans. on Graphics (SIGGRAPH 2007) 26, 3, 85:1--85:9. Google Scholar
- Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In SIGGRAPH 2001, Computer Graphics Proceedings, 433--442. Google Scholar
- Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. on Graphics (SIGGRAPH'03) 22, 3, 313--318. Google Scholar
- Pollefeys, M., Gool, L. V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., and Koch, R. 2004. Visual modeling with a hand-held camera. Int. J. of Comput. Vision 59, 3, 207--232. Google ScholarCross Ref
- Pollefeys, M., Nister, D., Frahm, J.-M., Akbarzadeh, A., Mordohai, P., et al. 2008. Detailed real-time urban 3d reconstruction from video. Int. J. of Computer Vision (in press). Google Scholar
- Rother, C. 2002. A new approach for vanishing point detection in architectural environments. In BMVC, vol. 20, 382--391.Google Scholar
- Sketchup. http://www.sketchup.com.Google Scholar
- Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Trans. on Graphics (SIGGRAPH'06), 835--846. Google Scholar
- Triggs, B., McLauchlan, P., Hartley, R., and Fitzgibbon, A. 2000. Bundle adjustment -- A modern synthesis. In Vision Algorithms: Theory and Practice, W. Triggs, A. Zisser-man, and R. Szeliski, Eds., LNCS. Springer Verlag, 298--375. Google Scholar
- van den Hengel, A., Dick, A., Thormahlen, T., Torr, P. H. S., and B. Ward. 2006. Fitting multiple models to multiple images with minimal user interaction. In the Intl. Workshop on the Representation and use of Prior Knowledge in Vision.Google Scholar
- van den Hengel, A., Dick, A., Thormählen, T., Ward, B., and Torr, P. H. S. 2007. Videotrace: rapid interactive scene modelling from video. ACM Trans. on Graphics (SIG-GRAPH'07), 86. Google Scholar
- Werner, T., and Zisserman, A. 2002. New techniques for automated architecture reconstruction from photographs. In ECCV (2), 541--555. Google Scholar
- Wilczkowiak, M., Sturm, P., and Boyer, E. 2005. Using geometric constraints through parallelepipeds for calibration and 3d modeling. IEEE Trans. on PAMI 27, 2, 194--207. Google ScholarDigital Library
- Zeleznik, R., Herndon, K., and Hughes, J. 1996. Sketch: An interface for sketching 3D scenes. ACM Trans. on Graphics (SIGGRAPH'96), 163--170. Google ScholarDigital Library
- Ziegler, R., Matusik, W., Pfister, H., and McMillan, L. 2003. 3d reconstruction using labeled image regions. In SGP '03: Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposium on Geometry processing, 248--259. Google Scholar
Index Terms
- Interactive 3D architectural modeling from unordered photo collections
Recommendations
Interactive 3D architectural modeling from unordered photo collections
SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papersWe present an interactive system for generating photorealistic, textured, piecewise-planar 3D models of architectural structures and urban scenes from unordered sets of photographs. To reconstruct 3D geometry in our system, the user draws outlines ...
Photo tourism: exploring photo collections in 3D
Seminal Graphics Papers: Pushing the Boundaries, Volume 2We present a system for interactively browsing and exploring large unstructured collections of photographs of a scene using a novel 3D interface. Our system consists of an image-based modeling front end that automatically computes the viewpoint of each ...
NeuLighting: Neural Lighting for Free Viewpoint Outdoor Scene Relighting with Unconstrained Photo Collections
SA '22: SIGGRAPH Asia 2022 Conference PapersWe propose NeuLighting, a new framework for free viewpoint outdoor scene relighting from a sparse set of unconstrained in-the-wild photo collections. Our framework represents all the scene components as continuous functions parameterized by MLPs that ...
Comments