Discrete-continuous optimization for large-scale structure from motion

David CrandallAndrew OwensNoah SnavelyDan Huttenlocher

Recent work in structure from motion (SfM) has successfully built 3D models from large unstructured collections of images downloaded from the Internet. Most approaches use incremental algorithms that solve progressively larger bundle adjustment problems. These incremental techniques scale poorly as the number of images grows, and can drift or fall into bad local minima. We present an alternative formulation for SfM based on finding a coarse initial solution using a hybrid discrete-continuous optimization, and then improving that solution using bundle adjustment. The initial optimization step uses a discrete Markov random field (MRF) formulation, coupled with a continuous Levenberg-Marquardt refinement. The formulation naturally incorporates various sources of information about both the cameras and the points, including noisy geotags and vanishing point estimates. We test our method on several large-scale photo collections, including one with measured camera positions, and show that it can produce models that are similar to or better than those produced with incremental bundle adjustment, but more robustly and in a fraction of the time.

For more details, please see our CVPR 2011 paper and slides from our CVPR talk.

Sample reconstruction videos

Central Rome
(after multi-view stereo)

Cornell Arts Quad

Cornell Arts Quad
(after multi-view stereo)


[papersandpresentations proj=socialmining:3d]


In the CVPR 2011 paper, the reference to “V. Govindu. Lie-algebraic averaging for globally consistent motion estimation. CVPR, 2004” should instead be to “V. Govindu.  Combining Two-view Constraints For Motion Estimation. CVPR, 2001”.


We would like to thank the Cornell Facilities Team for helping us collect the ground truth Arts Quad dataset. We also gratefully acknowledge the support of the following:

National Science FoundationMIT Lincoln LabsGoogleIntel CorporationLilly Endowment
National Science
MIT Lincoln LabsGoogleIntel CorporationLilly EndowmentIU Data to Insight Center