Linking past to present: Discovering style in two centuries of architecture

Stefan Lee, Nicolas Maisonneuve, David Crandall, Alexei A. Efros, and Josef Sivic

New: Check out the interactive online demo!

With vast quantities of imagery now available online, researchers have begun to explore whether visual patterns can be discovered automatically. Here we consider the particular domain of architecture, using huge collections of street-level imagery to find visual patterns that correspond to semantic-level architectural elements distinctive to particular time periods. We use this analysis both to date buildings, as well as to discover how functionally similar architectural elements (e.g. windows, doors, balconies, etc.) have changed over time due to evolving styles. We validate the methods by combining a large dataset of nearly 150,000 Google Street View images from Paris with a cadastre map to infer approximate construction date for each facade. Not only could our analysis be used for dating or geo-localizing buildings based on architectural features, but it also could give architects and historians new tools for confirming known theories or even discovering new ones.

Overview and results

Using thousands of Street View images aligned to a cadastral map, we automatically find visual elements distinctive to particular architectural periods. For example, the white patch in figure below was found to be distinctive to the Haussmann period (late 1800s) in Paris, while the heat map (inset) reveals that the ornate balcony supports are the most distinctive features. We can also find functionally-similar elements from the same and different time periods.

fig1

The figure below shows more detailed results. For each of ten architectural periods of Parisian architecture, it shows the eight most visually-distinctive elements found by our automatic method. To visualize the elements, we show 5 sample patches (arranged in rows) corresponding to each element.

fig2

 

We can also find “chains” of similar functional elements whose appearance have evolved over time. The figure below shows sample chains of varying length and differing elements. For example, (a) and (i) show increasingly ornate window dressings starting from very plain structures before 1800, to multiple decorative structures in the 1851–1914 period. Pane (c) shows the long window balconies of the 1850s to 1940s, while (d) shows an evolution of short balconies. Many similar chains are produced as there is a great deal of variety in balcony shapes over time. Some chains show consistent directions of change, for instance Figure (e) demonstrates the increasing depth of windows. The last chain in Figure (f) highlights railings for large buildings after 1940, with the railings transitioning to glass in 1982–1989 and into metal in the 1990s.

fig3

To see more results, check out our interactive online demo, as well as the paper, below.

Papers and presentations

BibTeX entries:

@inproceedings{linking2015iccp,
    title = {Linking past to present: Discovering style in two centuries of architecture},
    author = {Stefan Lee and Nicolas Maisonneuve and David Crandall and Alexei Efros and Josef Sivic},
    booktitle = {IEEE International Conference on Computational Photography (ICCP)},
    year = {2015}
}

The IU Computer Vision Lab's projects and activities have been funded, in part, by grants and contracts from the Air Force Office of Scientific Research (AFOSR), the Defense Threat Reduction Agency (DTRA), Dzyne Technologies, EgoVid, Inc., ETRI, Facebook, Google, Grant Thornton LLP, IARPA, the Indiana Innovation Institute (IN3), the IU Data to Insight Center, the IU Office of the Vice Provost for Research through an Emerging Areas of Research grant, the IU Social Sciences Research Commons, the Lilly Endowment, NASA, National Science Foundation (IIS-1253549, CNS-1834899, CNS-1408730, BCS-1842817, CNS-1744748, IIS-1257141, IIS-1852294), NVidia, ObjectVideo, Office of Naval Research (ONR), Pixm, Inc., and the U.S. Navy. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Government, or any sponsor.