Stefan Lee, Nicolas Maisonneuve, David Crandall, Alexei A. Efros, and Josef Sivic

New: Check out the interactive online demo!

With vast quantities of imagery now available online, researchers have begun to explore whether visual patterns can be discovered automatically. Here we consider the particular domain of architecture, using huge collections of street-level imagery to find visual patterns that correspond to semantic-level architectural elements distinctive to particular time periods. We use this analysis both to date buildings, as well as to discover how functionally similar architectural elements (e.g. windows, doors, balconies, etc.) have changed over time due to evolving styles. We validate the methods by combining a large dataset of nearly 150,000 Google Street View images from Paris with a cadastre map to infer approximate construction date for each facade. Not only could our analysis be used for dating or geo-localizing buildings based on architectural features, but it also could give architects and historians new tools for confirming known theories or even discovering new ones.

Overview and results

Using thousands of Street View images aligned to a cadastral map, we automatically find visual elements distinctive to particular architectural periods. For example, the white patch in figure below was found to be distinctive to the Haussmann period (late 1800s) in Paris, while the heat map (inset) reveals that the ornate balcony supports are the most distinctive features. We can also find functionally-similar elements from the same and different time periods.

The figure below shows more detailed results. For each of ten architectural periods of Parisian architecture, it shows the eight most visually-distinctive elements found by our automatic method. To visualize the elements, we show 5 sample patches (arranged in rows) corresponding to each element.

We can also find “chains” of similar functional elements whose appearance have evolved over time. The figure below shows sample chains of varying length and differing elements. For example, (a) and (i) show increasingly ornate window dressings starting from very plain structures before 1800, to multiple decorative structures in the 1851–1914 period. Pane (c) shows the long window balconies of the 1850s to 1940s, while (d) shows an evolution of short balconies. Many similar chains are produced as there is a great deal of variety in balcony shapes over time. Some chains show consistent directions of change, for instance Figure (e) demonstrates the increasing depth of windows. The last chain in Figure (f) highlights railings for large buildings after 1940, with the railings transitioning to glass in 1982–1989 and into metal in the 1990s.

To see more results, check out our interactive online demo, as well as the paper, below.

[papersandpresentations proj=socialmining:linking]