Tracking Natural Events through Social Media and Computer Vision

Jingya Wang, Mohammed Korayem, Saul Blanco, David Crandall, 

Accurate, efficient, global observation of natural events is important for ecologists, meteorologists, governments, and the public. Satellites are effective but limited by their perspective and by atmospheric conditions. Public images on photo-sharing websites could provide crowd-sourced ground data to complement satellites, since photos contain evidence of the state of the natural world. In this work, we test the ability of computer vision to observe natural events in millions of geo-tagged Flickr photos, over nine years and an entire continent. We use satellites as (noisy) ground truth to train two types of classifiers, one that estimates if a Flickr photo has evidence of an event, and one that aggregates these estimates to produce an observation for given times and places. We present a web tool for visualizing the satellite and photo observations, allow- ing scientists to explore this novel combination of data sources.

Papers and presentations

BibTeX entries:

@inproceedings{tracking2016mm,
    title = {Tracking Natural Events through Social Media and Computer Vision},
    year = {2016},
    booktitle = {ACM International Conference on Multimedia (MM)},
    author = {Jingya Wang and Mohammed Korayem and Saul Blanco and David Crandall}
}

Acknowledgements

We thank Dennis Chen and Alex Seewald for assisting with initial data collection and system configuration.

We also gratefully acknowledge the support of the following:

Lilly Endowmen National Science Foundation IBM
Lilly Endowment IU Data to Insight Center National Science Foundation IBM
The IU Computer Vision Lab's projects and activities have been funded, in part, by grants and contracts from the Air Force Office of Scientific Research (AFOSR), the Defense Threat Reduction Agency (DTRA), Dzyne Technologies, EgoVid, Inc., ETRI, Facebook, Google, Grant Thornton LLP, IARPA, the Indiana Innovation Institute (IN3), the IU Data to Insight Center, the IU Office of the Vice Provost for Research through an Emerging Areas of Research grant, the IU Social Sciences Research Commons, the Lilly Endowment, NASA, National Science Foundation (IIS-1253549, CNS-1834899, CNS-1408730, BCS-1842817, CNS-1744748, IIS-1257141, IIS-1852294), NVidia, ObjectVideo, Office of Naval Research (ONR), Pixm, Inc., and the U.S. Navy. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Government, or any sponsor.