Tracking Natural Events through Social Media and Computer Vision

Jingya Wang, Mohammed Korayem, Saul Blanco, David Crandall

Accurate, efficient, global observation of natural events is important for ecologists, meteorologists, governments, and the public. Satellites are effective but limited by their perspective and by atmospheric conditions. Public images on photo-sharing websites could provide crowd-sourced ground data to complement satellites, since photos contain evidence of the state of the natural world. In this work, we test the ability of computer vision to observe natural events in millions of geo-tagged Flickr photos, over nine years and an entire continent. We use satellites as (noisy) ground truth to train two types of classifiers, one that estimates if a Flickr photo has evidence of an event, and one that aggregates these estimates to produce an observation for given times and places. We present a web tool for visualizing the satellite and photo observations, allow- ing scientists to explore this novel combination of data sources.

Papers and presentations

BibTeX entries:

@inproceedings{tracking2016mm,
    title = {Tracking Natural Events through Social Media and Computer Vision},
    year = {2016},
    booktitle = {ACM International Conference on Multimedia (MM)},
    author = {Jingya Wang and Mohammed Korayem and Saul Blanco and David Crandall}
}

Acknowledgements

We thank Dennis Chen and Alex Seewald for assisting with initial data collection and system configuration.

We also gratefully acknowledge the support of the following:

Lily Endowment National Science Foundation IBM
Lilly Endowment IU Data to Insight Center National Science Foundation IBM