Faculty and researchers

Associate Professor
David received the Ph.D. in Computer Science from Cornell University in 2008 and the M.S. and B.S. degrees in Computer Science and Engineering from Penn State in 2001. He was a Postdoctoral Associate at Cornell from 2008-2010, and as a Research Scientist at Eastman Kodak Company from 2001-2003. His main research interests are in computer vision, applied machine learning, probabilistic graphical models, and data mining.
Postdoctoral Associate
Reza received his Ph.D. in Computer Science from George Mason University in 2018 and M.S. in Computer Science from Drexel University in 2011.He finished his B.Sc. in Computer Science & Engineering from Bangladesh University of Engineering and Technology (BUET) in 2007. His research focuses on developing novel algorithms for perception tasks such as semantic segmentation, object detection exploiting the 3D information.
Soon-heung Jung
Visiting Scholar
Soon-heung received Ph.D. and M.S. degrees in Electrical Engineering from Korea Advanced Institute of Science and Technology (KAIST), Korea, and the BS degree in Electronics from Pusan National University, Korea. Since 2005, he has been a principal researcher in Electronics and Telecommunications Research Institute (ETRI), Daejeon, Korea. His research interests include immersive media, computer vision, video coding, and realistic broadcasting system.

Current Ph.D. Students

Zhenhua Chen
Ph.D. Student, Computer Science
Zhenhua received his B.E. in Computer Engineering from Northeastern University (China) in 2012, and started to pursue his Ph.D. in Computer Science in 2015. Before he joined IU, he also worked as a software engineer in HappyElements (Beijing) for two years and as an assistant researcher in NLPR (National Laboratory of Pattern Recognition, Beijing) for one year. His current research focuses on object detection.
Ph.D. Candidate, Computer Science
Bardia is a Ph.D. student of Computer Science in IU where he also received his second MS. Back in Iran, he received a BS in Pure Mathematics and an MS in Computer Science both from University of Tehran. His research is focused on applying deep learning to culture analytics problems. His Ph.D. minor is in Music Informatics and he is interested in interdisciplinary work of vision and music.
Ph.D. Student, Informatics
Sam Goree received his BA in Computer Science and Music Theory from Oberlin College in 2017, and is now a PhD student in Informatics. He is interested in applying machine learning to creative and analytical problems in the arts.
Ph.D. Student, Computer Science
I am Eman Hassan, a PhD candidate in Computer Science. My main research interests are Machine Learning, Deep Learning and Computer Vision. Currently I am focusing on the use of generative models for domain adaptation, small object detection, and cross-domain retrieval.
Ph.D. Student, Computer Science
Shujon Naha is a Ph.D. student at Indiana University. He received his M.Sc. from University of Manitoba, Canada and his B.Sc. from BUET, Bangladesh. He is interested in developing computer vision algorithms which can effectively use weakly labeled or unlabeled data.
Ph.D. Student, Informatics
Satoshi Tsutsui is a Ph.D. student at Indiana University. He is interested in computer vision and deep learning (broadly defined). He has worked on scientific figure mining, multi-language image captioning, object detection, and semantic segmentation. Check out his publications on his website.
Chuhua Wang
Ph.D. Student, Computer Science
Chuhua received his M.S. in Technology Management from New York University in 2014 and M.S. in Data Science from Indiana University in 2018. He is now pursuing his Ph.D. in Computer Science and his research interest is focus on object recognition and generative adversarial networks.
Jingya Wang
Ph.D. Student, Computer Science
Jingya received her B.E. from Nankai University in 2009 and her M.S. in Computer Science from IU in 2011. Now she is pursuing her Ph.D in the Computer Science program. Her research interests are computer vision and social media data mining. More specifically, her passion is to find insights in images with object recognition and scene classification techniques.
Ph.D. Student, Intelligent Systems Engineering
Yuchen received his M.S. in Computer Science from Indiana University in 2017 and his B.E. in Software Engineering from Tongji University in 2011. He is currently pursuing his Ph.D. in Computer Engineering with a Statistics minor. He is interested in computer vision research, especially on action recognition and video understanding.
Qingyang (Keith) Xiao
Ph.D. Student, Computer Science
Keith received his M.E. in Computer Science from Tsinghua University in 2016, and he is currently pursuing Ph.D. in Computer Science at Indiana University. His research mainly focuses on video analysis, computer vision, mobile computing, and VR/AR.
Ph.D. Candidate, Computer Science
Mingze received his M.S. in Computer Science from Indiana University in 2014 and his B.E. in Software Engineering from Jilin University in 2012. He is currently pursuing his Ph.D. in Computer Science with a Statistics minor. His research interests are primarily in the area of computer vision and deep learning, especially on temporal modeling and reasoning in videos.
Ishtiak Zaman
Ph.D. Student, Computer Science
Ishtiak is currently pursuing his PhD on CS at IU. He completed his MS on CS from IU as well in 2016. His research is focused on deep learning and generative adversarial network. During pastime, he loves to explore photography, play billiard, or solve puzzles.
Zehua Zhang
Ph.D. Student, Computer Science
Zehua received his B.E. in Automation from Xi’an Jiaotong University in 2015 and came to IU to pursue his Ph.D. in Computer Science in 2016. As an undergraduate, he mainly focused on object tracking and applying machine learning and computer vision techniques to improving web security. Now he is interested in computer vision and machine learning problems in videos as well as graph convolution networks.

Current M.S. Students

Griffin Halloran
M.S. Student, Computer Science
Griffin received his B.S in Computer Science from IU in 2019 and is currently pursuing a MS degree in Computer Science. He is interested in researching the intersection of computer vision and data visualization to produce more explainable definitions of AI.
Xiwen Li
M.S. Student, Computer Science
Xiwen received his B.S. in Computer Science at Rose-Hulman Institute of Technology. He is currently pursuing a MSCS degree in WUSTL and is visiting IU in Summer 2019. He is interested in designing deep learning models to solve computer vision problems. During his visit to the lab, he was mainly working on application of capsule network on image segmentation and satellite image 3D reconstruction.
Violet Xiang
M.S. Student, Computer Science
Violet received her B.S. in Informatics and BA in Mathematics from IU in 2016. She is currently pursuing a MS degree in computer science. She is interested in studying the intersection of human and machine vision, neuro-inspired machine learning and scientific applications of computer vision.

Current Undergraduates

Kai Chen
Undergrad, Computer Science
Kai is currently a Computer Science undergraduate student at Fudan University and visited the IU Computer Vision Lab in Summer 2019. His research is focused on computer vision applications using deep learning, especially in object detection and segmentation. During his visit in IU, he was mainly working on automatic image segmentation, human-inspired computer vision, and satellite image 3D reconstruction.
Zunaeed Salahuddin
Undergrad, class of 2021
Zunaeed is an undergrad at IU studying Computer Science w/ an AI specialization, while minoring in Mathematics. He is working at the Computer Vision lab to fulfill his Cox Scholarship work component, while simultaneously familiarizing himself with tools and applications of CV and AI as a whole. He will be interning at Discover Financial Services as a Software Developer in the summer of 2020.

Ph.D. Student Alumni

Ph.D., 2016
Sven received a B.Eng. degree with a focus in media and imaging technology from Cologne University of Applied Sciences in Germany (2010). He came to Indiana in 2011 and received his M.S. in computer science in 2013, Ph.D. in both Computer Science and Cognitive Science 2016, and was a Postdoc until 2018. His main research interest is in understanding egocentric video, from both technical and human perspectives. He is now with Nationwide Children’s Hospital.
Ph.D., 2014
Kun’s work has studied Conditional Random Field Models for Structured Visual Object Recognition, presenting novel CRF-based solutions for three computer vision applications: human pose recognition, large-scale multimodal image clustering, and local attribute discovery for object recognition. He received his Ph.D. in August 2014 and worked as a Computer Scientist at GE before joining Snap in 2016.
Ph.D., 2019
Chenyou received his Ph.D. in 2019 and his M.S. in 2014 in Computer Science from Indiana University. He received the B.S. in Computer Science from Nanjing University, China. He is now with Google.
Ph.D., 2017
Jeff’s research is in planning and decision making in multi-agent robotic systems. His thesis developed a general framework to efficiently compute navigation solutions in partially observable systems. He has since worked to integrate the framework tightly with computer vision to enable robust, camera-based navigation. He received his Ph.D. in September 2017 and since 2012 has worked as a robotics and software engineer at major automotive and Silicon Valley companies.
Ph.D., 2015
Mohammed’s Ph.D. research studied large scale textual and visual mining in social media under the supervision of Prof. David Crandall. His other research interests include Machine Learning, Computer Vision, Text Mining, Web Mining, and Soft computing. He received his PhD in June 2015 and is now a Data Scientist at CareerBuilder, Inc.
Ph.D. 2018
Jangwon received the Ph.D. in the Intelligent and Interactive Systems track of Informatics in 2018. He was a software engineer at Samsung Electronics before coming to IU. He received his B.S. in Information and Communication Engineering from Sungkyunkwan University (SKKU) in 2006 and his M.S in Electrical and Computer Engineering from SKKU in 2008. His broad research interests are Human-Robot Interaction and computer vision for robotics. He is now at ObjectVideo Labs.
Ph.D., 2016
Stefan received his B.S. in Computer Science from the University of West Florida in 2011, his M.S. in Computer Science in 2013 from Indiana University, and his Ph.D. from IU in 2016. His thesis research was primarily focused on uncertainty in large-scale weakly labeled vision problems and inference in probabilistic graphical models. He is now an Assistant Professor at Oregon State University.
Ali Varamesh
M.S., 2016
Ali received his B.S. in Software Engineering and M.S. in Information Technology from Amirkabir University of Technology (Iran). He joined IU in 2014, where his research focused on detecting objects and activities using deep neural networks, improving reliability and interpretability of deep models by understanding adversarial attacks, and studying mental health with social media. In January 2018, Ali transferred to KU Leuven (Belgium) as a Ph.D. student in Engineering Science.
Ph.D., 2014
Haipeng’s work has studied how to use online and mobile data to estimate properties of and find connections in the physical world, demonstrated with applications to predicting consumer behavior, ecological events, mobile user properties, and concept relationships. He received his Ph.D. in August 2014 and is now an Assistant Professor at Shanghai Tech.

M.S. Student Alumni

M.S., 2015
Devendra received his M.S. degree in Computer Science from Indiana University in 2015. His Master’s thesis work was on ‘Morphological classification of galaxies into spirals and non-spirals’ using computer vision and machine learning techniques under the guidance of Prof. David Crandall. He is now in the Ph.D. program at University of Texas at Dallas.
M.S., 2018
Gurleen completed his Master’s in Computer Science from IU. His focus area is deep neural network learning, object detection, and Generative Adversarial Networks. Currently he is working to deploy models on embedded systems. He interned at Google in summer 2017 and is now a Software Engineer at Microsoft. When not working, he sketches, follows video game releases, and watches rally-racing.
M.S., 2016
Sumit received his M.S. in Computer Science from Indiana University Bloomington in 2016. His master’s thesis work involved application of deep learning techniques in infrared spectrum images, fine-grained recognition and egocentric scene classification.
Benjamin Cutilli
M.S. Student, Computer Science
Ben graduated from Haverford College with a Bachelor of Science degree in computer science. He is a master’s student at Indiana University Bloomington focusing on computer vision. He worked for NVIDIA as a Solutions Architect intern in the summer of 2017, focusing on style transfer using deep learning. In his free time he runs, skis, and follows his unrealistic dream of becoming good at Counter-Strike: Global Offensive.
Ryder McMinn
M.S. Student, Computer Science
Ryder graduated from Indiana University with a Bachelor of Science degree in Computer Science. He is continuing his education with a Master’s of Science in Data Science at Indiana. When not studying, he enjoys playing fetch with his dog, fantasizing about space flight, and trying to find the best noodles in town.
Accelerated M.S./B.S. student
Katie is currently pursuing her degree in computer science, with research interests in anomaly detection, mathematics, data visualization, and applications of computer vision to health and education, like her thesis detecting dyslexia with neural networks. She worked at IBM Research in the summer of 2018 on neuromorphic computing, and will be returning there full-time. She hopes to potentially get a PhD and become a corporate research scientist.
M.S., 2018
Ramya received her M.S. in Computer Science from Indiana University. Previously she was a software engineer at Accenture Services Private limited, and received her B.E. in Electronics and Communications Engineering from Visveswaraya Technological University in 2012.
Harsh Seth
M.S., 2015
Harsh received the M.S. degree in Computer Science from Indiana University in 2015. His Master’s thesis work (under the guidance of Prof. David Crandall) involved developing automated answering apps for the visually impaired, using Google Glass. He is now at Sears.
M.S., 2016
Manu Singh received his Masters degree in Computer Science from the School of Informatics and Computing in 2016. His master’s thesis work involved investigating tag selection and propagation methods for large scale social image classification. His research interests include computer vision, machine learning and data mining. He now works as a Cognitive Engineer at IBM Watson.
Tiangang Song
M.S., 2015
Tiangang Song received his Masters degree in Computer Science from the School of Informatics and Computing in 2015, and
received his B.E. from Wuhan University in 2013. His Master’s work investigated semi-automatic recognition using Human-in-the-loop techniques. His research interests include computer vision and machine learning. He now works at ESRI.
Tingyi Wanyan
M.S., 2016
Tingyi is currently a Ph.D. student in the Intelligent Systems Engineering department at IU. His current research direction is Artificial Intelligence, Computer Vision and Neural Imaging. He received his M.S degree at Indiana University in 2016, majoring in Computer Science and working in the Computer Vision Lab.

Undergraduate Student Alumni

Research Intern, 2014
Dennis is a senior in Electrical and Computer Engineering at Olin College. He was an undergraduate intern at IU during Summer 2014. His research interests include deep learning and applying computer vision to image classification problems.
Research Intern, 2015
Demetris studied Artificial Neural Networks at IU during the summer of 2015. He is an undergraduate student in the 5th year of his 3/2 engineering program, and plans to graduate in the Fall 2016 with degrees from Auburn University and Berea College in Electrical Engineering and Applied Math and Sciences.
B.S. Informatics, 2012
Russell, CEO and Founder of Ornicept, is a passionate entrepreneur and computer scientist. He left the lab to pursue the commercialization of his computer vision research created under Professor Crandall’s mentorship. This research continues today under a Department of Energy grant. At Ornicept, he leads the development of SPECTEO, an enterprise software package designed to improve the collection, processing, and accuracy of data and fieldwork for engineers, scientists, and researchers.
Tayla Frizell
Research Intern, Summer 2015
Tayla is an undergraduate student at Mississippi Valley State University. She studied handwriting recognition and scoring to study child development at IU in Summer 2015.
Gustavo Goncalves
Research Intern, Summer 2014
Gustavo is an undergraduate student at Dillard University. He studied object recognition and tracking for quadcopter at IU in Summer 2014.
Paul Grubbs
B.S. 2012
Paul is a Ph.D. student at Cornell University. He studied accelerating computer vision algorithms with GPUs in 2011-2012.
Jerry Gu
Research Intern, Fall 2019
Jerry Gu studies Computer Science and Technology at the University of Science and Technology of China, graduating in 2020. He was intern at IU in Fall 2019, working on image recognition and classification for several projects including an interdisciplinary application with the Department of Chemistry.
Emmanuel Klutse
Research Intern, Summer 2018
Emmanuel Klutse is an undergraduate majoring in Computer Science at Tougaloo College. His research focus for IU’s NSF REU program was Image Based Matching and Text Detection on Integrated Circuits. He plans on furthering his Computer Science education by earning a Master’s, and possibly a Ph.D. in either Computer Science or Software Engineering.
Research Intern, Summer 2017
Alan is a sophomore majoring in Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign (UIUC). He was a student in the summer of 2017 and worked on the project, Examining Computer Vision Techniques for Providing Useful Information to People with Visual Impairments. On the UIUC campus, Alan spends most of his free time involved in other research topics or in engineering groups.
Benjamin Newman
B.S., 2016
Ben is now a Ph.D. student at Carnegie Mellon University. At IU, he worked with Profs. David Crandall and Chen Yu on eye tracking for studying child learning.
Abraham Oliver
Undergrad, class of 2022
Abe worked in the Computer Vision Lab as a senior at Brown County High School, and is now an undergraduate student at Stanford University. His research interests include reinforcement learning, data visualization, and the mathematical foundations of machine learning. His long-range goal is to study quantum computing and quantum algorithms.
Research Intern, Summer 2017
Ethan is a Computer Science and Mathematics double major at Rose-Hulman Institute of Technology, graduating in 2018. He worked with Dr. David Crandall to use deep learning to study attention in driving. The research focused on teaching a convolutional net to extract the most relevant regions in high-resolution video data.
Gerald Pineda
Research Intern, Summer 2018
Gerald Pineda is a Computer Science and Studio Art double major at Depauw University, graduating in 2020. He worked with David Crandall and former partner, Emmanuel Klutse, in testing different approaches to do image matching for integrated circuits. He hopes to do more work with Computer Visions or User Experience Design in order to pursue his love for computers and the arts.
Research Intern, Summer 2019
Alejandra is an undergraduate at Case Western Reserve University. She interned at the IU Computer Vision Lab in the summer of 2019 focusing her research on computer vision for food recognition, studying both the models and their ethical implementations.
Research Intern, Summer 2017
Tyler is a student at Rose-Hulman Institute of Technology, studying Computer Engineering and Computer Science, 2018. Personally interested in computer vision, artificial intelligence, deep learning, neural networks, and embedded systems.
Alex Seewald
Research Intern, Summer 2014
Alex worked on deep learning for scene classification in Summer 2014 while an undergraduate at Earlham College. He is now a Ph.D. student in Computer Science at Indiana University.
Research Intern, Summer 2018
Anubhab Sen is a student at IIIT Hyderabad, India, studying Computer Science and Engineering, graduating in 2019. He worked with David Crandall in making a system that can identify notes played and fingers used from a piano video. His research is primarily focused on 3D reconstruction and pose estimation. He enjoys all the new exciting developments in computer vision and the underlying deep learning.
Joshua Sherfield
Research Intern, Summer 2013
Joshua investigated color distributions on Flickr at IU in Summer 2013, while an undergraduate at worked on deep learning for scene classification in Summer 2014 while an undergraduate at Norfolk State University.
Dylan Vener
Research Intern, Summer 2017
Dylan is a Computer Science and Mathematics double major at Rose-Hulman Institute of Technology, graduating 2019. He worked on evaluating viability and reliability of various computer vision techniques for application towards helping visually impaired people.
David Zhang
B.S., 2015
David investigated optical music recognition in summer and fall 2013.
David Crandall

David J. Crandall


Associate Professor
Director of Graduate Studies, Computer Science
Grant Thornton Scholar
Luddy School of Informatics, Computing, and Engineering
Indiana University

Mailing address: 901 E Tenth St
Physical address: 611 N Park Ave
(one block west of Informatics West)
Bloomington, IN 47408
djcran@indiana.edu

Fall 2019 office hours:
   Mondays 5-6pm, 611 N Park
   Sundays 10pm-11pm, online

Curriculum vitae

David Crandall photo

News
Oct 1, 2019: New grant from Office of Naval Research on explainable machine learning, in collaboration with Prof. David Leake.
Sept 27, 2019: David will speak about computer vision research at IU Research Unplugged, celebrating IU's 200th anniversary.
Aug 1, 2019: New grant from NSF with Linda Smith and Chen Yu, "Visual category learning by toddlers provides new principles for teaching rapid generalization."
June 27, 2019: New contract to collaborate with Dzyne on satellite image analysis.
May 1, 2019: New contract to collaborate with ETRI on depth map prediction using deep learning.
Apr 9, 2019: Congratulations to Katie Spoon for an IU Provost's Award for Outstanding Undergraduate Research, an NCWIT Award, and an NSF GRF! Check out her research project.
Jan 25, 2019: Congratulations to Dr. Chenyou Fan on a successful PhD defense!
Jan 15, 2019: David is pleased to be named a Grant Thornton Scholar for 2019-2020.
Oct 29, 2018: Thanks to all who participated in OurCS (aka HelloResearch) this weekend; we had a great time working with such talented students! More info in our SIGCSE 2019 paper.
Sep 12, 2018: We're looking for new faculty in Machine Learning, especially those who are broadly interested in potential connections with human learning. Consider applying!
Aug 13, 2018: Welcome to new postdoc Md Alimoor Reza!
Jul 19, 2018: Congratulations Dr. Jangwon Lee for successfully defending his Ph.D. thesis!
May 30, 2018: New contract with Purdue and the Indiana Innovation Institute on computer vision to help protect the microelectronics supply chain.
Apr 3, 2018: Congratulations to lab intern Abraham Oliver on winning best 12th grade project at the Indiana State Science Fair!
Mar 29, 2018: Congratulations to Dr. Tian (Linger) Xu on a successful PhD defense!
Feb 15, 2018: Co-organizing Third ICWSM workshop on The Social Web for Environmental and Ecological Monitoring. Please consider submitting!
Jan 24, 2018: New contract from NASA on quantifying lymphatic function using computer vision, with Dawn Kernagis (IHMC) and Eva Sevick (TMC).
Dec 5, 2017: Co-organizing second CVPR workshop on The Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security. Consider submitting a paper!
Nov 1, 2017: We have openings for postdoc researchers -- please contact David for more information!
Oct 20, 2017: Two fun remote sensing papers: 3d reconstruction of ice bed surfaces (at ICIP) and combining ground and satellite images to make maps (at ICCV).
Sep 1, 2017: Thanks to Google for sponsoring a new project investigating the limits of current computer vision technology.
Aug 23, 2017: IU is hiring faculty in the general area of Machine and Human Learning. Consider applying!
Jul 22, 2017: Check out our CVPR paper on associating wearable (egocentric) video with third-person video!
Jul 21, 2018: Congratulations to Jangwon Lee and Michael Ryoo for winning Best Paper at the CVPR Deep Learning for Robotic Vision Workshop.
Jun 26, 2017: How has visual design on the web changed over time? Two new papers use expert surveys and computer vision to help find out.
Jun 10, 2017: How can developmental psychology inform computer vision, and vice-versa? Check out the talks from our Workshop on Egocentric Vision: From Science to Real-World Applications.
May 20, 2017: I'm very honored for the IU Trustees Teaching Award. But the credit goes to great IU students who make teaching fun -- thanks everyone!
Apr 28, 2017: Congratulations to Dr. Jeff Johnson on a successful proposal defense!
Mar 20, 2017: Co-organizing WebSci workshop on The Social Web for Environmental and Ecological Monitoring. Deadline is May 1. Please consider submitting!
Feb 15, 2017: Co-organizing CVPR workshop on The Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security. Consider submitting a paper or abstract!
Jan 11, 2017: Very excited to lead the CS part of a new $2.5 million award to study interdisciplinary connections between human and machine learning!
Dec 15, 2016: Co-editing special issue on Egocentric Vision and Lifelogging Tools in the Journal of Visual Communication and Image Representation. Please consider submitting a manuscript!
Dec 7, 2016: Check out our NIPS 2016 paper on training diverse networks using multiple choice learning.
Oct 15, 2016: Robert Pless and I are CVPR 2017 tutorial co-chairs. Please consider submitting a proposal!
Oct 8, 2016: Could wearable cameras narrate our lives? Our "DeepDiary" paper at ECCV EPIC is a first step.
Aug 22, 2016: Best paper award at ICDL-Epirob 2016 on wearable cameras to study infant word learning, with Sven Bambach and Profs. Linda Smith and Chen Yu.
Aug 10, 2016: What may machine learning reveal about infant learning? Check out ourCogSci 2016 paper!
Aug 22, 2016: Congratulations to Dr. Sven Bambach on successfully defending his dissertation to earn two PhDs, in Computer Science and Cognitive Science!
Jul 30, 2016: We've moved! Come visit our new lab (and my new office) at 611 N Park Ave (one block west of Informatics West).
Jul 20, 2016: Congratulations to Dr. Stefan Lee on his dissertation defense!
Jul 13, 2016: New contract to collaborate with NSWC Crane on analyzing images of microelectronic devices!
Jul 1, 2016: Promoted to Associate Professor (with tenure).
Jun 27, 2016: Interested in improving diversity of your deep networks? Check out this talk by my student Stefan Lee at a CVPR tutorial.
May 26, 2016: Grant from the IU Social Science Research Commons to investigate applications of computer vision to geography, with Prof. Tom Evans. Thanks SSRC!!
Mar 29, 2016: Congratulations to senior Ben Newman on winning an NSF Graduate Research Fellowship!
Feb 12, 2016: Our new CHI 2016 paper on detecting computer screens to enhance lifelogger privacy is a CHI best paper honorable mention!
Jan 15, 2016: I'm co-organizing an International Workshop on the Social Web for Environmental and Ecological Monitoring at ICWSM 2016. Short and long papers due Feb 20. Consider submitting!
Sept 15, 2015: New ICCV 2015 paper, website, and dataset on hand recognition in first person video. Check out the video spotlight!
Aug 1, 2015: Welcome to new IU faculty member Prof. Michael Ryoo! Check out his interesting work on vision and robotics.
May 26, 2015: Congratulations Dr. Mohammed Korayem on a successful dissertation defense!
Apr 20, 2015: ICCP paper on mining architectural elements in Google Street View. Check out the demo!
Apr 15, 2015: Two CHI papers on privacy challenges and opportunities of wearable cameras
Jan 1, 2015: Recent and upcoming papers: WACV paper on geo-informative attributes, Signal Processing article on pose recognition, ICIP paper on ice layer detection
Dec 18, 2014: New NSF grant to study privacy in first-person cameras, with Apu Kapadia and Denise Anthony. Check out the video!
Oct 28, 2014: New NSF grant to help build high-performance computing libraries for computer vision and other applications (with G. Fox, J. Qiu, and others)
Sep 8, 2014: Looking for a faculty job? Our school is hiring in Interactive and Intelligent Systems (as well as several other areas).
Aug 22, 2014: Congratulations Dr. Haipeng Zhang on a successful dissertation defense!
Aug 6, 2014: Check out our redesigned lab website, including a nice new overview of our projects!
Jul 21, 2014: Congratulations Dr. Kun Duan on a successful dissertation defense!
Jun 28, 2014: Best paper award at CVPR Workshop on Egocentric Vision
Feb 18, 2014: Google Research Award to study privacy in first-person cameras, with Apu Kapadia
Feb 8, 2014: Our EMBO paper on science funding featured in Science
Jan 30, 2014: Our PlaceAvoider paper featured in MIT Tech Review and elsewhere
Dec 9, 2013: Best paper award at ICCV Workshop on CV for Converging Perspectives
Oct 1, 2013: Air Force Office of Scientific Research contractto study cloud robotics, with G. Fox and K. Hauser
May 2, 2013: IU Collaborative Research Grant to use first-person cameras to study parent interactions with deaf children (with Houston, Smith, Pisoni, Bergeson-Dana, Yu)
Mar 25, 2013: Awarded NSF CAREER grant
Jan 1, 2013: Awarded IU Faculty Research Support Program (FRSP) grant for studying privacy in first-person cameras, with Apu Kapadia
Sept 12, 2012: Awarded NSF EAGER grant with Chris Raphael, to study optical music recognition
June 15, 2012: Communications of the ACM paper on social photo mining (with Noah Snavely)
May 5, 2012: Our WWW 2012 paper on using Flickr to study ecology featured in New Scientist
April 11, 2012: Awarded subcontract from ObjectVideo to participate in the IARPA Finder program
Sept 12, 2011: Awarded IU Faculty Research Support Program grant to use first-person cameras for studying autistic and typically-developing children, with Chen Yu
June 22, 2011: Our paper on 3d reconstruction won runner-up best paper at CVPR 2011!
Dec 21, 2010: Our PNAS paper on inferring social ties from geo-tags featured in New Scientist and BBC World Service
Aug 24, 2010: Awarded grant from the IU D2I Center for mining social media for ecology
Sept 12, 2011: Awarded IU Faculty Research Support Program grant to use first-person cameras for studying autistic and typically-developing children, with Chen Yu
June 22, 2011: Our paper on 3d reconstruction won runner-up best paper at CVPR 2011!
Dec 21, 2010: Our PNAS paper on inferring social ties from geo-tags featured in New Scientist and BBC World Service
Aug 24, 2010: Awarded grant from the IU D2I Center for mining social media for ecology
Aug 1, 2010: Joining the faculty of Indiana University School of Informatics and Computing! -->

Research
For an overview of my research, check out our vision lab projects page.

I work in computer vision, the area of computer science concerned with automatically inferring semantic meaning from images -- teaching computers to "see." More generally, I am interested in problems that involve analyzing and modeling large amounts of uncertain data, like mining data from social networking websites.

I'm an Associate Professor in the School of Informatics, Computing, and Engineering at Indiana University, where I direct the IU Computer Vision Lab. I'm on the faculty of the Computer Science, Informatics, Cognitive Science, and Data Science programs, and I'm adjunct in the Department of Statistics. My work has been funded by grants and contracts from the National Science Foundation (including through a CAREER award), Google, Kodak, the Department of Defense, and Indiana University.

Before joining IU, I was a graduate student and postdoc in Computer Science at Cornell University. I worked with Professor Dan Huttenlocher on statistical part-based object recognition algorithms, and with Professor Jon Kleinberg on modeling and mining data from online social networks. I completed the Ph.D. degree in August 2008.

Before Cornell, I spent two years in the research labs of Eastman Kodak Company. There I worked mostly on image understanding and enhancement algorithms for both consumer and medical images.

As an undergraduate and Masters student at Penn State University, I worked with Professor Rangachar Kasturi on content-based video indexing, and specifically on detecting, tracking and recognizing text in video (both captions and text appearing naturally in a scene).


Publications
This is a partial list – please see my lab publications page or Google Scholar profile for more.

3-d reconstruction
  • "SfM with MRFs: Discrete-Continuous Optimization for Large-Scale Structure from Motion," in PAMI 2013 (with A. Owens, N. Snavely, D. Huttenlocher) [pdf] [project website]
  • "Discrete-Continuous Optimization for Large-Scale Structure from Motion," in CVPR 2011 (with A. Owens, N. Snavely, D. Huttenlocher) [pdf] [project website] Runner-up best paper!
First-person and opportunistic imagery
  • "Joint Person Segmentation and Identification in Synchronized First- and Third-person Videos," in ECCV 2018 (with M. Xu, C. Fan, Y. Wang, M. Ryoo) [pdf]
  • "From Coarse Attention to Fine-Grained Gaze: A Two-stage 3D Fully Convolutional Network for Predicting Eye Gaze in First Person Video," in BMVC 2018 (with Z. Zhang, S. Bambach, C. Yu) [pdf]
  • "Estimating Head Motion from Egocentric Vision," in ICMI 2018 (with S. Tsutsui, S. Bambach, C. Yu) [pdf]
  • "Identifying first-person camera wearers in third-person videos," in CVPR 2017 (with C. Fan, J. Lee, M. Xu, K.K. Singh, Y.J. Lee, M. Ryoo) [pdf]
  • "DeepDiary: Automatically Captioning Lifelogging Image Streams," in ECCV EPIC 2016 (with C. Fan) [pdf] [www with code and data]
  • "Enhancing Lifelogging Privacy by Detecting Screens," in CHI 2016 (with M. Korayem, R. Templeman, D. Chen, A. Kapadia) [pdf] [www] Best paper honorable mention!
  • "Lending A Hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions," in ICCV 2015 (with S. Bambach, S. Lee, C. Yu) [pdf] [www] [dataset]
  • "Viewpoint Integration for Hand-Based Recognition of Social Interactions from a First-Person View," in ICMI 2015 (with S. Bambach, C. Yu) [pdf] [dataset]
  • "PlaceAvoider: Steering First-Person Cameras away from Sensitive Spaces," in NDSS 2014 (with R. Templeman, M. Korayem, A. Kapadia) [pdf]
  • "This Hand is My Hand: A Probabilistic Approach to Hand Disambiguation in Egocentric Video," in CVPR Workshop on Egocentric Vision 2014 (with S. Lee, S. Bambach, J. Franchak, C. Yu) [pdf] Best paper!
  • "PlaceRaider: Virtual Theft in Physical Spaces with Smartphones," in NDSS 2013 (with R. Templeman, Z. Rahman, A. Kapadia) [pdf] [project website]
Recognition
  • "Fully-Coupled Two-Stream Spatiotemporal Networks for Extremely Low Resolution Action Recognition," in WACV 2018 (with M. Xu, A. Sharghi, X. Chen)
  • "Vehicle Recognition with Constrained Multiple Instance SVMs," in WACV 2014 (with K. Duan, L. Marchesotti) [pdf]
  • "Discovering localized attributes for fine-grained recognition," in CVPR 2012 (with K. Duan, D. Parikh, K. Grauman) [pdf] [project website]
  • "A Multi-layer Composite Model for Human Pose Estimation," in BMVC 2012 (with K. Duan, D. Batra) [pdf] [project website]
  • "Landmark classification in large-scale image collections," in ICCV 2009 (with Y. Li and D. Huttenlocher) [pdf] [expanded book chapter]
  • "Composite models of objects and scenes for category recognition," in CVPR 2007 (with D. Huttenlocher) [pdf]
  • "Weakly-supervised learning of part-based spatial models for visual object recognition," in ECCV 2006 (with D. Huttenlocher) [pdf] [source code]
  • "Spatial Priors for Part-Based Recognition using Statistical Models," in CVPR 2005 (with P. Felzenszwalb and D. Huttenlocher) [pdf] [expanded LNCS version] [source code]
  • "Robust Color Object Detection using Spatial-Color Joint Probability Functions," in IEEE Transactions on Image Processing, 2006 (with J. Luo) [CVPR 2004 version]
  • "Part-based statistical models for visual object class recognition," 2008 (Ph.D. thesis) [pdf]
Mining and modeling large-scale photo collections
  • A Unified Model for Near and Remote Sensing," in ICCV 2017 (with S. Workman, M. Zhai, N. Jacobs) [pdf]
  • "Tracking Natural Events through Social Media and Computer Vision," in ACM Multimedia 2016 (with J. Wang, M. Korayem, S. Blanco) [pdf]
  • "Linking Past to Present: Discovering Style in Two Centuries of Architecture," in ICCP 2015 (with S. Lee, N. Maisonneuve, A. Efros, J. Sivic) [pdf] [project website]
    See also an online demo of some results.
  • "Predicting Geo-informative Attributes in Large-scale Image Collections using Convolutional Neural Networks," in WACV 2015 (with S. Lee, H. Zhang) [pdf]
  • "Multimodal Learning in Loosely-organized Web Images," in CVPR 2014 (with K. Duan, D. Batra) [pdf]
  • "Observing the natural world with Flickr," in ICCV Workshop on Computer Vision for Converging Perspectives, 2013 (with J. Wang, M. Korayem) [pdf] Best paper!
  • "Modeling people and places with internet photo collections," in Communications of the ACM (with N. Snavely) [pdf]
  • "Mining photo-sharing websites to study ecological phenomena," in WWW 2012 (with H. Zhang, M. Korayem, G. LeBuhn) [pdf] [project website]
  • "Beyond co-occurrence: Discovering and visualizing tag relationships from geo-spatial and temporal similarities," in WSDM 2012 (with H. Zhang, M. Korayem, E. You) [pdf] [project website]
  • "Mapping the World's Photos," in WWW 2009 (with L. Backstrom, D. Huttenlocher, J. Kleinberg) [pdf] Runner-up best paper!
    See also a gallery of automatically-generated maps.
Mining and modeling online social networks
  • "Utilizing remote sensing and big data to quantify conflict intensity: The Arab Spring as a case study," in Applied Geography 2018 (with N. Levin, S. Ali) [pdf]
  • "Where have all the people gone? Enhancing global conservation using night lights and social media," in Ecological Applications 2015 (with N. Levin, S. Kark) [pdf]
  • "De-anonymizing users across heterogeneous social computing platforms," in ICWSM 2013 (with M. Korayem) [pdf] [project website]
  • "Inferring Social Ties from Geographic Coincidences," in Proc. National Academy of Sciences (PNAS), 8 December 2010 (with L. Backstrom, D. Cosley, S. Suri, D. Huttenlocher, J. Kleinberg) [pdf]
  • "Feedback Effects between Similarity and Social Influence in Online Communities," in KDD 2008 (with D. Cosley, J. Kleinberg, D. Huttenlocher, S. Suri) [pdf]
Image privacy attitudes and applications
  • "Viewer Experience of Obscuring Scene Elements in Photos to Enhance Privacy," in CHI 2018 (with R. Hasan, E. Hassan, Y. Li, K. Caine, R. Hoyle, A. Kapadia) [pdf]
  • "Cartooning for enhanced privacy in lifelogging and streaming video," in CVPR CV-COPS 2017 (with E. Hassan, R. Hasan, P. Shaffer, A. Kapadia) [pdf]
  • "Addressing physical safety, security, and privacy for people with visual impairments," in SOUPS 2016 (with T. Ahmed, P. Shaffer, K. Connelly, A. Kapadia) [pdf]
  • "Sensitive Lifelogs: A Privacy Analysis of Photos from Wearable Cameras," in CHI 2015 (with R. Hoyle, R. Templeman, D. Anthony, A. Kapadia) [pdf]
  • "Privacy Concerns and Behaviors of People with Visual Impairments" in CHI 2015 (with T. Ahmed, R. Hoyle, K. Connelly, A. Kapadia) [pdf]
  • "Privacy Behaviors of Lifeloggers using Wearable Cameras," in Ubicomp 2014 (with R. Hoyle, R. Templeman, S. Armes, D. Anthony, A. Kapadia) [pdf]
Human vision and learning
  • "Exploring inter-observer differences in first-person object views using deep learning models," in ICCV MBCCV workshop, 2017 (with S. Bambach, Z. Zhang, C.Yu) [pdf]
  • "An Egocentric Perspective on Active Vision and Visual Object Learning in Toddlers," in ICDL 2017 (with S. Bambach, L. Smith, C. Yu) [pdf]
  • "Active Viewing in Toddlers Facilitates Visual Object Learning: An Egocentric Vision Approach," in CogSci 2016 (with S. Bambach, L. Smith, C. Yu) [pdf]
  • "Objects in the Center: How the Infant's Body Constrains Infant Science," in ICDL 2016 (with S. Bambach, L. Smith, C. Yu) [pdf] Best paper!
  • "Detecting Hands in Children's Egocentric Views to Understand Embodied Attention during Social Interaction," in CogSci 2014 (with S. Bambach, J. Franchak, C. Yu) [pdf]
  • "Understanding embodied visual attention in child-parent interaction," in ICDL 2013 (with S. Bambach, C. Yu) [pdf]
  • "Psychophysical study of image orientation perception," in Spatial Vision, 2003, pp. 429-457. (with J. Luo, A. Singhal, M. Boutell and R. Gray.) [pdf]
Deep networks
  • "Diverse Beam Search for Improved Description of Complex Scenes," AAAI Conference on Artificial Intelligence," AAAI 2018 (with A. Vijayakumar, M. Cogswell, R. Selvaraju, Q. Sun, S. Lee, D. Batra)
  • "Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles," NIPS 2016 (with S. Lee, S. Purushwalkam, M. Cogswell, V. Ranjan, D. Batra) [pdf]
  • "Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models," arXiv 2016 (with A. Vijayakumar, M Cogswell, R. Selvaraju, Q. Sun, S. Lee, D. Batra) [arXiv link]
  • "Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks," arXiv 2015 (with S. Lee, S. Purushwalkam, M. Cogswell, D. Batra) [arXiv link]
Computer vision for polar science
  • "Multi-Task Spatiotemporal Neural Networks for Structured Surface Reconstruction," in WACV 2018 (with M. Xu, C. Fan, J. Paden, G. Fox, D. Crandall) [pdf]
  • "Automatic estimation of ice bottom surfaces from radar imagery", in ICIP 2017 (with M. Xu, G. Fox, J. Paden) [pdf]
  • "Estimating bedrock and surface layer boundaries and confidence intervals in ice sheet radar imagery using MCMC", in ICIP 2014 (with S. Lee, J. Mitchell, G. Fox) [pdf] [project website]
  • "Layer-finding in radar echograms using probabilistic graphical models", in ICPR 2012 (with G. Fox, J. Paden) [pdf] [project website]
Text and document analysis
  • A Data Driven Approach for Compound Figure Separation Using Convolutional Neural Networks," in ICDAR 2017 (with S. Tsutsui) [pdf] [project website]
  • "Extraction of special effects caption text events from digital video," in International Journal on Document Analysis and Recognition, 2002 (with S. Antani and R. Kasturi) [pdf] [M.S. thesis version] [ICDAR01 version]
Other interests :)
  • "A deep study into the history of web design," in WebSci 2017 (with B. Doosti, N. Su) [pdf] [project website]
  • "Understanding the Aesthetic Evolution of Websites: Towards a Notion of Design Periods," in CHI 2017 (with W. Chen, N. Su) [pdf]
  • "From funding agencies to scientific agency: Collective allocation of science funding as an alternative to peer review," in EMBO Reports, 2014 (with J. Bollen, D. Junk, Y. Ding, K. Borner) [pdf]
  • "Learning visual features for the Avatar Captcha Recognition Challenge," in ICMLA 2012 (with M. Korayem, A. Mohammed, D. Crandall, R. Yampolskiy) [pdf]
  • "Subjectivity and Sentiment Analysis of Arabic: A Survey", in AMLTA, 2012 (with M. Korayem, M. Abdul-Mageed) [pdf]

Teaching
Current:
  • CS B657, Computer Vision (online and hybrid sections)

Past:
  • CS B551, Elements of Artificial Intelligence (Fall 2015, Fall 2016, Spring 2017, Fall 2017, Fall 2018)
  • CS B657, Computer Vision, (Fall 2010, Spring 2012, Spring 2014, Spring 2016, Spring 2017, Spring 2018, Spring 2019)
  • CS B490/B659, Image processing and recognition (Spring 2015)
  • Info I210, Information Infrastructure I (Fall 2014)
  • Info I399, Research Methods for Undergraduates (Fall 2013)
  • CS B553, Probabilistic Approaches to Artificial Intelligence (Spring 2013)
  • Info I427, Search Informatics (Fall 2011, Spring 2011, Fall 2012, Fall 2013, Fall 2014, Fall 2015)
  • Info 2950, Mathematical Methods for Information Science, Fall 2008 (at Cornell)
  • CS 113, Introduction to C, Fall 2007 and Spring 2008 (at Cornell)
  • CS 211, Algorithms and Data Structures in Java, Summer 2007 (at Cornell)
  • T.A. for CS 664, Computer Vision, Fall 2003 (at Cornell)
  • CSE 275, Digital Design Lab, Fall 2000 and Spring 2001 (at Penn State)

Graduated Ph.D. Students
For a complete list of current and former graduate and undergraduate students, please see our lab web page.

Other projects
These are other projects I've worked on that were never polished enough to be publishable, but that someone somewhere might still find interesting. :-)

Fun
I love cycling. During graduate school I would take multi-day bike trips spanning hundreds of miles, but the tenure track has made those trips less feasible. These days I stay closer to home, enjoying weekend rides along the beautiful country roads of southern Indiana. I used to ski during winter until I moved to the second flattest state east of the Mississippi.

My current preferred winter activity is swimming in IU's fantastic pools. (My unofficial hobby is to swim in a public lap pool in as many cities of the world as possible (so far: Amsterdam, Barcelona, Beijing, Blacksburg, Bloomington, Boulder, Chicago, Columbus, Copenhagen, Dallas, D.C., Denver, Detroit, Erie, Fort Collins, Glasgow, Hong Kong, Honolulu, Indianapolis, Ithaca, Jamestown, Jerusalem, Kailua-Kona, Las Vegas, Los Angeles, Madrid, Minneapolis, Montreal, Munich, Nashville, New Orleans, Newcastle upon Tyne, Paris, Phoenix, Quebec City, Salt Lake City, San Diego, San Jose, Santiago, Seoul, Shanghai, State College, Sydney, Tokyo, Vancouver, Venice).

I took piano for 12 years as a child and recently started picking it back up again. I've also been trying to learn organ. Since I can't produce great music myself, I'm happy to live just a few blocks from the amazing IU Jacobs School of Music.

I'm on a perpetual quest to become fluent in Spanish. In addition to taking classes in high school and throughout college and graduate school, I've studied abroad in Spain and Mexico, and I've also traveled to Puerto Rico, Chile, and El Salvador. Despite all that effort, my Spanish is still remarkably lousy. :)

If you like pretty pictures, you might check out my dad's amazing bird photos. My own household flock includes Pichu, Ron, Sally, Betsy, and Sebastian; alumni are Toto, Zuma, Rameses, Rameses II, and, of course, Cocoa.