Dartmouth Events

[Abstract]

Every day, billions of images uploaded to photo-sharing websites feature people — selfies, family gatherings, parades, picnics, social events, etc. Thanks to recent advances in deep neural networks, we have automated tools that can reliably understand basic elements of these photos such as classifying the image category. However, developing a complex understanding of scenes and the rich interaction between people and their environment remains a challenging task.

In this talk, I will discuss People Analysis in Context (PAC) problems: reasoning and understanding humans in complex environments from visual data. In this setting, people must be considered jointly with their environment and not as isolated subjects. I define “context” broadly as all factors in a scene other than the person of interest: for example, these may include a picnic table, traffic signals, or a friend waiting across the street. Solving PAC problems has applications in many areas that can benefit from human behavior analysis such as autonomous driving, healthcare, and urban planning. I will talk about my research on PAC: including detecting existing and missing objects using context with the application on accessibility assessment, learning to predict walkable regions in images, and understanding contextual factors on people’s appearance. Finally, I will discuss the practical impact of PAC algorithms and how they could be useful tools in building better, safer, and more livable cities.

[Bio]

Jin Sun is a Postdoctoral Associate in the Department of Computer Science at Cornell University and Cornell Tech advised by Noah Snavely. Jin’s research interest is in developing computer vision methods for a complex understanding of objects and scenes. He has proposed algorithms to learn contextual information from visual data with limited supervision. He is also interested in applying computer vision in applications to improve people’s quality of life. Jin received his Ph.D. in computer science from the University of Maryland where he was advised by David W. Jacobs. His work has been published at top computer vision conferences such as CVPR, ICCV, and ECCV, selected as “Notable Books and Articles” in the 19th Annual ACM Best of Computing 2014, and nominated for the best paper award at CVPR 2020.