Dartmouth Events

Abstract: Video communication connects our world. The ongoing global pandemic has highlighted the necessity of video as a medium for virtual education, tele-health, business, political discourse, and creative contents. However, the visual quality of these videos significantly lack that of the professionally created videos captured with expensive equipment and edited with human expertise. In this talk, I will discuss my ongoing research on creating next-gen video communication and content creation framework by democratizing high-quality video production and editing. To improve various components of a video computationally (e.g. change lighting, replace the background etc.), we first need to infer those intrinsic components of the video related to the 3D world, and then edit them. This problem, which is often known as Inverse Graphics, is a holy-grail problem in Computer Vision. It is computationally challenging and severely under-constrained. To solve Inverse Graphics for democratizing video production, my research adopts a unique user-centric and personalized AI based approach. I will first discuss how simple user interaction strategies, such as capturing an extra image, can help us perform high-quality background replacement in real-time. Then, I will discuss how we can personalize AI models by training on the data of a specific user captured by their webcam to improve lighting.

Bio: Soumyadip Sengupta is a Postdoctoral Research Associate in Paul G. Allen School of Computer Science and Engineering at the University of Washington, Seattle. He is affiliated with GRAIL (Graphics and Imaging Laboratory), working with Steve Seitz, Brian Curless, and Ira Kemelmacher-Shlizerman. He did his Ph.D. under the supervision of David Jacobs at University of Maryland, College Park and graduated in 2019. His research interest lies at the intersection of Computer Vision and Computer Graphics, with a focus on solving Inverse Graphics for democratizing high-quality video production. His work on Background Matting received Best Student Paper Honorable Mentions at CVPR 2021 (Top 7 papers out of 1600 accepted) and has been adopted by various companies, e.g. Microsoft, Inter State Studio, etc. More information can be found on his webpage https://homes.cs.washington.edu/~soumya91/.