Biography：
Wenzheng Chen is a 5th-year Ph.D. student at University of Toronto, supervised by Prof. Sanja Fidler and Prof. Kyros Kutulakos. He is also a research scientist at NVIDIA. His research mainlyfocuses on the intersection of computational photography and machine learning, with a special interest in estimating 3D properties from various imaging systems. His long term research goal isto explore all kinds of optical and physical rules in the imaging systems and embed them in deep learning to boost 3D perception. He received his MS and BS in Shandong University. For more information, please visit http://www.cs.toronto.edu/~wenzheng/.
Abstract：
The ability to infer 3D properties, such as geometry, texture, material, and light from 2D photographs is one of the most fundamental problems in computer vision and plays as key in many domains such as AR/VR, robotics, autonomous driving, and Metaverse. To tackle this problem, various imaging systems and reconstruction algorithms are proposed. As a two-stage procedure, 3D perception contains "imaging" and "reconstruction". The former adopts devices to measure the scene and acquire the measurement (e.g. photos) while the latter applies reconstruction algorithms on the measurement to recover 3D properties. While previous methods focus more on reconstruction algorithms and ignore the knowledge in imaging stage, in this talk, we show how to develop a holistic method to connect "imaging" and "reconstruction" together, utilizing the information from both sides. Specifically, we propose to model various imaging systems in a differentiable way and connect them with the reconstruction algorithms, such that the information(e.g. gradients) can be freely propagated bidirectionally. Such a connection boosts 3D perception in 2 directions: "learning for imaging"' and "imaging for learning". The former passes the information of imaging systems to the reconstruction algorithms, which helps design the algorithms that perfectly match the chosen imaging systems. The later propagates the information from reconstruction algorithms to imaging systems, helping explore the best settings they should adopt under specific algorithms. We demonstrate the proposed method brings huge improvement and successfully push the boundary of 3D perception.
