Recently, great progress has been made in physics-based differentiable rendering. Existing differentiable rendering techniques typically focus on static scenes, but during inverse rendering---a key application for differentiable rendering---the scene is updated dynamically by each gradient step. In this paper, we take a first step to leverage temporal data in the context of inverse direct illumination. By adopting reservoir-based spatiotemporal resampled importance resampling (ReSTIR), we introduce new Monte Carlo estimators for both interior and boundary components of differential direct illumination integrals. We also integrate ReSTIR with antithetic sampling to further improve its effectiveness. At equal frame time, our methods produce gradient estimates with up to 100X lower relative error than baseline methods. Additionally, we propose an inverse-rendering pipeline that incorporates these estimators and provides reconstructions with up to 20X lower error.
@article{Wang:2023:PSDR-ReSTIR,
title={Amortizing Samples in Physics-Based Inverse Rendering using ReSTIR},
author={Wang, Yu-Chen and Wyman, Chris and Wu, Lifan and Zhao, Shuang},
journal={ACM Trans. Graph.},
volume={42},
number={6},
year={2023},
pages={214:1--214:17}
}
We thank the anonymous reviewers for their helpful suggestions. We are also grateful to Aaron Lefohn for his support. This work started when Yu-Chen Wang was an intern at NVIDIA and was partially funded by NSF grant 1900927 and an NVIDIA gift.