jan novák

Denoising with Kernel Prediction and Asymmetric Loss Functions

Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Gerhard Röthlin,
Alex Harvill, David Adler, Mark Meyer, Jan Novák

ACM Transactions on Graphics (Proceedings of SIGGRAPH 2018) - to appear

We introduce a novel, modular architecture based on kernel-predicting neural networks that performs multi-scale, temporal denoising of rendered sequences. Our network can be retrained using a small amount of data and performs robustly with artist-control over the variance-bias tradeoff on novel data, such as these two examples from our test set. © Disney / Pixar, © Disney.

abstract

We present a modular convolutional architecture for denoising rendered images. We expand on the capabilities of kernel-predicting networks by combining them with a number of task-specific modules, and optimizing the assembly using an asymmetric loss. The source-aware encoder--the first module in the assembly--extracts low-level features and embeds them into a common feature space, enabling quick adaptation of a trained network to novel data. The spatial and temporal modules extract abstract, high-level features for kernel-based reconstruction, which is performed at three different spatial scales to reduce low-frequency artifacts. The complete network is trained using a class of asymmetric loss functions that are designed to preserve details and provide the user with a direct control over the variance-bias trade-off during inference. We also propose an error-predicting module for inferring reconstruction error maps that can be used to drive adaptive sampling. Finally, we present a theoretical analysis of convergence rates of kernel-predicting architectures, shedding light on why kernel prediction performs better than synthesizing the colors directly, complementing the empirical evidence presented in this and previous works. We demonstrate that our networks attain results that compare favorably to state-of-the-art methods in terms of detail preservation, low-frequency noise removal, and temporal stability on a variety of production and academic data sets.

downloads

publication

supplementals

video

citation

video

bibtex

@article{Vogels2018KPAL,
    title     = {Denoising with Kernel Prediction and Asymmetric Loss Functions},
    author    = {Vogels, Thijs and Rousselle, Fabrice and McWilliams, Brian and R\"othlin, Gerhard and Harvill, Alex and Adler, David and Meyer, Mark and Nov\'ak, Jan},
    journal   = {ACM Transactions on Graphics (Proceedings of SIGGRAPH 2018)},
    volume    = {37},
    number    = {4},
    year      = {2018},
    articleno = {tbd},    
    pages     = {tbd},
    publisher = {ACM},
    address   = {New York, NY, USA},
    doi       = {10.1145/3197517.3201388},
}