# MintNet: Building Invertible Neural Networks with Masked Convolutions

@article{Song2019MintNetBI, title={MintNet: Building Invertible Neural Networks with Masked Convolutions}, author={Yang Song and Chenlin Meng and Stefano Ermon}, journal={ArXiv}, year={2019}, volume={abs/1907.07945} }

We propose a new way of constructing invertible neural networks by combining simple building blocks with a novel set of composition rules. This leads to a rich set of invertible architectures, including those similar to ResNets. Inversion is achieved with a locally convergent iterative procedure that is parallelizable and very fast in practice. Additionally, the determinant of the Jacobian can be computed analytically and efficiently, enabling their generative use as flow models. To demonstrate… Expand

#### Supplemental Code

Github Repo

Via Papers with Code

MintNet: Building Invertible Neural Networks with Masked Convolutions

#### Figures, Tables, and Topics from this paper

#### 32 Citations

Understanding and mitigating exploding inverses in invertible neural networks

- Computer Science, Mathematics
- AISTATS
- 2021

This work shows that commonly-used INN architectures suffer from exploding inverses and are thus prone to becoming numerically non-invertible, and proposes a flexible and efficient regularizer for tasks where local invertibility is sufficient. Expand

HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inference

- Computer Science, Mathematics
- AAAI
- 2021

The power of the HINT method for density estimation and Bayesian inference on a novel data set of 2D shapes in Fourier parameterization, which enables consistent visualization of samples for different dimensionalities, is demonstrated. Expand

The Convolution Exponential and Generalized Sylvester Flows

- 2020

This paper introduces a new method to build linear flows, by taking the exponential of a linear transformation. This linear transformation does not need to be invertible itself, and the exponential… Expand

Semi-Supervised Learning with Normalizing Flows

- Computer Science, Mathematics
- ICML
- 2020

FlowGMM, an end-to-end approach to generative semi supervised learning with normalizing flows, using a latent Gaussian mixture model, is proposed, distinct in its simplicity, unified treatment of labelled and unlabelled data with an exact likelihood, interpretability, and broad applicability beyond image data. Expand

Relative Pixel Prediction For Autoregressive Image Generation

- Computer Science
- 2019

This paper proposes to predict pixels relatively, by predicting new pixels relative to previously generated pixels (or pixels from the conditioning context, when available), and shows that this form of prediction fare favorably to its absolute counterpart when used independently, but their coordination under an unified probabilistic model yields optimal performance. Expand

Large-Capacity Image Steganography Based on Invertible Neural Networks

- Computer Science
- CVPR
- 2021

The proposed large-capacity Invertible Steganography Network (ISN) takes steganography and the recovery of hidden images as a pair of inverse problems on image domain transformation, and introduces the forward and backward propagation operations of a single invertible network to leverage the image embedding and extracting problems. Expand

The Invertible U-Net for Optical-Flow-free Video Interframe Generation

- Computer Science
- ArXiv
- 2021

This paper is the world’s first attempt to use invertible networks instead of optical flows for video interpolation and proposes a learning method with a new consistency loss in the latent space to maintain semantic temporal consistency between frames. Expand

Argmax Flows and Multinomial Diffusion: Towards Non-Autoregressive Language Models

- Computer Science
- ArXiv
- 2021

This paper introduces two new classes of generative models for categorical data such as language or image segmentation: Argmax Flows and Multinomial Diffusion. Expand

Featurized Density Ratio Estimation

- Computer Science, Mathematics
- ArXiv
- 2021

This work proposes to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation, bringing the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. Expand

Learning Discrete Distributions by Dequantization

- Computer Science, Mathematics
- ArXiv
- 2020

A general framework for dequantization is presented that captures existing methods as a special case and derives two new de quantization objectives: importance-weighted (iw) dequantized and Renyi dequantification. Expand

#### References

SHOWING 1-10 OF 27 REFERENCES

Invertible Residual Networks

- Computer Science, Mathematics
- ICML
- 2019

The empirical evaluation shows that invertible ResNets perform competitively with both state-of-the-art image classifiers and flow-based generative models, something that has not been previously achieved with a single architecture. Expand

i-RevNet: Deep Invertible Networks

- Mathematics, Computer Science
- ICLR
- 2018

The i-RevNet is built, a network that can be fully inverted up to the final projection onto the classes, i.e. no information is discarded, and linear interpolations between natural image representations are reconstructed. Expand

FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models

- Computer Science, Mathematics
- ICLR
- 2019

This paper uses Hutchinson's trace estimator to give a scalable unbiased estimate of the log-density and demonstrates the approach on high-dimensional density estimation, image generation, and variational inference, achieving the state-of-the-art among exact likelihood methods with efficient sampling. Expand

Glow: Generative Flow with Invertible 1x1 Convolutions

- Computer Science, Mathematics
- NeurIPS
- 2018

Glow, a simple type of generative flow using an invertible 1x1 convolution, is proposed, demonstrating that a generative model optimized towards the plain log-likelihood objective is capable of efficient realistic-looking synthesis and manipulation of large images. Expand

Identity Mappings in Deep Residual Networks

- Computer Science, Mathematics
- ECCV
- 2016

The propagation formulations behind the residual building blocks suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation. Expand

MaCow: Masked Convolutional Generative Flow

- Computer Science, Mathematics
- NeurIPS
- 2019

MaCow is introduced, a simple yet effective architecture of generative flow using masked convolution, which achieves significant improvements over Glow for density estimation on standard image benchmarks, considerably narrowing the gap to autoregressive models. Expand

The Reversible Residual Network: Backpropagation Without Storing Activations

- Computer Science
- NIPS
- 2017

The Reversible Residual Network (RevNet) is presented, a variant of ResNets where each layer's activations can be reconstructed exactly from the next layer's, therefore, the activations for most layers need not be stored in memory during backpropagation. Expand

Emerging Convolutions for Generative Normalizing Flows

- Computer Science, Mathematics
- ICML
- 2019

The flexibility of d x d convolutions significantly improves the performance of generative flow models on galaxy images, CIFAR10 and ImageNet and is generalized to 1 x 1 convolutions proposed in Glow. Expand

Pixel Recurrent Neural Networks

- Computer Science
- ICML
- 2016

A deep neural network is presented that sequentially predicts the pixels in an image along the two spatial dimensions and encodes the complete set of dependencies in the image to achieve log-likelihood scores on natural images that are considerably better than the previous state of the art. Expand

Reversible Recurrent Neural Networks

- Computer Science, Mathematics
- NeurIPS
- 2018

This work shows that perfectly reversible RNNs, which require no storage of the hidden activations, are fundamentally limited, and provides a scheme for storing a small number of bits in order to allow perfect reversal with forgetting. Expand