We study zero-shot inverse problems, where a clean signal is recovered from a single degraded observation without external training data. Contrary to the common belief that such problems require highly complex models, we show that a lightweight neural network, when combined with entropy and complexity regularization in a compression-based formulation, is sufficient for high-quality restoration. We propose Lottery Prior, a compression-based inverse solver that leverages architectural priors from random networks and induces a family of implicit priors through randomness, enabling ensemble-based refinement. We further derive non-asymptotic error bounds for compression-based maximum-likelihood inverse solvers, revealing how rate–distortion constraints act as implicit regularizers. Experiments on denoising, noisy super-resolution, and inpainting demonstrate that our method achieves state-of-the-art with significantly fewer effective parameters.
Let \( d \) denote a distortion function and \( H \) the entropy function. For any overfitted image codec \( g_{\mathbf{W}}(\mathbf{z}) \), there exists an over-parameterized and randomly initialized network \( g_{\mathbf{W'}} \) with \( |\mathbf{W'}| > |\mathbf{W}| \) and a pair \( (\mathbf{\tau'}, \mathbf{z'}) \) as the ‘winning tickets’, such that \( d(\mathbf{S}, \mathbf{S}') \le d(\mathbf{S}, \mathbf{S}^*) \) and \( H(\mathbf{\hat{z}}') = H(\mathbf{\hat{z}}) \).
Fig. 1: From AE-based neural codec to LotteryCodec.
Fig. 2: (a) RD curve and BD-rate under different over-parameterization, (b) BD-rate vs. mask ratios, (c)-(d) BD-rate across width/depth variations, where \( (N_t, d) \) refers to \( N_t \) hidden layers and \( d \) hidden dimensions.
Fig. 3: The source image is encoded into a binary mask and latent modulations. During decoding, the receiver initializes a random network and uses a modulated subnetwork to reconstruct the source.
Fig. 4: SuperMask maps coordinates to RGB values by identifying subnetworks within a randomly initialized network, guided by modulations generated by the ModNet using latent input.
(a) RD curve and BD rate on Kodak dataset. (b) RD curve and BD rate on CLIC2020 dataset. (c) BD-rate and decoding complexity across different mask ratios on Kodak dataset.
@article{LotteryCodec,
title={LotteryCodec: Searching the Implicit Representation in a Random Network for Low-Complexity Image Compression},
author={Haotian Wu, Gongpu Chen, Pier Luigi Dragotti, and Deniz Gündüz},
journal={International Conference on Machine Learning (ICML) 2025},
year={2025}
}