Generating neural networks from equivariant embeddings with latent diffusion

Event: Boston Symmetry Day (2025), Boston, US

Date: 2025/03/31

Abstract

Recently neural networks themselves have become the input of machine learning models. Applying generative models like diffusion models to generate neural network weight matrices is an especially promising direction for more efficient uncertainty quantification and transfer learning. However, the permutation symmetry and high dimensionality of weight matrices pose challenges for applying diffusion models to weight matrix generation. Here, we propose using latent diffusion for this task and combine different approaches to incorporate symmetry in our model. This includes using a permutation invariant auto-encoder to construct latent representations and choosing canonical representations for training.

Poster

If you use this material please cite:

@inproceedings{gupta2026_deepweightflow,
    title={DeepWeightFlow: Re-Basined Flow Matching for Generating Neural Network Weights},
    author={Saumya Gupta and Scott Biggs and Moritz Laber and Zohair Shafi and Robin Walters and Ayan Paul},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026},
    url={https://openreview.net/forum?id=fOwsr1VTi8}
}