Generalization and efficiency of graph neural ODEs for network dynamical systems

Event: NetSci 2025, Maastricht, NL

Date: 2025/06/05

Abstract

Complex systems from many domains, such as epidemiology, ecology, biochemistry, or neuroscience, can be modeled as a graph with node states that evolve over time according to a system of coupled ordinary differential equations (ODEs). Traditionally, these models are hand-crafted by domain experts based on prior knowledge about the system, using simple functional forms and a limited number of interpretable parameters. More recently machine learning methods have been used to predict the behavior of complex systems based on observational data alone, resulting in potentially more predictive but less interpretable models. Neural ODEs [1,2], a family of machine learning models that allow extracting the vector field of an ODE from time series data, promise to combine the best of both worlds, as they allow to structuring the vector field according to prior theoretical insight but are able to learn subtleties of a given system from data. While neural ODEs have become an important tool for data-driven modeling of complex systems, questions of robustness, generalization to out-of-distribution examples, and sample efficiency have only recently moved into the spotlight [3]. Here, we study how incorporating different levels of prior knowledge about network dynamical systems into the design of neural ODEs influences sample efficiency, generalization within and across random network ensembles, and the influence of different network structural features on predictive performance. We compare neural ODEs modeled directly after a family of network dynamical systems, sometimes referred to as Barzel-Barabási (BB) family [4], to neural ODEs basedon message-passing neural networks (MPNNs), and simple multi-layer perceptrons (MLPs). We find that architecturesbased on the BB family are capable of generalizations to graphs from different network ensembles both in terms of size and degree heterogeneity, which we vary using the non-linear preferential attachment model. We also find that incorporating stronger inductive bias in neural ODEs has the potential to substantially improve sample efficiency compared to a naïve, uninformative parametrization in terms of a MLP. Our results show how carefully designed inductive biases can improve generalization and efficiency of data-driven methods for forecasting dynamical systems, and motivate further research in using insights from network science to improve machine learning methods.

References

[1] Kidger, P. On Neural Differential Equations, PhD Thesis, University of Oxford (2021)

[2] Chen, R. et al. Advances in Neural Information Processing Systems, 31 (2018)

[3] Vasiliauskaite, V. & Antulov-Fantulin, N. Communications Physics 7, 348 (2024)

[4] Meena, C., Hens., C., Acharyya, S. et al. Nature Physics 19, 1033–1042 (2023)

Slides

When using these materials, please cite:

@misc{laber2026_neuralODE,
  title = {When do neural ordinary differential equations generalize on complex networks},
  author = {Laber, Moritz and Klein, Brennan and Eliassi-Rad, Tina},
  year = {2026},
  archiveprefix = {arXiv}
  eprint = {2602.08980},
  doi = {10.48550/arXiv.2602.08980}
}