Spatiotemporal System Forecasting with Irregular Time Steps via Masked Autoencoder

Authors: Kewei Zhu, Yanze Xin, Jinwei Hu, Xiaoyuan Cheng, Yiming Yang, Sibo Cheng Date: 2026-03-26 Paper ID: arxiv:2603.25597

Summary

This paper introduces the Physics-Spatiotemporal Masked Autoencoder (PSMAE) to address the challenge of forecasting high-dimensional systems with irregular time steps, often caused by sparse or missing observations. The method employs a hybrid architecture combining convolutional autoencoders for spatial feature extraction and masked autoencoders optimized for irregular temporal modeling, utilizing attention to reconstruct the full sequence in one step. By avoiding explicit data imputation, PSMAE preserves the underlying physical integrity of the dynamical system’s field representation. Evaluations across simulated and real-world ocean temperature datasets show that PSMAE significantly outperforms conventional convolutional and recurrent network baselines in accuracy and robustness.

Key Contributions

• Introduction of the Physics-Spatiotemporal Masked Autoencoder (PSMAE) tailored for irregular time-step forecasting, avoiding explicit data imputation.
• Integration of convolutional autoencoders for spatial feature learning and masked autoencoders adapted for irregular temporal patterns within a single prediction pass.
• Demonstrated significant improvement in prediction accuracy and robustness over traditional CNN and RNN methods on simulated and real-world high-dimensional system data.

Limitations

The evaluation is primarily demonstrated on simulated datasets and ocean temperature data, and performance on other highly complex dynamical systems remains to be fully explored.

Open Questions & Future Work

• transformer-quadratic-complexity-scalability
• advanced-positional-embeddings-irregular-time
• advanced-spatial-encoding-techniques
• multi-objective-optimization-structural-fidelity
• broader-validation-real-world-generalization

Key Concepts

• Physics-Spatiotemporal Masked Autoencoder: A novel forecasting method integrating convolutional autoencoders for spatial feature extraction with masked autoencoders optimized for irregular time series, using attention for single-pass sequence reconstruction.

Datasets

• ocean temperature data

Limitations

The evaluation is primarily demonstrated on simulated datasets and ocean temperature data, and performance on other highly complex dynamical systems remains to be fully explored.

Links

• ArXiv Abstract
• PDF