Accurate and Efficient Multi-Channel Time Series Forecasting via Sparse Attention Mechanism

Authors: Lei Gao, Hengda Bao, Jingfei Fang, Guangzheng Wu, Weihua Zhou, Yun Zhou Date: 2026-03-19 Paper ID: openalex:2603.18712

Summary

This paper introduces Li-Net, a novel architecture designed for accurate and efficient multi-channel time series forecasting, addressing the challenge of effectively modeling cross-channel dependencies. Li-Net operates by dynamically compressing representations across sequence and channel dimensions, processing them through a configurable non-linear module, and reconstructing forecasts. A key feature is the integration of a sparse Top-K Softmax attention mechanism guided by fused multi-modal embeddings to focus selectively on critical temporal and feature dimensions. Experimental results on benchmark datasets show that Li-Net achieves competitive accuracy while significantly reducing memory usage and inference time compared to existing state-of-the-art methods.

Key Contributions

• Introduction of Li-Net, a novel architecture for multi-channel time series forecasting that effectively captures both linear and non-linear dependencies across channels.
• Integration of a sparse Top-K Softmax attention mechanism within a multi-scale projection framework to selectively focus computation based on informative time steps and features.
• Ability to seamlessly incorporate and fuse multi-modal embeddings to guide the sparse attention mechanism, enhancing prediction accuracy.
• Demonstration of a superior balance between prediction accuracy and computational efficiency, exhibiting lower memory usage and faster inference times than state-of-the-art methods.

Limitations

The abstract does not explicitly state limitations, but the focus on linear/non-linear dependencies might imply challenges in capturing extremely complex, high-order interactions without further architectural depth.

Key Concepts

• Sparse Top-K Softmax Attention: A sparse attention mechanism used in Li-Net that integrates multi-modal embeddings to selectively focus on the most informative time steps and feature channels during multi-channel time series forecasting.

Datasets

• real-world benchmark datasets

Limitations

The abstract does not explicitly state limitations, but the focus on linear/non-linear dependencies might imply challenges in capturing extremely complex, high-order interactions without further architectural depth.

Links

• Abstract
• PDF