DyAt Nets: Dynamic Attention Networks for State Forecasting
in Cyber-Physical Systems
Abstract
Multivariate time series forecasting is an important task in state forecasting for cyber-physical systems (CPS). State forecasting in CPS is imperative for optimal planning of system energy utility
and understanding normal operational characteristics of the system thus enabling anomaly detection. Forecasting models can also be used to identify sub-optimal or worn out components and are
thereby useful for overall system monitoring. Most
existing work only performs single step forecasting but in CPS it is imperative to forecast the next
sequence of system states (i.e curve forecasting).
In this paper, we propose DyAt (Dynamic Attention) networks, a novel deep learning sequence to
sequence (Seq2Seq) model with a novel hierarchical attention mechanism for long-term time series
state forecasting. We evaluate our method on several CPS state forecasting and electric load forecasting tasks and find that our proposed DyAt models yield a performance improvement of at least
13.69% for the CPS state forecasting task and a
performance improvement of at least 18.83% for
the electric load forecasting task over other stateof-the-art forecasting baselines. We perform rigorous experimentation with several variants of the
DyAt model and demonstrate that the DyAt models indeed learn better representations over the entire course of the long term forecast as compared
to their counterparts with or without traditional attention mechanisms. All data and source code has
been made available online