NEW preprint out on a Koopman operator-based prediction algorithm and its applications to predicting COVID-19 and other pandemics! Done with the fantastic AIMdyn team (Dr. Maria Fonoberova, Dr. Ryan Mohr, Dr. Allan Avila, Aleksandr Andrejcuk and Prof. Igor Mezic) and collaborators at the University of Zagreb (Prof. Zlatko Drmac and Iva Manojlovic) and University of Rijeka (Prof. Nelida Crnjaric-Zic and Senka Macesic), and generously funded by the DARPA, DARPA SBIR, NIH/NIAA, the University of Rijeka, and the Croatian Science Foundation.
https://arxiv.org/abs/2304.13601
The problem of prediction of behavior of dynamical systems has undergone a paradigm shift in the second half of the 20th century with the discovery of the possibility of chaotic dynamics in simple, physical, dynamical systems for which the laws of evolution do not change in time. The essence of the paradigm is the long-term exponential divergence of trajectories. However, that paradigm does not account for another type of unpredictability: the “Black Swan” event. It also does not account for the fact that short-term prediction is often possible even in systems with exponential divergence. In our framework, the Black Swan type dynamics occurs when an underlying dynamical system suddenly shifts between dynamics of different types. A learning and prediction system should be capable of recognizing the shift in behavior, exemplified by “confidence loss”. In this paradigm, the predictive power is assessed dynamically, and confidence level is used to switch between long term prediction and local-in-time prediction. Here we explore the problem of prediction in systems that exhibit such behavior. The mathematical underpinnings of our theory and algorithms are based on an operator-theoretic approach in which the dynamics of the system are embedded into an infinite-dimensional space.
The dynamical switching from global to local prediction algorithm enabled a successful prediction of influenza cases. We show that the framework correctly identifies the 2009-2010 flu pandemic as a Black Swan event, that prevented machine learning-based algorithms from showing subsequent good performance.
The world has recently experienced a Black swan event that led to the COVID-19 pandemic. We deployed our algorithm to assess its evolution. The results show that, despite being capable of capturing the dynamics of the observed cases of the disease locally, in states and counties, the prediction algorithm is robust to perturbations of the available data, induced for example by delays in reporting or sudden increase in cases due to increase in testing capability. This is achieved in an entirely data-driven fashion, with no underlying mathematical model of the disease.
We discuss the prediction problem in other complex dynamics datasets, such as signature indices of geomagnetic substorms. In addition, fundamental limits on predictability that our theory implies are discussed.