Discovering Time-Series Building Blocks Using the ‘Significance Engine’

Abstract

This paper presents a time-series prediction framework called the “Significance Engine” which contributes to the area of time-series prediction by presenting a novel, noise resistant mechanism that combines together the techniques of Takens' Theorem and dynamic self-organising maps. It is built around the idea that time-series can be broken down into both random and non-random components and that the non-random components can be used to help predict future time-series movement. It is a highly parallel system which combines together multiple Dynamic Self-Organising Map units of the ‘Grow When Required’ type and time delay embedding techniques. It learns iteratively, so the more data the system is fed from the input distribution, the better the predictions it is able to make. After the framework has been initialised and ideally primed using historical data, it is able to recognise re-occurring patterns in highly noisy waveforms and to successfully make future predictions based on historically what occurred when these motifs were previously observed.

Keywords

Self-organising neural network delay embedding time-series prediction

References

Bollerslev

, Generalized Autoregressive Conditional Heteroskedasticity, Journal of Econometrics, 1986, 31, 307–327.

Box

G. E. P.

Jenkins

G.M.

, Time Series Analysis: Forecasting and Control, 2nd edition, Holden Day, San Francisco, CA, 1976.

Engle

, Autorregressive Conditional Heteroskedasticity with Estimates of United Kingdom Inflation, Econometrica, 1982, 50, 987–1008.

Hamilton

J. D.

, Time Series Analysis, Princeton University Press, 1994.

Murphy

J. P.

, Technical Analysis of the Financial Markets, New York Institute of Finance, 1999.

Keogh

Lonardi

Chiu

, Finding Surprising Patterns in a Time Series Database in Linear Time and Space, International Conference on Knowledge Discovery and Data Mining, 2002, 23–26.

Patel

Keogh

Lin

Lonardi

, Mining Motifs in Massive Time Series Databases, Proceedings of the IEEE International Conference on Data Mining, 2002.

Kohonen

, Self-Organized formation of topologically correct feature maps, Biological Cybernetics, 1982, 43.

Kohonen

, Self-Organizing Maps, 2nd edition, Springer-Verlag, Berlin, Germany, 1997.

10.

Marsland

Shapiro

Nehmzow

, A self-organising network that grows when required, Neural Networks, Special Issue 2002, 15, 1041–1058.

11.

Takens

, Detecting strange attractors in turbulence, Dynamical Systems and Turbulence, Springer-Verlag, Lecture Notes in Mathematics, 1981, 898, 366–381.

12.

Povinelli

R. J.

, Identifying Temporal Patterns for Characterization and Prediction of Financial Time-Series Events, PhD dissertation, Department of Electrical and Computer Engineering, Marquette University, Milwaukee, USA, 1999.

13.

Laerhoven

K. V.

, Combining the Self-Organizing Map and K-Means Clustering for On-line Classification of Sensor Data, Proceedings of the International Conference on Artificial Neural Networks, 2001, 2130, 464–469.

14.

MacQueen

, On convergence of k-means and partitions with minimum average variance, Annals of the Institute of Statistical Mathematics, 1965, 36, 1084.

15.

Malsburg

, Self-organization of orientation sensitive cells in the striate cortex, Springer-Verlag, Kybernetic, 1973, 14, 85–100.

16.

Grossberg

, Adaptive pattern classification and universal recording, 1: Parallel development and coding of neural feature detectors. Biological Cybernetics, 1976, 23, 121–134.

17.

Martinetz

T.M.

Schulten

K. J.

, Topology representing networks, Neural Networks 1994, 3, 507–522.

18.

Fritzke

, Unsupervised ontogenic networks, Handbook of Neural Computation, Institute of Physics Publishing and Oxford University Press 1997, 2.4, 1–16.

19.

Martinetz

T. M.

Berkovich

S. G.

Schulten

K. J.

, Neural-gas network for vector quantization and its application to time series prediction. IEEE Transactions on Neural Networks, 1993, 4, 558–569.

20.

Fritzke

, Growing grid: A self-organizing network with constant neighbourhood range and adaptation strength, Neural Processing Letters, 1995, 2, 5.

21.

Merkl

S. H.

Dittenbach

Rauber

, Adaptive Hierarchical Incremental Grid Growing: An architecture for high-dimensional data visualization, Proceedings of the Workshop on Self-Organizing Maps, 2003, 293–298.

22.

Fritzke

, Growing cell structures: A self-organizing network for unsupervised and supervised learning, Neural Networks, 1994, 7, 9.

23.

Hodge

Austin

, Hierarchical Growing Cell Structures: TreeGCS, IEEE Transactions on Knowledge and Data Engineering, 2001, 13, 2.

24.

Whitney

, Differentiable manifolds, Annals of Mathematics, 1936, 37(3), 645–680.