Sage Journals: Discover world-class research

Abstract

In this paper, a new data-driven method for short-range forecasting of spatio-temporal systems is proposed. It uses NCEP data as raw data to construct forecasting model. The global model consists of several local models. Each local model is constructed in three steps. In the first step, a local dataset is constructed based on NCEP raw data. This dataset is a very high-dimensional data with huge number of redundant and irrelevant features. In the second step, a feature selection method named GRASP is applied on the local dataset and produces a new local dataset whose features are reduced significantly. In the third step, a regression ensemble method called Bagging is used to construct a local model. Both GRASP and Bagging methods are scalable modules with respect to the computational power needed. The proposed method makes it possible to control the trade-off between speed and precision. In addition to the scalability, the proposed method, in some points produces forecasts more precise than the GFS system.

Keywords

Feature selection regression ensemble spatio-temporal modeling data driven modeling Numerical Weather Prediction

Get full access to this article

View all access options for this article.

References

Anastasio

T.J.

, Data-driven modeling of Alzheimer disease pathogenesis, J Theor Biol 290 (2011), 60–72.

Antoch

and Hlubinka

, Data driven modelling of vertical atmospheric radiation, J Environ Radioact 102 (2011), 1085–1095.

Badin

A.S.

Eraifej

and Greenfield

, High-resolution spatio-temporal bioactivity of a novel peptide revealed by optical imaging in rat orbitofrontal cortex in vitro: Possible implications for neurodegenerative diseases, Neuropharmacology 73 (2013), 10–18.

Baker

Lake

R.T.

Rivet

Benston

Bommersbach

and Kirk

, Point-trained models in a grid environment: Transforming a potato late blight risk forecast for use with the national digital forecast database, Computers and Electronics in Agriculture 105 (2014), 1–8.

Bermejo

Gámez

J.A.

and Puerta

J.M.

, A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets, Pattern Recognition Letters 32 (2011), 701–711.

Breiman

, Bagging predictors, Machine Learning 24 (1996), 123–140.

Buizza

Leutbecher

and Isaksen

, Potential use of an ensemble of analyses in the ECMWF ensemble prediction system, Quarterly Journal of the Royal Meteorological Society 134 (2008), 2051–2066.

Chakravarty

Mukhopadhyay

and Taraphdar

, Cloud microphysical properties as revealed by the CAIPEEX and satellite observations and evaluation of a cloud system resolving model simulation of contrasting large scale environments, Journal of Atmospheric and Solar-Terrestrial Physics 73 (2011), 1790–1797.

Chang

H.K.

and Chien

W.A.

, Neural network with multi-trend simulating transfer function for forecasting typhoon wave, Advances in Engineering Software 37 (2006), 184–194.

10.

Coelho

A.L.V.

and Nascimento

D.S.C.

, On the evolutionary design of heterogeneous Bagging models, Neurocomputing 73 (2010), 3319–3322.

11.

Cuevas

and García

, Improved background modeling for real-time spatio-temporal non-parametric moving object detection strategies, Image and Vision Computing 31 (2013), 616–630.

12.

Dash

and Ignatov

, Validation of clear-sky radiances over oceans simulated with MODTRAN4.2 and global NCEP GDAS fields against nighttime NOAA15–18 and MetOp-A AVHRR data, Remote Sensing of Environment 112 (2008), 3012–3029.

13.

Dong

Gong

Valdes-Sosa

P.A.

Xia

Luo

and Yao

, Simultaneous EEG-fMRI: Trial level spatio-temporal fusion for hierarchically reliable information discovery, Neuro Image 99 (2014), 28–41.

14.

Dubé

Thériault

and Des Rosiers

, Commuter rail accessibility and house values: The case of the montreal south shore, Canada, 1992–2009, Transportation Research Part A: Policy and Practice 54 (2013), 49–66.

15.

Everaert

Pauwels

I.S.

Boets

Buysschaert

and Goethals

P.L.M.

, Development and assessment of ecological models in the context of the european water framework directive: Key issues for trainers in data-driven modeling approaches, Ecological Informatics 17 (2013), 111–116.

16.

Awal

F.R.

Michaud

Chu

P.S.

Fares

Kodama

and Rosener

, Rainfall-runoff modeling in a flashy tropical watershed using the distributed HL-RDHM model, Journal of Hydrology (2014).

17.

Feo

T.A.

and Resende

M.G.C.

, A probabilistic heuristic for a computationally difficult set covering problem, Operations Research Letters 8 (1989), 67–71.

18.

Feo

T.A.

and Resende

M.G.C.

, Greedy randomized adaptive search procedures, Journal of Global Optimization 6 (1995).

19.

Fernando

H.J.

Mammarella

M.C.

Grandoni

Fedele

Di Marco

Dimitrova

and Hyde

, Forecasting PM10 in metropolitan areas: Efficacy of neural networks, Environ Pollut 163 (2012), 62–67.

20.

Festa

and Resende

M.G.C.

, Effective application of GRASP, Wiley Encyclopedia of Operations Research and Management Sciences 3 (2011), 1609–1617.

21.

Filippo Rebelo Torres

Kjerfve

and Monat

, Application of artificial neural network (ANN) to improve forecasting of sea level, Ocean & Coastal Management 55 (2012), 101–110.

22.

Friedel

M.J.

, Data-driven modeling of surface temperature anomaly and solar activity trends, Environmental Modelling & Software 37 (2012), 217–232.

23.

Gartner, What is big data, In: University of Villanova, 2012.

24.

Golparvar-Fard

Heydarian

and Niebles

J.C.

, Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers, Advanced Engineering Informatics 27 (2013), 652–663.

25.

Y.C.

Chan

P.W.

and Li

Q.S.

, Standardization of raw wind speed data under complex terrain conditions: A data-driven scheme, Journal of Wind Engineering and Industrial Aerodynamics 131 (2014), 12–30.

26.

Hernández-Lobato

Martínez-Muñoz

and Suárez

, Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles, Neurocomputing 74 (2011), 2250–2264.

27.

Hill

D.J.

and Minsker

B.S.

, Anomaly detection in streaming environmental sensor data: A data-driven modeling approach, Environmental Modelling & Software 25 (2010), 1014–1022.

28.

Holly

Pesaran

M.H.

and Yamagata

, The spatial and temporal diffusion of house prices in the UK, Journal of Urban Economics 69 (2011), 2–23.

29.

Hsiao

L.F.

Yang

M.J.

Lee

C.S.

Kuo

H.C.

Shih

D.S.

Tsai

C.C.

Wang

C.J.

Chang

L.Y.

Chen

D.Y.C.

Feng

Hong

J.S.

Fong

C.T.

Chen

D.S.

Yeh

T.C.

Huang

C.Y.

Guo

W.D.

and Lin

G.F.

, Ensemble forecasting of typhoon rainfall and floods over a mountainous watershed in Taiwan, Journal of Hydrology 506 (2013), 55–68.

30.

Judd

, Forecasting with imperfect models, dynamically constrained inverse problems, and gradient descent algorithms, Physica D: Nonlinear Phenomena 237 (2008), 216–232.

31.

Koren

Smith

and Cui

, Physically-based modifications to the sacramento soil moisture accounting model, Part A: Modeling the effects of frozen ground on the runoff generation process, Journal of Hydrology (2014).

32.

Lang

S.T.K.

Leutbecher

and Jones

S.C.

, Impact of perturbation methods in the ECMWF ensemble prediction system on tropical cyclone forecasts, Quarterly Journal of the Royal Meteorological Society 138 (2012), 2030–2046.

33.

Lange

Siemen

Blome

and Thulke

H.H.

, Analysis of spatio-temporal patterns of African swine fever cases in Russian wild boar does not reveal an endemic situation, Preventive Veterinary Medicine (2014).

34.

Lin

Chen

Qiu

Krishnan

and Zou

, LibD3C: Ensemble classifiers with a clustering and dynamic selection strategy, Neurocomputing 123 (2014), 424–435.

35.

Mathiesen

Collier

and Kleissl

, A high-resolution, cloud-assimilating numerical weather prediction model for solar irradiance forecasting, Solar Energy 92 (2013), 47–61.

36.

Mathiesen

and Kleissl

, Evaluation of numerical weather prediction for intra-day solar forecasting in the continental United States, Solar Energy 85 (2011), 967–977.

37.

Milne

and Bennett

, Understanding landscape patterns of temporal variability in avian populations to improve environmental impact assessments, Ecological Informatics 14 (2013), 75–78.

38.

Moscone

Tosetti

and Canepa

, Real estate market and financial stability in US metropolitan areas: A dynamic model with spatial effects, Regional Science and Urban Economics 49 (2014), 129–146.

39.

Moshki

Kabiri

and Mohebalhojeh

, Scalable feature selection in high-dimensional data based on GRASP, Applied Artificial Intelligence 29 (2015), 283–296.

40.

Padilla

C.M.

Kihal-Talantikite

Vieira

V.M.

Rossello

Nir

G.L.

Zmirou-Navier

and Deguen

, Air quality and social deprivation in four French metropolitan areas – A localized spatio-temporal environmental inequality analysis, Environmental Research 134 (2014), 315–324.

41.

Pelly

J.L.

and Hoskins

B.J.

, How well does the ECMWF ensemble prediction system predict blocking? Quarterly Journal of the Royal Meteorological Society 129 (2003), 1683–1702.

42.

Rasheed

Sørli

Holdahl

and Kvamsdal

, A multiscale approach to micrositing of wind turbines, Energy Procedia 14 (2012), 1458–1463.

43.

Resende

M.G.C.

and Ribeiro

C.C.

, Greedy randomized adaptive search procedures: Advances, hybridizations, and applications, Handbook of Metaheuristics International Series in Operations Research & Management Science 146 (2010), 283–319.

44.

Rokach

, Ensemble-based classifiers, Artificial Intelligence Review 33 (2010), 1–39.

45.

Schnase

J.L.

Duffy

D.Q.

Tamkin

G.S.

Nadeau

Thompson

J.H.

Grieg

C.M.

McInerney

M.A.

and Webster

W.P.

, MERRA analytic services: Meeting the big data challenges of climate science through cloud-enabled climate analytics-as-a-service, Computers, Environment and Urban Systems (2014).

46.

Shaddick

and Zidek

J.V.

, A case study in preferential sampling: Long term monitoring of air pollution in the UK, Spatial Statistic 9 (2014), 51–65.

47.

Shih

D.S.

Chen

C.H.

and Yeh

G.T.

, Improving our understanding of flood forecasting using earlier hydro-meteorological intelligence, Journal of Hydrology 512 (2014), 470–481.

48.

Stopa

J.E.

and Cheung

K.F.

, Intercomparison of wind and wave data from the ECMWF reanalysis interim and the NCEP climate forecast system reanalysis, Ocean Modelling 75 (2014), 65–83.

49.

Valverde

M.C.

Araujo

and Campos Velho

, Neural network and fuzzy logic statistical downscaling of atmospheric circulation-type specific weather pattern for rainfall forecasting, Applied Soft Computing 22 (2014), 681–694.

50.

Valverde Ramírez

M.C.

de Campos Velho

H.F.

and Ferreira

N.J.

, Artificial neural network technique for rainfall forecasting applied to the São Paulo region, Journal of Hydrology 301 (2005), 146–162.

51.

Verkade

J.S.

Brown

J.D.

Reggiani

and Weerts

A.H.

, Post-processing ECMWF precipitation and temperature ensemble reforecasts for operational hydrologic forecasting at various spatial scales, Journal of Hydrology 501 (2013), 73–91.

52.

Villarini

Seo

B.C.

Serinaldi

and Krajewski

W.F.

, Spatial and temporal modeling of radar rainfall uncertainties, Atmospheric Research 135–136 (2014), 91–101.

53.

Wei

and Watkins

D.W.

, Data mining methods for hydroclimatic forecasting, Advances in Water Resources 34 (2011), 1390–1400.

54.

Wiegand

and Knippertz

, Equatorward breaking Rossby waves over the north atlantic and mediterranean region in the ECMWF operational ensemble prediction system, Quarterly Journal of the Royal Meteorological Society 140 (2014), 58–71.

55.

WMO, Definitions of meteorological forecasting ranges, in: WMO (2014).

56.

Xie

and Zhu

, Margin distribution based bagging pruning, Neurocomputing 85 (2012), 11–19.

57.

Zhang

Wang

and Wang

, Review on probabilistic forecasting of wind power generation, Renewable and Sustainable Energy Reviews 32 (2014), 255–270.

Scalable data-driven modeling of spatio-temporal systems: Weather forecasting

Abstract

Keywords

Get full access to this article

References