Research output: Contribution to journal › Article › peer-review
Research output: Contribution to journal › Article › peer-review
}
TY - JOUR
T1 - A permutation approach to evaluating the performance of a forecasting model of methane content in the atmospheric surface layer of arctic region
AU - Sergeev, Aleksandr
AU - Shichkin, Andrey
AU - Baglaeva, Elena
AU - Buevich, Alexander
AU - Butorova, Anastasia
PY - 2024/2/1
Y1 - 2024/2/1
N2 - The time series forecasting models of climate and ecological systems have grown in number, variety and complexity over the past years. In our work, we proposed to use the ideology of testing statistical hypotheses to evaluate the performance of a predictive time series model. The data for this study were obtained during July–August 2017 from the surface concentration dynamics monitoring of the main greenhouse gases at the Arctic Island of Belyy, Yamalo-Nenets Autonomous Okrug, Russia. Models based on the two types of artificial neural networks - simple recurrent Elman network and nonlinear autoregressive model with exogenous input (NARX), predicted changes in methane concentration. Totally, six models were considered: Elman, NARX and their hybrid model using Daubechies wavelet (db4) and Symlet wavelet (sym4) transformed data for training. The performance of the models was evaluated using 12 ″traditional” indices (correlation and coefficient of determination, Wilmott agreement indices, the mean absolute and mean square errors, etc.) and the proposed permutation approach using the ideology of testing statistical hypotheses. In general, on the data under study, the estimates of the permutational approach and other accuracy indicators and error are the same, but there are some discrepancies. Effect sizes do not always determine the statistical significance of differences. Different artificial neural networks forecasting models are tuned to capture different patterns in the time series of greenhouse gases in the atmosphere. The permutational approach is useful in determining the best model for prediction. So, to predict the variability of methane content, the hybrid model autoregressive neural networks NARX (sym4) is the best performing (means difference = −0.00373 ppm; p-value = 0.01327; correlation coefficient = 0.83320; p-value = 0.00000), and to predict the average concentration of methane content for the entire forecast period, the recurrent neural network Elman (sym4) model is the best (means difference = 0.00241 ppm; p-value = 0.17758; correlation coefficient = 0.50388; p-value = 0.01986).
AB - The time series forecasting models of climate and ecological systems have grown in number, variety and complexity over the past years. In our work, we proposed to use the ideology of testing statistical hypotheses to evaluate the performance of a predictive time series model. The data for this study were obtained during July–August 2017 from the surface concentration dynamics monitoring of the main greenhouse gases at the Arctic Island of Belyy, Yamalo-Nenets Autonomous Okrug, Russia. Models based on the two types of artificial neural networks - simple recurrent Elman network and nonlinear autoregressive model with exogenous input (NARX), predicted changes in methane concentration. Totally, six models were considered: Elman, NARX and their hybrid model using Daubechies wavelet (db4) and Symlet wavelet (sym4) transformed data for training. The performance of the models was evaluated using 12 ″traditional” indices (correlation and coefficient of determination, Wilmott agreement indices, the mean absolute and mean square errors, etc.) and the proposed permutation approach using the ideology of testing statistical hypotheses. In general, on the data under study, the estimates of the permutational approach and other accuracy indicators and error are the same, but there are some discrepancies. Effect sizes do not always determine the statistical significance of differences. Different artificial neural networks forecasting models are tuned to capture different patterns in the time series of greenhouse gases in the atmosphere. The permutational approach is useful in determining the best model for prediction. So, to predict the variability of methane content, the hybrid model autoregressive neural networks NARX (sym4) is the best performing (means difference = −0.00373 ppm; p-value = 0.01327; correlation coefficient = 0.83320; p-value = 0.00000), and to predict the average concentration of methane content for the entire forecast period, the recurrent neural network Elman (sym4) model is the best (means difference = 0.00241 ppm; p-value = 0.17758; correlation coefficient = 0.50388; p-value = 0.01986).
UR - http://www.scopus.com/inward/record.url?partnerID=8YFLogxK&scp=85178576964
UR - https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcAuth=tsmetrics&SrcApp=tsm_test&DestApp=WOS_CPL&DestLinkType=FullRecord&KeyUT=001128894800001
U2 - 10.1016/j.apr.2023.102000
DO - 10.1016/j.apr.2023.102000
M3 - Article
VL - 15
JO - Atmospheric Pollution Research
JF - Atmospheric Pollution Research
SN - 1309-1042
IS - 2
M1 - 102000
ER -
ID: 49820671