A Neural Network Architecture with an Attention-based Layer for Spatial Prediction of Fine Particulate Matter

Abstract

Several epidemiological studies indicate that fine particulate matter $PM_{2.5}$ affect human health, provoking cardiovascular and respiratory diseases, among other. It is therefore important to assess the spatial distribution of this pollutant. Air quality monitoring (AQM) networks are used to this end. However, they are usually spatially sparse due to their high costs, leaving large areas without monitoring. Numerical models have traditionally been proposed to infer the spatial distribution of air pollutants by simulating the diffusion and reaction process of air pollutants. However, such models usually need highly precise emission data and high-end computing hardware. In this paper, we propose a novel neural network architecture for $PM_{2.5}$ spatial estimation. This model uses a recently proposed attention layer to build an structured graph of the AQM stations (nodes) and to weight the k nearest neighbors for certain nodes based on attention kernels. The learned attention layer can generate a transformed feature representation for a testing node, which is further processed by a fully connected neural network (FCNN) to infer the pollutant concentration. Results on data from Sao Paulo AQM network showed that our approach has better predictive performance than classical methods like Inverse Distance Weighting (IDW), Ordinary Kriging (OK), and FCNN without attention layer, according to different performance metrics. Additionally, the normalized attention weights computed by our model showed that in some cases, the attention given to the nearest nodes is independent of their spatial distances. This shows that the model is more flexible, since it can learn to interpolate $PM_{2.5}$ concentration levels based on the available data of the AQM network and some context information. As for this information we supply to the model different variables like vegetation index (NDVI), surface elevation data, Nighttime Lights (NTL) information and meteorological information.