Self-Supervised Climate Model Downscaling Using Temporal-Spatial Transformer Networks

Abstract

Global Climate Models (GCMs) offer valuable insights into large-scale climate dynamics but suffer from coarse spatial resolution, typically ranging from 100 to 250 km. This limitation makes them unsuitable for regional or local-scale climate impact assessments, especially in topographically complex or data-scarce regions. This study aims to develop a self-supervised learning framework for climate model downscaling that captures both spatial and temporal dependencies while minimizing reliance on labeled high-resolution datasets. We propose a Temporal-Spatial Transformer Network (TSTN) trained using a masked token modeling strategy to learn from unlabeled low-resolution climate data. The architecture incorporates both temporal and spatial attention mechanisms to extract long-range dependencies across time and space. The model is evaluated on ERA5 and CMIP6 datasets using three key metrics: Root Mean Square Error (RMSE), Pearson Correlation Coefficient (PCC), and Skill Score (SS). Baseline comparisons include BCSD, CNN, and U-Net models. The proposed TSTN achieved an RMSE of 1.92, a PCC of 0.92, and a Skill Score of 0.84 on the held-out test set, outperforming all baseline models by significant margins. For example, RMSE was reduced by 17% compared to U-Net (2.30), and PCC improved by 4.5% over CNN (0.88). A paired t-test confirmed the statistical significance of these improvements with a p-value of 0.0007. This research demonstrates that self-supervised transformer-based architectures can effectively downscale climate data while reducing dependency on labeled observations. The model offers a scalable, generalizable solution for producing high-resolution climate projections, particularly in regions where traditional methods are limited by data availability or computational cost