Transformer-Based Models for Short-to-Medium Range Tropical Cyclone Track and Intensity Forecasting

Abstract

This study presents a systematic benchmark of six Transformer-based architectures, Vanilla Transformer, Informer, Autoformer, FEDformer, PatchTST, and iTransformer, for short-to-medium range tropical cyclone (TC) track and intensity forecasting over the Southern Indian Ocean basins surrounding Indonesia. Using a 3-hourly resampled IBTrACS dataset during (1965-2025), multi-lead-time supervised sequences were constructed with a 72-hour (24-step) input window across six forecast horizons (12–72 hours). All models were implemented under a unified dual-output framework with identical hyperparameters and training conditions to ensure fair comparison. Performance was evaluated using MAE, RMSE, MAPE, R², and categorical skill scores at positional thresholds of 50, 100, and 200 km. A Weighted Sum Model (WSM) composite ranking identified iTransformer as the top performer (WSM = 0.075), followed by PatchTST (0.092) and Autoformer (0.105). Horizon-stratified analysis shows Autoformer excels at short lead times (≤24 h), while iTransformer dominates from 36 to 72 hours. FEDformer ranked last, highlighting limitations of frequency-domain mixing on sparse kinematic data.   Keywords - Tropical Cyclone Forecasting; Transformer Architecture; IBTrACS; Weighted Sum Model; Deep Learning.