Change-Point-Aware Probabilistic Forecasting of Bursty LLM Demand with Conformal Intervals and Tail-Risk Control

Abstract

Short-horizon demand forecasts can support capacity reservation for large language model (LLM) services, but burstiness and regime change make both point accuracy and uncertainty calibration difficult. This study evaluates causal one-step-ahead forecasts on 1,429,737 BurstGPT records aggregated into 17,566 complete five-minute intervals. A chronological design used 12,384 intervals for training, 2,592 for conformal calibration, and 2,590 for held-out testing. Persistence, daily seasonal-naive, ARIMA, histogram gradient boosting (HGB), change-point-augmented HGB (CP-HGB), and a compact causal Transformer were compared. Raw quantiles, split conformalized quantile regression, rolling calibration, adaptive conformal inference (ACI), and a change-point-aware ACI reset (CP-ACI) were evaluated with pinball loss, coverage, width, Winkler score, capacity exceedance, and conditional value-at-risk (CVaR). The CP-HGB conditional median achieved the lowest point error, with an MAE of 17,912 tokens per interval and a 4.29% reduction relative to persistence. The compact Transformer’s raw 90% interval covered 83.09% of test outcomes; CP-ACI increased coverage to 90.12%. For a one-sided 95% CP-HGB capacity bound, CP-ACI attained 95.10% nonexceedance and reduced shortfall CVaR from 47,100 under ACI to 45,632 tokens, while increasing mean unused headroom from 59,126 to 60,306 tokens. Change resets were therefore most useful for correcting undercoverage and limiting upper-tail shortfall, whereas CP-HGB’s already conservative central interval gained little. The results support joint reporting of point error, calibration, sharpness, and tail risk when token workload is used as a capacity-planning proxy.