Energy-Efficient Narrative-Aware Climate Claim Verification with Calibrated Evidence Ranking and Selective Abstention

Abstract

Climate claims circulating in public discourse often compress complex scientific findings into categorical statements, making evidence selection and uncertainty communication central to reliable automated verification. This study evaluates a resource-conscious pipeline on ClimateCheck 2025, comprising 3,048 human-annotated claim–abstract pairs. The experiments address closed-pool evidence reranking and three-way classification as Supports, Refutes, or Not Enough Information. Comparators include BM25, TF–IDF cosine similarity, latent semantic analysis, a quantized 135M-parameter causal language model, a character n-gram classifier, sparse logistic regression, and lexical–semantic models with and without cross-fitted predicted narrative probabilities. Claim-grouped cross-fitting prevents identifier leakage, while temperature scaling and training-locked abstention screening address uncertainty. On 1,904 held-out pairs, Word+Char TF–IDF Logistic led ranking with mean average precision 0.670; LSA-128, the narrative-aware reranker, BM25, and the compact causal-LM hybrid obtained 0.665, 0.657, 0.649, and 0.647, respectively. The narrative-aware verifier produced the highest macro-F1, 0.394, compared with 0.392 for Word+Char TF–IDF Logistic, although the paired difference was not significant after Holm adjustment. Narrative-conditioned calibration reduced adaptive expected calibration error from 0.035 under global scaling to 0.025 and produced an area under the risk–coverage curve of 0.551. Neither prespecified target-risk screen yielded a feasible abstention threshold. Warm narrative-aware inference used 2.798 CPU seconds per 1,000 pairs, corresponding to a standardized 15 W CPU-time proxy of 1.166×10⁻⁵ kWh. The results establish a measured local benchmark in which ranking, verification, probability quality, and computational cost remain separately interpretable.