Cost-Aware Tool-Routing Agents for Verifiable Agricultural Decision Support: An Offline Replay Benchmark
DOI:
https://doi.org/10.61424/d5mjap37Keywords:
Agricultural decision support, tool routing, language models, evidence verification, cost-aware execution, offline replay, selective predictionAbstract
Agricultural assistants must choose data and analytical tools without confusing source coverage, prediction, and verified evidence. We evaluated a constrained routing workflow using South Asian soil grids, Ethiopian cereal statistics, World Development Indicators, a flood archive, and the ND-GAIN composite index. The benchmark comprised 550 held-out requests: 420 supported queries, 70 unavailable-record probes, and 60 out-of-scope requests. Fixed execution, keyword rules, a supervised TF–IDF router, a selective variant, and a 77-million-parameter FLAN-T5-small router were compared under four operation budgets. At budget four, rules completed all supported tasks, TF–IDF completed 95.0%, and constrained FLAN-T5-small completed 38.6%. Rules used 1.15 source calls per request, compared with six for unrestricted fixed execution. Schema and identity checks rejected all 420 real-record substitution challenges, while removing verification produced incorrect accepted responses. A separate 284-observation forecasting test found that persistence achieved the lowest mean absolute error, 322.51 kg/ha. These results locate the main benefit in explicit evidence scope and efficient execution; they do not establish a general advantage for language-model routing. The evaluation concerns templated offline requests with resolved arguments, rather than autonomous field decisions.
References
Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4), 494–591. https://doi.org/10.1561/2200000101
Chen, A., & Han, L. (2024). Small-sample tree-ensemble models for climate-smart crop selection in low-resource Ethiopian agriculture. Journal of Advanced Computing Systems, 4(6), 104–117. https://doi.org/10.69987/JACS.2024.40608
Chen, C., Noble, I., Hellmann, J., Coffee, J., Murillo, M., & Chawla, N. (2023). University of Notre Dame Global Adaptation Initiative: Country index technical report. University of Notre Dame. https://gain.nd.edu/assets/522870/nd_gain_countryindextechreport_2023_01.pdf
Chen, L., Zaharia, M., & Zou, J. (2024). FrugalGPT: How to use large language models while reducing cost and improving performance. Transactions on Machine Learning Research. https://openreview.net/pdf/3bc37598b9fc1a12afe49c84e43e6beb9d8e2c6a.pdf
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., Webson, A., Gu, S. S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Castro-Ros, A., Pellat, M., Robinson, K., … Wei, J. (2024). Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70), 1–53. https://www.jmlr.org/papers/v25/23-0870.html
Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., & Neubig, G. (2023). PAL: Program-aided language models. Proceedings of Machine Learning Research, 202, 10764–10799. https://proceedings.mlr.press/v202/gao23f.html
Geifman, Y., & El-Yaniv, R. (2017). Selective classification for deep neural networks. Advances in Neural Information Processing Systems, 30, 4878–4887. https://papers.nips.cc/paper_files/paper/2017/hash/4a8423d5e91fda00bb7e46540e2b0cf1-Abstract.html
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2024). CRITIC: Large language models can self-correct with tool-interactive critiquing. In International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2024/hash/fef126561bbf9d4467dbb8d27334b8fe-Abstract-Conference.html
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html
Han, L., & Chen, A. (2024). Climate-resilience and cereal-yield joint forecasting for low- and middle-income countries with public data and grounded LLM policy explanation. Journal of Advanced Computing Systems, 4(1), 121–139. https://doi.org/10.69987/JACS.2024.40109
Han, L., Nie, N., & Kim, P. (2026). Few-shot subnational crop portfolio ranking in Ethiopia with prior-fitted tabular models, spatial soil constraints, conformal decision sets, and evidence-locked explanations. Smart and Intelligent Engineering Systems, 1(1), 55–71. https://atlaspubs.com/index.php/SIES/article/view/34
Han, L., & Zhou, D. (2023a). Lightweight multimodal rice leaf disease screening and Nepal-localized climate-smart advisory support: Evaluations on real rice images, Nepal SOTER soil profiles, and WorldClim climate data. Journal of Advanced Computing Systems, 3(4), 67–82. https://doi.org/10.69987/JACS.2023.30405
Han, L., & Zhou, D. (2023b). Nepal-localized climate-smart crop advisory from crop-recommendation, cereal-yield, climate, boundary, and soil-parent data: A small-data evaluation. Journal of Advanced Computing Systems, 3(7), 58–73. https://doi.org/10.69987/JACS.2023.30705
Han, L., & Zhou, D. (2025a). Domain-shift-aware climate-smart crop portfolio advisory for Nepal with self-supervised tabular learning and conformal shortlists. Journal of Technology Informatics and Engineering, 4(2), 534–558. https://doi.org/10.51903/jtie.v4i2.592
Han, L., & Zhou, D. (2025b). Lightweight image–text triage for rice leaf diseases with clarify-or-abstain decisions and Nepal-context advisory support. Journal of Technology Informatics and Engineering, 4(1), 347–370. https://doi.org/10.51903/jtie.v4i1.593
He, S., Tu, H., & Liu, I. (2023). Safe PD capacity forecasting with time-series foundation models and calibrated uncertainty for heterogeneous GPU clusters. Journal of Advanced Computing Systems, 3(4), 48–66. https://doi.org/10.69987/JACS.2023.30404
Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9), Article 100804. https://doi.org/10.1016/j.patter.2023.100804
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. https://papers.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
Li, M., Zhao, Y., Yu, B., Song, F., Li, H., Yu, H., Li, Z., Huang, F., & Li, Y. (2023). API-Bank: A comprehensive benchmark for tool-augmented LLMs. In H. Bouamor, J. Pino, & K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 3102–3116). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.187
Liakos, K. G., Busato, P., Moshou, D., Pearson, S., & Bochtis, D. (2018). Machine learning in agriculture: A review. Sensors, 18(8), Article 2674. https://doi.org/10.3390/s18082674
Liu, E., & Han, L. (2025). From field geometry to jewelry interfaces: Multimodal AI-assisted shape translation and evidence-grounded agricultural storytelling. International Journal of Graphic Design, 3(1), 218–240. https://doi.org/10.51903/ijgd.v3i1.4276
Ma, R., & Zhang, L. (2026). Incident-aligned probabilistic graph traffic forecasting for DOT corridor operations: Full METR-LA/PEMS-BAY benchmarking, FT-AED early event detection, and verifiable LLM-RAG briefings. Global Sustainability & Development, 1(2), 95–111. https://atlaspubs.com/index.php/GSD/article/view/26
Ma, R., Zhang, L., & Bai, A. (2026). Cross-dataset parcel workload priors for last-mile capacity forecasting: Integrating package segmentation with delivery operation signals. Journal of Technology Informatics and Engineering, 5(1), 441–473. https://doi.org/10.51903/jtie.v5i1.564
Ma, R., Zhang, L., & Song, T. (2025). Computer-vision-informed visual explanation cards for autonomous-driving traffic-sign alerts: Localization, classification, and retrieved evidence on GTSDB. International Journal of Graphic Design, 3(2), 455–473. https://doi.org/10.51903/ijgd.v3i2.3991
Ong, I., Almahairi, A., Wu, V., Chiang, W.-L., Wu, T., Gonzalez, J. E., Kadous, M. W., & Stoica, I. (2024). RouteLLM: Learning to route LLMs with preference data (Version 1) [Preprint]. arXiv. https://arxiv.org/abs/2406.18665v1
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12(85), 2825–2830. https://www.jmlr.org/papers/v12/pedregosa11a.html
Poggio, L., de Sousa, L. M., Batjes, N. H., Heuvelink, G. B. M., Kempen, B., Ribeiro, E., & Rossiter, D. (2021). SoilGrids 2.0: Producing soil information for the globe with quantified spatial uncertainty. SOIL, 7(1), 217–240. https://doi.org/10.5194/soil-7-217-2021
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., Zhao, S., Hong, L., Tian, R., Xie, R., Zhou, J., Gerstein, M., Li, D., Liu, Z., & Sun, M. (2024). ToolLLM: Facilitating large language models to master 16000+ real-world APIs. In International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2024/hash/28e50ee5b72e90b50e7196fde8ea260e-Abstract-Conference.html
Roberts, D. R., Bahn, V., Ciuti, S., Boyce, M. S., Elith, J., Guillera-Arroita, G., Hauenstein, S., Lahoz-Monfort, J. J., Schröder, B., Thuiller, W., Warton, D. I., Wintle, B. A., Hartig, F., & Dormann, C. F. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography, 40(8), 913–929. https://doi.org/10.1111/ecog.02881
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 68539–68551. https://doi.org/10.52202/075280-2997
Tellman, B., Sullivan, J. A., Kuhn, C., Kettner, A. J., Doyle, C. S., Brakenridge, G. R., Erickson, T. A., & Slayback, D. A. (2021). Satellite imaging reveals increased proportion of population exposed to floods. Nature, 596(7870), 80–86. https://doi.org/10.1038/s41586-021-03695-w
Tu, H., Zhao, S., & He, S. (2024). LLM-augmented salable GPU supply forecasting for disaggregated recommendation serving: Predicting instance readiness, scheduling delay, and capacity risk. Journal of Advanced Computing Systems, 4(9), 85–102. https://doi.org/10.69987/JACS.2024.40908
Tu, H., Zhao, S., & Zhou, A. (2025). Visual brief cards for advertising design: A structured UI/UX framework for turning creative intentions into graphic design decisions. International Journal of Graphic Design, 3(1), 210–226. https://doi.org/10.51903/ijgd.v3i1.3714
Wang, E., & Wang, H. (2023). QoE-driven reinforcement learning for joint bitrate, rebuffering, and TTFF optimization in HLS/DASH. Journal of Advanced Computing Systems, 3(2), 50–59. https://doi.org/10.69987/JACS.2023.30204
Wang, H., Li, S., Li, C., Wang, Z., & Peng, H. (2020). Low complexity hybrid beamforming methods for single-user millimeter wave MIMO systems. The Frontiers of Society, Science and Technology, 2(12), 16–29. https://francis-press.com/papers/2548
Wang, H., Ren, Y., & Chang, X. (2025). Layout-aware progressive PDF rendering: AI prioritization of PDF slices to reduce time-to-functional-first-frame on FUNSD. Journal of Technology Informatics and Engineering, 4(2), 425–446. https://doi.org/10.51903/jtie.v4i2.523
Wang, H., & Wang, S. (2026). Query-conditioned document-VLM streaming for time-to-first-correct-answer: Reinforcement learning over progressive PDF tiles. Global Sustainability & Development, 1(2), 65–81. https://atlaspubs.com/index.php/GSD/article/view/23
Wang, H., & Wu, P. (2024). Content-adaptive codec selection between AV1 and H.264: A controlled case study of bandwidth savings versus latency at matched quality. Journal of Advanced Computing Systems, 4(7), 83–92. https://doi.org/10.69987/JACS.2024.40707
Wen, Z. (2025). Research on a deep learning-based dynamic credit risk assessment model for enterprises. In Proceedings of the 2024 5th International Conference on Big Data Economy and Information Management (pp. 389–393). Association for Computing Machinery. https://doi.org/10.1145/3724154.3724218
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations. https://arxiv.org/abs/2210.03629
Zhang, L., & Ma, R. (2026). Calibrated spatiotemporal GraphRAG for urban road safety: Network-constrained severe-crash hotspot forecasting, evidence-grounded countermeasure ranking, and risk-controlled abstention. Smart and Intelligent Engineering Systems, 1(1), 36–54. https://atlaspubs.com/index.php/SIES/article/view/30
Zhang, L., Ma, R., & Greg, P. (2025). Digital-twin dispatching for urban mobility via spatio-temporal transformers and offline reinforcement learning. Journal of Technology Informatics and Engineering, 4(2), 337–363. https://doi.org/10.51903/jtie.v4i2.501
Zhang, L., & Zhang, E. (2024). LLM-explained graph traffic forecasting for DOT corridor operations: Full empirical evaluation on METR-LA and PEMS-BAY with crash evidence. Journal of Advanced Computing Systems, 4(5), 135–145. https://doi.org/10.69987/JACS.2024.40510
Zhong, Z. S., Zhang, L., & Ma, D. (2023). Spatial RAG for urban crash hotspot discovery and safety countermeasure recommendation. Journal of Advanced Computing Systems, 3(12), 45–61. https://doi.org/10.69987/JACS.2023.31206
Zhou, B., Wang, H., & Chang, X. (2025). Distilling VMAF into an edge-deployable quality predictor: A pilot shot-level proxy with LLM-ready quality tokens. Journal of Technology Informatics and Engineering, 4(2), 447–463. https://doi.org/10.51903/jtie.v4i2.522
Zhou, H., & Han, L. (2026). Flood-shock-aware climate-resilience and cereal-yield forecasting for low- and middle-income countries with cross-series pretrained time-series models, conformal downside risk, and evidence-locked LLM-compatible policy explanations. Global Sustainability & Development, 1(2), 124–141. https://atlaspubs.com/index.php/GSD/article/view/32
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Kevin Zhang, Yuxin Chen, Daniel Wu (Author)

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.