Cost-Aware Tool-Routing Agents for Verifiable Agricultural Decision Support: An Offline Replay Benchmark

Authors

  • Kevin Zhang Author
  • Yuxin Chen Author
  • Daniel Wu Author

DOI:

https://doi.org/10.61424/d5mjap37

Keywords:

Agricultural decision support, tool routing, language models, evidence verification, cost-aware execution, offline replay, selective prediction

Abstract

Agricultural assistants must choose data and analytical tools without confusing source coverage, prediction, and verified evidence. We evaluated a constrained routing workflow using South Asian soil grids, Ethiopian cereal statistics, World Development Indicators, a flood archive, and the ND-GAIN composite index. The benchmark comprised 550 held-out requests: 420 supported queries, 70 unavailable-record probes, and 60 out-of-scope requests. Fixed execution, keyword rules, a supervised TF–IDF router, a selective variant, and a 77-million-parameter FLAN-T5-small router were compared under four operation budgets. At budget four, rules completed all supported tasks, TF–IDF completed 95.0%, and constrained FLAN-T5-small completed 38.6%. Rules used 1.15 source calls per request, compared with six for unrestricted fixed execution. Schema and identity checks rejected all 420 real-record substitution challenges, while removing verification produced incorrect accepted responses. A separate 284-observation forecasting test found that persistence achieved the lowest mean absolute error, 322.51 kg/ha. These results locate the main benefit in explicit evidence scope and efficient execution; they do not establish a general advantage for language-model routing. The evaluation concerns templated offline requests with resolved arguments, rather than autonomous field decisions.

References

Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4), 494–591. https://doi.org/10.1561/2200000101

Chen, A., & Han, L. (2024). Small-sample tree-ensemble models for climate-smart crop selection in low-resource Ethiopian agriculture. Journal of Advanced Computing Systems, 4(6), 104–117. https://doi.org/10.69987/JACS.2024.40608

Chen, C., Noble, I., Hellmann, J., Coffee, J., Murillo, M., & Chawla, N. (2023). University of Notre Dame Global Adaptation Initiative: Country index technical report. University of Notre Dame. https://gain.nd.edu/assets/522870/nd_gain_countryindextechreport_2023_01.pdf

Chen, L., Zaharia, M., & Zou, J. (2024). FrugalGPT: How to use large language models while reducing cost and improving performance. Transactions on Machine Learning Research. https://openreview.net/pdf/3bc37598b9fc1a12afe49c84e43e6beb9d8e2c6a.pdf

Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., Webson, A., Gu, S. S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Castro-Ros, A., Pellat, M., Robinson, K., … Wei, J. (2024). Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70), 1–53. https://www.jmlr.org/papers/v25/23-0870.html

Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., & Neubig, G. (2023). PAL: Program-aided language models. Proceedings of Machine Learning Research, 202, 10764–10799. https://proceedings.mlr.press/v202/gao23f.html

Geifman, Y., & El-Yaniv, R. (2017). Selective classification for deep neural networks. Advances in Neural Information Processing Systems, 30, 4878–4887. https://papers.nips.cc/paper_files/paper/2017/hash/4a8423d5e91fda00bb7e46540e2b0cf1-Abstract.html

Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2024). CRITIC: Large language models can self-correct with tool-interactive critiquing. In International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2024/hash/fef126561bbf9d4467dbb8d27334b8fe-Abstract-Conference.html

Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html

Han, L., & Chen, A. (2024). Climate-resilience and cereal-yield joint forecasting for low- and middle-income countries with public data and grounded LLM policy explanation. Journal of Advanced Computing Systems, 4(1), 121–139. https://doi.org/10.69987/JACS.2024.40109

Han, L., Nie, N., & Kim, P. (2026). Few-shot subnational crop portfolio ranking in Ethiopia with prior-fitted tabular models, spatial soil constraints, conformal decision sets, and evidence-locked explanations. Smart and Intelligent Engineering Systems, 1(1), 55–71. https://atlaspubs.com/index.php/SIES/article/view/34

Han, L., & Zhou, D. (2023a). Lightweight multimodal rice leaf disease screening and Nepal-localized climate-smart advisory support: Evaluations on real rice images, Nepal SOTER soil profiles, and WorldClim climate data. Journal of Advanced Computing Systems, 3(4), 67–82. https://doi.org/10.69987/JACS.2023.30405

Han, L., & Zhou, D. (2023b). Nepal-localized climate-smart crop advisory from crop-recommendation, cereal-yield, climate, boundary, and soil-parent data: A small-data evaluation. Journal of Advanced Computing Systems, 3(7), 58–73. https://doi.org/10.69987/JACS.2023.30705

Han, L., & Zhou, D. (2025a). Domain-shift-aware climate-smart crop portfolio advisory for Nepal with self-supervised tabular learning and conformal shortlists. Journal of Technology Informatics and Engineering, 4(2), 534–558. https://doi.org/10.51903/jtie.v4i2.592

Han, L., & Zhou, D. (2025b). Lightweight image–text triage for rice leaf diseases with clarify-or-abstain decisions and Nepal-context advisory support. Journal of Technology Informatics and Engineering, 4(1), 347–370. https://doi.org/10.51903/jtie.v4i1.593

He, S., Tu, H., & Liu, I. (2023). Safe PD capacity forecasting with time-series foundation models and calibrated uncertainty for heterogeneous GPU clusters. Journal of Advanced Computing Systems, 3(4), 48–66. https://doi.org/10.69987/JACS.2023.30404

Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9), Article 100804. https://doi.org/10.1016/j.patter.2023.100804

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. https://papers.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html

Li, M., Zhao, Y., Yu, B., Song, F., Li, H., Yu, H., Li, Z., Huang, F., & Li, Y. (2023). API-Bank: A comprehensive benchmark for tool-augmented LLMs. In H. Bouamor, J. Pino, & K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 3102–3116). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.187

Liakos, K. G., Busato, P., Moshou, D., Pearson, S., & Bochtis, D. (2018). Machine learning in agriculture: A review. Sensors, 18(8), Article 2674. https://doi.org/10.3390/s18082674

Liu, E., & Han, L. (2025). From field geometry to jewelry interfaces: Multimodal AI-assisted shape translation and evidence-grounded agricultural storytelling. International Journal of Graphic Design, 3(1), 218–240. https://doi.org/10.51903/ijgd.v3i1.4276

Ma, R., & Zhang, L. (2026). Incident-aligned probabilistic graph traffic forecasting for DOT corridor operations: Full METR-LA/PEMS-BAY benchmarking, FT-AED early event detection, and verifiable LLM-RAG briefings. Global Sustainability & Development, 1(2), 95–111. https://atlaspubs.com/index.php/GSD/article/view/26

Ma, R., Zhang, L., & Bai, A. (2026). Cross-dataset parcel workload priors for last-mile capacity forecasting: Integrating package segmentation with delivery operation signals. Journal of Technology Informatics and Engineering, 5(1), 441–473. https://doi.org/10.51903/jtie.v5i1.564

Ma, R., Zhang, L., & Song, T. (2025). Computer-vision-informed visual explanation cards for autonomous-driving traffic-sign alerts: Localization, classification, and retrieved evidence on GTSDB. International Journal of Graphic Design, 3(2), 455–473. https://doi.org/10.51903/ijgd.v3i2.3991

Ong, I., Almahairi, A., Wu, V., Chiang, W.-L., Wu, T., Gonzalez, J. E., Kadous, M. W., & Stoica, I. (2024). RouteLLM: Learning to route LLMs with preference data (Version 1) [Preprint]. arXiv. https://arxiv.org/abs/2406.18665v1

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12(85), 2825–2830. https://www.jmlr.org/papers/v12/pedregosa11a.html

Poggio, L., de Sousa, L. M., Batjes, N. H., Heuvelink, G. B. M., Kempen, B., Ribeiro, E., & Rossiter, D. (2021). SoilGrids 2.0: Producing soil information for the globe with quantified spatial uncertainty. SOIL, 7(1), 217–240. https://doi.org/10.5194/soil-7-217-2021

Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., Zhao, S., Hong, L., Tian, R., Xie, R., Zhou, J., Gerstein, M., Li, D., Liu, Z., & Sun, M. (2024). ToolLLM: Facilitating large language models to master 16000+ real-world APIs. In International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2024/hash/28e50ee5b72e90b50e7196fde8ea260e-Abstract-Conference.html

Roberts, D. R., Bahn, V., Ciuti, S., Boyce, M. S., Elith, J., Guillera-Arroita, G., Hauenstein, S., Lahoz-Monfort, J. J., Schröder, B., Thuiller, W., Warton, D. I., Wintle, B. A., Hartig, F., & Dormann, C. F. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography, 40(8), 913–929. https://doi.org/10.1111/ecog.02881

Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 68539–68551. https://doi.org/10.52202/075280-2997

Tellman, B., Sullivan, J. A., Kuhn, C., Kettner, A. J., Doyle, C. S., Brakenridge, G. R., Erickson, T. A., & Slayback, D. A. (2021). Satellite imaging reveals increased proportion of population exposed to floods. Nature, 596(7870), 80–86. https://doi.org/10.1038/s41586-021-03695-w

Tu, H., Zhao, S., & He, S. (2024). LLM-augmented salable GPU supply forecasting for disaggregated recommendation serving: Predicting instance readiness, scheduling delay, and capacity risk. Journal of Advanced Computing Systems, 4(9), 85–102. https://doi.org/10.69987/JACS.2024.40908

Tu, H., Zhao, S., & Zhou, A. (2025). Visual brief cards for advertising design: A structured UI/UX framework for turning creative intentions into graphic design decisions. International Journal of Graphic Design, 3(1), 210–226. https://doi.org/10.51903/ijgd.v3i1.3714

Wang, E., & Wang, H. (2023). QoE-driven reinforcement learning for joint bitrate, rebuffering, and TTFF optimization in HLS/DASH. Journal of Advanced Computing Systems, 3(2), 50–59. https://doi.org/10.69987/JACS.2023.30204

Wang, H., Li, S., Li, C., Wang, Z., & Peng, H. (2020). Low complexity hybrid beamforming methods for single-user millimeter wave MIMO systems. The Frontiers of Society, Science and Technology, 2(12), 16–29. https://francis-press.com/papers/2548

Wang, H., Ren, Y., & Chang, X. (2025). Layout-aware progressive PDF rendering: AI prioritization of PDF slices to reduce time-to-functional-first-frame on FUNSD. Journal of Technology Informatics and Engineering, 4(2), 425–446. https://doi.org/10.51903/jtie.v4i2.523

Wang, H., & Wang, S. (2026). Query-conditioned document-VLM streaming for time-to-first-correct-answer: Reinforcement learning over progressive PDF tiles. Global Sustainability & Development, 1(2), 65–81. https://atlaspubs.com/index.php/GSD/article/view/23

Wang, H., & Wu, P. (2024). Content-adaptive codec selection between AV1 and H.264: A controlled case study of bandwidth savings versus latency at matched quality. Journal of Advanced Computing Systems, 4(7), 83–92. https://doi.org/10.69987/JACS.2024.40707

Wen, Z. (2025). Research on a deep learning-based dynamic credit risk assessment model for enterprises. In Proceedings of the 2024 5th International Conference on Big Data Economy and Information Management (pp. 389–393). Association for Computing Machinery. https://doi.org/10.1145/3724154.3724218

Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations. https://arxiv.org/abs/2210.03629

Zhang, L., & Ma, R. (2026). Calibrated spatiotemporal GraphRAG for urban road safety: Network-constrained severe-crash hotspot forecasting, evidence-grounded countermeasure ranking, and risk-controlled abstention. Smart and Intelligent Engineering Systems, 1(1), 36–54. https://atlaspubs.com/index.php/SIES/article/view/30

Zhang, L., Ma, R., & Greg, P. (2025). Digital-twin dispatching for urban mobility via spatio-temporal transformers and offline reinforcement learning. Journal of Technology Informatics and Engineering, 4(2), 337–363. https://doi.org/10.51903/jtie.v4i2.501

Zhang, L., & Zhang, E. (2024). LLM-explained graph traffic forecasting for DOT corridor operations: Full empirical evaluation on METR-LA and PEMS-BAY with crash evidence. Journal of Advanced Computing Systems, 4(5), 135–145. https://doi.org/10.69987/JACS.2024.40510

Zhong, Z. S., Zhang, L., & Ma, D. (2023). Spatial RAG for urban crash hotspot discovery and safety countermeasure recommendation. Journal of Advanced Computing Systems, 3(12), 45–61. https://doi.org/10.69987/JACS.2023.31206

Zhou, B., Wang, H., & Chang, X. (2025). Distilling VMAF into an edge-deployable quality predictor: A pilot shot-level proxy with LLM-ready quality tokens. Journal of Technology Informatics and Engineering, 4(2), 447–463. https://doi.org/10.51903/jtie.v4i2.522

Zhou, H., & Han, L. (2026). Flood-shock-aware climate-resilience and cereal-yield forecasting for low- and middle-income countries with cross-series pretrained time-series models, conformal downside risk, and evidence-locked LLM-compatible policy explanations. Global Sustainability & Development, 1(2), 124–141. https://atlaspubs.com/index.php/GSD/article/view/32

Downloads

Published

2026-11-03