2026
1.
Konstantinos I. Roumeliotis; Ranjan Sapkota; Manoj Karkee; Nikolaos D. Tselikas
Agentic AI With Orchestrator-Agent Trust: A Modular Visual Classification Framework With Trust-Aware Orchestration and RAG-Based Reasoning Journal Article
In: IEEE Access, vol. 14, pp. 26965–26982, 2026, ISSN: 2169-3536.
Abstract | Links | BibTeX | Tags: Accuracy, Adaptation models, Agentic AI, Artificial intelligence, Calibration, Cognition, Costs, orchestrator agent trust, Retrieval augmented generation, retrieval augmented reasoning, Training, trust orchestration, visual classification, Visualization
@article{roumeliotis_agentic_2026,
title = {Agentic AI With Orchestrator-Agent Trust: A Modular Visual Classification Framework With Trust-Aware Orchestration and RAG-Based Reasoning},
author = {Konstantinos I. Roumeliotis and Ranjan Sapkota and Manoj Karkee and Nikolaos D. Tselikas},
url = {https://ieeexplore.ieee.org/document/11373381/},
doi = {10.1109/ACCESS.2026.3662282},
issn = {2169-3536},
year = {2026},
date = {2026-01-01},
urldate = {2026-01-01},
journal = {IEEE Access},
volume = {14},
pages = {26965\textendash26982},
abstract = {Modern Artificial Intelligence (AI) increasingly relies on multi-agent architectures that blend visual and language understanding. Yet, a pressing challenge remains: How can we trust these agents especially in zero-shot settings with no fine-tuning? We introduce a novel modular Agentic AI visual classification framework that integrates generalist multimodal agents with a non-visual reasoning orchestrator and a Retrieval-Augmented Generation (RAG) module. Applied to apple leaf disease diagnosis, we benchmark three configurations: (I) zero-shot with confidence-based orchestration, (II) fine-tuned agents with improved performance, and (III) trust-calibrated orchestration enhanced by CLIP-based image retrieval and re-evaluation loops. Using confidence calibration metrics (ECE, OCR, CCC), the orchestrator modulates trust across agents. Our results demonstrate a 77.94% accuracy improvement in the zero-shot setting using trust-aware orchestration and RAG, achieving 85.63% overall. GPT-4o showed better calibration, while Qwen-2.5-VL displayed overconfidence. Furthermore, image-RAG grounded predictions with visually similar cases, enabling correction of agent overconfidence via iterative re-evaluation. The proposed system separates perception (vision agents) from meta-reasoning (orchestrator), enabling scalable and interpretable multi-agent AI. This blueprint illustrates how Agentic AI can deliver trustworthy, modular, and transparent reasoning, and is extensible to diagnostics, biology, and other trust-critical domains. In doing so, we highlight Agentic AI not just as an architecture but as a paradigm for building reliable multi-agent intelligence. All models, prompts, results, and system components including the complete software source code are openly released to support reproducibility, transparency, and community benchmarking at our Github page.},
keywords = {Accuracy, Adaptation models, Agentic AI, Artificial intelligence, Calibration, Cognition, Costs, orchestrator agent trust, Retrieval augmented generation, retrieval augmented reasoning, Training, trust orchestration, visual classification, Visualization},
pubstate = {published},
tppubtype = {article}
}
Modern Artificial Intelligence (AI) increasingly relies on multi-agent architectures that blend visual and language understanding. Yet, a pressing challenge remains: How can we trust these agents especially in zero-shot settings with no fine-tuning? We introduce a novel modular Agentic AI visual classification framework that integrates generalist multimodal agents with a non-visual reasoning orchestrator and a Retrieval-Augmented Generation (RAG) module. Applied to apple leaf disease diagnosis, we benchmark three configurations: (I) zero-shot with confidence-based orchestration, (II) fine-tuned agents with improved performance, and (III) trust-calibrated orchestration enhanced by CLIP-based image retrieval and re-evaluation loops. Using confidence calibration metrics (ECE, OCR, CCC), the orchestrator modulates trust across agents. Our results demonstrate a 77.94% accuracy improvement in the zero-shot setting using trust-aware orchestration and RAG, achieving 85.63% overall. GPT-4o showed better calibration, while Qwen-2.5-VL displayed overconfidence. Furthermore, image-RAG grounded predictions with visually similar cases, enabling correction of agent overconfidence via iterative re-evaluation. The proposed system separates perception (vision agents) from meta-reasoning (orchestrator), enabling scalable and interpretable multi-agent AI. This blueprint illustrates how Agentic AI can deliver trustworthy, modular, and transparent reasoning, and is extensible to diagnostics, biology, and other trust-critical domains. In doing so, we highlight Agentic AI not just as an architecture but as a paradigm for building reliable multi-agent intelligence. All models, prompts, results, and system components including the complete software source code are openly released to support reproducibility, transparency, and community benchmarking at our Github page.
2025
2.
Ranjan Sapkota; Rizwan Qureshi; Muhammad Usman Hadi; Syed Zohaib Hassan; Ferhat Sadak; Maged Shoman; Muhammad Sajjad; Fayaz Ali Dharejo; Achyut Paudel; Jiajia Li; Zhichao Meng; John Shutske; Manoj Karkee
Multi-Modal LLMs in Agriculture: A Comprehensive Review Journal Article
In: IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 22510–22540, 2025, ISSN: 1558-3783.
Abstract | Links | BibTeX | Tags: Agriculture, Analytical models, ChatGPT, Computational modeling, Computer vision, Data models, Deep learning, Farming, generative artificial intelligence, Hidden Markov models, Large language models (LLMs), Machine Learning, Precision agriculture, Reviews, Training, Transformers, Translation, Vision-language models
@article{sapkota_multi-modal_2025,
title = {Multi-Modal LLMs in Agriculture: A Comprehensive Review},
author = {Ranjan Sapkota and Rizwan Qureshi and Muhammad Usman Hadi and Syed Zohaib Hassan and Ferhat Sadak and Maged Shoman and Muhammad Sajjad and Fayaz Ali Dharejo and Achyut Paudel and Jiajia Li and Zhichao Meng and John Shutske and Manoj Karkee},
url = {https://ieeexplore.ieee.org/document/11173627},
doi = {10.1109/TASE.2025.3612154},
issn = {1558-3783},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
journal = {IEEE Transactions on Automation Science and Engineering},
volume = {22},
pages = {22510\textendash22540},
abstract = {Given the rapid emergence and applications of Multi-Modal Large Language Models (MM-LLMs) across various scientific fields, insights regarding their applicability in agriculture are still only partially explored. This paper conducts an in-depth review of MM-LLMs in agriculture, focusing on understanding how MM-LLMs can be developed and implemented to optimize agricultural processes, increase efficiency, and reduce costs. Recent studies have explored the capabilities of MM-LLMs in agricultural information processing and decision-making. Despite these advancements, significant gaps persist, particularly in addressing domain-specific challenges such as variable data quality and availability, integration with existing agricultural systems, and the creation of robust training datasets that accurately represent complex agricultural environments. Moreover, a comprehensive understanding of the capabilities, challenges, and limitations of MM-LLMs in agricultural information processing and application is still missing. Exploring these areas is crucial to providing the community with a broader perspective and a clearer understanding of MM-LLMs’ applications, establishing a benchmark for the current state and emerging trends in this field. To bridge this gap, this survey reviews the progress of MM-LLMs and their utilization in agriculture, with an additional focus on 11 key research questions (RQs), where 4 RQs are general and 7 RQs are agriculture focused. By addressing these RQs, this review outlines the current opportunities and challenges, limitations, and future roadmap for MM-LLMs in agriculture. The findings indicate that multi-modal MM-LLMs not only simplify complex agricultural challenges but also significantly enhance decision-making and improve the efficiency of agricultural image processing. These advancements position MM-LLMs as an essential tool for the future of farming. For continued research and understanding, an organized and regularly updated list of papers on MM-LLMs is available at https://github.com/JiajiaLi04/Multi-Modal-LLMs-in-Agriculture Note to Practitioners\textemdashMotivated by the need to optimize agricultural practices, this paper investigates the use of Large Language Models (MM-LLMs) to improve efficiency and decision-making in agriculture. We delve into critical RQs to reveal the capabilities and challenges of MM-LLMs, and their potential applications in the agricultural sector. Looking ahead, our findings suggest a promising future for the integration of MM-LLMs in agriculture, potentially revolutionizing how we manage and operate farms.},
keywords = {Agriculture, Analytical models, ChatGPT, Computational modeling, Computer vision, Data models, Deep learning, Farming, generative artificial intelligence, Hidden Markov models, Large language models (LLMs), Machine Learning, Precision agriculture, Reviews, Training, Transformers, Translation, Vision-language models},
pubstate = {published},
tppubtype = {article}
}
Given the rapid emergence and applications of Multi-Modal Large Language Models (MM-LLMs) across various scientific fields, insights regarding their applicability in agriculture are still only partially explored. This paper conducts an in-depth review of MM-LLMs in agriculture, focusing on understanding how MM-LLMs can be developed and implemented to optimize agricultural processes, increase efficiency, and reduce costs. Recent studies have explored the capabilities of MM-LLMs in agricultural information processing and decision-making. Despite these advancements, significant gaps persist, particularly in addressing domain-specific challenges such as variable data quality and availability, integration with existing agricultural systems, and the creation of robust training datasets that accurately represent complex agricultural environments. Moreover, a comprehensive understanding of the capabilities, challenges, and limitations of MM-LLMs in agricultural information processing and application is still missing. Exploring these areas is crucial to providing the community with a broader perspective and a clearer understanding of MM-LLMs’ applications, establishing a benchmark for the current state and emerging trends in this field. To bridge this gap, this survey reviews the progress of MM-LLMs and their utilization in agriculture, with an additional focus on 11 key research questions (RQs), where 4 RQs are general and 7 RQs are agriculture focused. By addressing these RQs, this review outlines the current opportunities and challenges, limitations, and future roadmap for MM-LLMs in agriculture. The findings indicate that multi-modal MM-LLMs not only simplify complex agricultural challenges but also significantly enhance decision-making and improve the efficiency of agricultural image processing. These advancements position MM-LLMs as an essential tool for the future of farming. For continued research and understanding, an organized and regularly updated list of papers on MM-LLMs is available at https://github.com/JiajiaLi04/Multi-Modal-LLMs-in-Agriculture Note to Practitioners—Motivated by the need to optimize agricultural practices, this paper investigates the use of Large Language Models (MM-LLMs) to improve efficiency and decision-making in agriculture. We delve into critical RQs to reveal the capabilities and challenges of MM-LLMs, and their potential applications in the agricultural sector. Looking ahead, our findings suggest a promising future for the integration of MM-LLMs in agriculture, potentially revolutionizing how we manage and operate farms.
2024
3.
Ozmen Erkin Kokten; Raviv Raich; James Holmes; Alan Fern
Learning Extended Forecasts of Soil Water Content via Physically-Inspired Autoregressive Models Proceedings Article
In: 2024 International Conference on Machine Learning and Applications (ICMLA), pp. 400–407, 2024, (ISSN: 1946-0759).
Abstract | Links | BibTeX | Tags: autoregressive training, non-linear state-space models, Pipelines, Predictive models, Soil measurements, Soil Water Content, State-space methods, Stress, teacher-forcing, time-series, Training, Training data, Weather forecasting
@inproceedings{kokten_learning_2024,
title = {Learning Extended Forecasts of Soil Water Content via Physically-Inspired Autoregressive Models},
author = {Ozmen Erkin Kokten and Raviv Raich and James Holmes and Alan Fern},
url = {https://ieeexplore.ieee.org/abstract/document/10903312},
doi = {10.1109/ICMLA61862.2024.00060},
year = {2024},
date = {2024-12-01},
urldate = {2024-12-01},
booktitle = {2024 International Conference on Machine Learning and Applications (ICMLA)},
pages = {400\textendash407},
abstract = {Vine stress resulting from soil water content (SWC) restrictions allows growers to improve grape and subsequent wine quality. In this work, we consider learning models that can forecast SWC to assist growers' irrigation decisions. In particular, we investigate training auto-regressive recurrent neural networks to make multi-day hourly forecasts of SWC based on historical data from soil-moisture sensors, irrigation sched-ules, and evapotranspiration estimates. Our work addresses two practical challenges in training such models. First, trained auto-regressive models are prone to error propagation, which quickly degrades longer-term forecasts. Second, it is difficult to learn the underlying causal relationship between irrigation and soil moisture due to the training data having limited coverage of the primary control input, irrigation. We propose a training strategy that combines one-step teacher forcing loss with a loss over multi-step autoregressive predictions and novel regularization terms to ensure SWC forecasts align with scientific models, effectively addressing the key challenges. We present results from five irrigation blocks with two cultivars, using datasets ranging from 2947 to 4784 hourly measurements of SWC, irrigation, and weather. Our methodology achieves precise SWC predictions and generates realistic forecasts for untrained irrigation scenarios.},
note = {ISSN: 1946-0759},
keywords = {autoregressive training, non-linear state-space models, Pipelines, Predictive models, Soil measurements, Soil Water Content, State-space methods, Stress, teacher-forcing, time-series, Training, Training data, Weather forecasting},
pubstate = {published},
tppubtype = {inproceedings}
}
Vine stress resulting from soil water content (SWC) restrictions allows growers to improve grape and subsequent wine quality. In this work, we consider learning models that can forecast SWC to assist growers' irrigation decisions. In particular, we investigate training auto-regressive recurrent neural networks to make multi-day hourly forecasts of SWC based on historical data from soil-moisture sensors, irrigation sched-ules, and evapotranspiration estimates. Our work addresses two practical challenges in training such models. First, trained auto-regressive models are prone to error propagation, which quickly degrades longer-term forecasts. Second, it is difficult to learn the underlying causal relationship between irrigation and soil moisture due to the training data having limited coverage of the primary control input, irrigation. We propose a training strategy that combines one-step teacher forcing loss with a loss over multi-step autoregressive predictions and novel regularization terms to ensure SWC forecasts align with scientific models, effectively addressing the key challenges. We present results from five irrigation blocks with two cultivars, using datasets ranging from 2947 to 4784 hourly measurements of SWC, irrigation, and weather. Our methodology achieves precise SWC predictions and generates realistic forecasts for untrained irrigation scenarios.
