2024
111.
Ozmen Erkin Kokten; Raviv Raich; James Holmes; Alan Fern
Learning Extended Forecasts of Soil Water Content via Physically-Inspired Autoregressive Models Proceedings Article
In: 2024 International Conference on Machine Learning and Applications (ICMLA), pp. 400–407, 2024, (ISSN: 1946-0759).
Abstract | Links | BibTeX | Tags: autoregressive training, non-linear state-space models, Pipelines, Predictive models, Soil measurements, Soil Water Content, State-space methods, Stress, teacher-forcing, time-series, Training, Training data, Weather forecasting
@inproceedings{kokten_learning_2024,
title = {Learning Extended Forecasts of Soil Water Content via Physically-Inspired Autoregressive Models},
author = {Ozmen Erkin Kokten and Raviv Raich and James Holmes and Alan Fern},
url = {https://ieeexplore.ieee.org/abstract/document/10903312},
doi = {10.1109/ICMLA61862.2024.00060},
year = {2024},
date = {2024-12-01},
urldate = {2024-12-01},
booktitle = {2024 International Conference on Machine Learning and Applications (ICMLA)},
pages = {400\textendash407},
abstract = {Vine stress resulting from soil water content (SWC) restrictions allows growers to improve grape and subsequent wine quality. In this work, we consider learning models that can forecast SWC to assist growers\' irrigation decisions. In particular, we investigate training auto-regressive recurrent neural networks to make multi-day hourly forecasts of SWC based on historical data from soil-moisture sensors, irrigation sched-ules, and evapotranspiration estimates. Our work addresses two practical challenges in training such models. First, trained auto-regressive models are prone to error propagation, which quickly degrades longer-term forecasts. Second, it is difficult to learn the underlying causal relationship between irrigation and soil moisture due to the training data having limited coverage of the primary control input, irrigation. We propose a training strategy that combines one-step teacher forcing loss with a loss over multi-step autoregressive predictions and novel regularization terms to ensure SWC forecasts align with scientific models, effectively addressing the key challenges. We present results from five irrigation blocks with two cultivars, using datasets ranging from 2947 to 4784 hourly measurements of SWC, irrigation, and weather. Our methodology achieves precise SWC predictions and generates realistic forecasts for untrained irrigation scenarios.},
note = {ISSN: 1946-0759},
keywords = {autoregressive training, non-linear state-space models, Pipelines, Predictive models, Soil measurements, Soil Water Content, State-space methods, Stress, teacher-forcing, time-series, Training, Training data, Weather forecasting},
pubstate = {published},
tppubtype = {inproceedings}
}
Vine stress resulting from soil water content (SWC) restrictions allows growers to improve grape and subsequent wine quality. In this work, we consider learning models that can forecast SWC to assist growers' irrigation decisions. In particular, we investigate training auto-regressive recurrent neural networks to make multi-day hourly forecasts of SWC based on historical data from soil-moisture sensors, irrigation sched-ules, and evapotranspiration estimates. Our work addresses two practical challenges in training such models. First, trained auto-regressive models are prone to error propagation, which quickly degrades longer-term forecasts. Second, it is difficult to learn the underlying causal relationship between irrigation and soil moisture due to the training data having limited coverage of the primary control input, irrigation. We propose a training strategy that combines one-step teacher forcing loss with a loss over multi-step autoregressive predictions and novel regularization terms to ensure SWC forecasts align with scientific models, effectively addressing the key challenges. We present results from five irrigation blocks with two cultivars, using datasets ranging from 2947 to 4784 hourly measurements of SWC, irrigation, and weather. Our methodology achieves precise SWC predictions and generates realistic forecasts for untrained irrigation scenarios.
112.
Jostan Brown; Achyut Paudel; Deven Biehler; Ashley Thompson; Manoj Karkee; Cindy Grimm; Joseph R. Davidson
Tree detection and in-row localization for autonomous precision orchard management Journal Article
In: Computers and Electronics in Agriculture, vol. 227, pp. 109454, 2024, ISSN: 0168-1699.
Abstract | Links | BibTeX | Tags: AI, Labor
@article{brown_tree_2024,
title = {Tree detection and in-row localization for autonomous precision orchard management},
author = {Jostan Brown and Achyut Paudel and Deven Biehler and Ashley Thompson and Manoj Karkee and Cindy Grimm and Joseph R. Davidson},
url = {https://www.sciencedirect.com/science/article/pii/S0168169924008457},
doi = {10.1016/j.compag.2024.109454},
issn = {0168-1699},
year = {2024},
date = {2024-12-01},
urldate = {2024-12-01},
journal = {Computers and Electronics in Agriculture},
volume = {227},
pages = {109454},
abstract = {This work presents a framework for localizing ground robots within fruit tree orchards. The standard practice of managing orchards at the large block-level does not maximize the potential of farms \textemdash individual plants have different needs due to variations in soil, pests, disease, irrigation, etc. In order to make selective management decisions for individual trees, such as precision fertilization, a robot must be able to accurately localize itself within the row. This is a challenge since in high density, modern orchard systems it is often difficult to obtain accurate GNSS measurements. Our algorithm begins by using deep learning to segment a tree trunk in an RGB-D image and then estimate its width. We then use the trunk segmentations and widths to calculate particle weights in a particle filter-based localization system. We show that integrating trunk width into the particle update step led to a 45% decrease in the distance traveled before convergence, and a 31% decrease in convergence time, alongside a marginal increase in the rate of correct convergence. We also demonstrate autonomous tree-level localization with a large ground robot in realistic field experiments in a commercial apple orchard.},
keywords = {AI, Labor},
pubstate = {published},
tppubtype = {article}
}
This work presents a framework for localizing ground robots within fruit tree orchards. The standard practice of managing orchards at the large block-level does not maximize the potential of farms — individual plants have different needs due to variations in soil, pests, disease, irrigation, etc. In order to make selective management decisions for individual trees, such as precision fertilization, a robot must be able to accurately localize itself within the row. This is a challenge since in high density, modern orchard systems it is often difficult to obtain accurate GNSS measurements. Our algorithm begins by using deep learning to segment a tree trunk in an RGB-D image and then estimate its width. We then use the trunk segmentations and widths to calculate particle weights in a particle filter-based localization system. We show that integrating trunk width into the particle update step led to a 45% decrease in the distance traveled before convergence, and a 31% decrease in convergence time, alongside a marginal increase in the rate of correct convergence. We also demonstrate autonomous tree-level localization with a large ground robot in realistic field experiments in a commercial apple orchard.
113.
Syrine Belakaria; Benjamin Letham; Janardhan Rao Doppa; Barbara Engelhardt; Stefano Ermon; Eytan Bakshy
Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes Proceedings Article
In: The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Abstract | Links | BibTeX | Tags: Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning
@inproceedings{belakaria_active_2024,
title = {Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes},
author = {Syrine Belakaria and Benjamin Letham and Janardhan Rao Doppa and Barbara Engelhardt and Stefano Ermon and Eytan Bakshy},
url = {https://openreview.net/forum?id=da0ZJatRCN},
doi = {10.48550/arXiv.2407.09739},
year = {2024},
date = {2024-11-06},
urldate = {2024-11-06},
publisher = {The Thirty-eighth Annual Conference on Neural Information Processing Systems},
abstract = {We consider the problem of active learning for global sensitivity analysis of expensive black-box functions. Our aim is to efficiently learn the importance of different input variables, e.g., in vehicle safety experimentation, we study the impact of the thickness of various components on safety objectives. Since function evaluations are expensive, we use active learning to prioritize experimental resources where they yield the most value. We propose novel active learning acquisition functions that directly target key quantities of derivative-based global sensitivity measures (DGSMs) under Gaussian process surrogate models. We showcase the first application of active learning directly to DGSMs, and develop tractable uncertainty reduction and information gain acquisition functions for these measures. Through comprehensive evaluation on synthetic and real-world problems, our study demonstrates how these active learning acquisition strategies substantially enhance the sample efficiency of DGSM estimation, particularly with limited evaluation budgets. Our work paves the way for more efficient and accurate sensitivity analysis in various scientific and engineering applications.},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
We consider the problem of active learning for global sensitivity analysis of expensive black-box functions. Our aim is to efficiently learn the importance of different input variables, e.g., in vehicle safety experimentation, we study the impact of the thickness of various components on safety objectives. Since function evaluations are expensive, we use active learning to prioritize experimental resources where they yield the most value. We propose novel active learning acquisition functions that directly target key quantities of derivative-based global sensitivity measures (DGSMs) under Gaussian process surrogate models. We showcase the first application of active learning directly to DGSMs, and develop tractable uncertainty reduction and information gain acquisition functions for these measures. Through comprehensive evaluation on synthetic and real-world problems, our study demonstrates how these active learning acquisition strategies substantially enhance the sample efficiency of DGSM estimation, particularly with limited evaluation budgets. Our work paves the way for more efficient and accurate sensitivity analysis in various scientific and engineering applications.
114.
Ranjan Sapkota; Achyut Paudel; Manoj Karkee
2024, (arXiv:2411.11285).
Abstract | Links | BibTeX | Tags: Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition
@misc{sapkota_zero-shot_2024,
title = {Zero-Shot Automatic Annotation and Instance Segmentation using LLM-Generated Datasets: Eliminating Field Imaging and Manual Annotation for Deep Learning Model Development},
author = {Ranjan Sapkota and Achyut Paudel and Manoj Karkee},
url = {http://arxiv.org/abs/2411.11285},
doi = {10.48550/arXiv.2411.11285},
year = {2024},
date = {2024-11-01},
urldate = {2024-11-01},
publisher = {arXiv},
abstract = {Currently, deep learning-based instance segmentation for various applications (e.g., Agriculture) is predominantly performed using a labor-intensive process involving extensive field data collection using sophisticated sensors, followed by careful manual annotation of images, presenting significant logistical and financial challenges to researchers and organizations. The process also slows down the model development and training process. In this study, we presented a novel method for deep learning-based instance segmentation of apples in commercial orchards that eliminates the need for labor-intensive field data collection and manual annotation. Utilizing a Large Language Model (LLM), we synthetically generated orchard images and automatically annotated them using the Segment Anything Model (SAM) integrated with a YOLO11 base model. This method significantly reduces reliance on physical sensors and manual data processing, presenting a major advancement in "Agricultural AI". The synthetic, auto-annotated dataset was used to train the YOLO11 model for Apple instance segmentation, which was then validated on real orchard images. The results showed that the automatically generated annotations achieved a Dice Coefficient of 0.9513 and an IoU of 0.9303, validating the accuracy and overlap of the mask annotations. All YOLO11 configurations, trained solely on these synthetic datasets with automated annotations, accurately recognized and delineated apples, highlighting the method\'s efficacy. Specifically, the YOLO11m-seg configuration achieved a mask precision of 0.902 and a mask mAP@50 of 0.833 on test images collected from a commercial orchard. Additionally, the YOLO11l-seg configuration outperformed other models in validation on 40 LLM-generated images, achieving the highest mask precision and mAP@50 metrics. Keywords: YOLO, SAM, SAMv2, YOLO11, YOLOv11, Segment Anything, YOLO-SAM},
note = {arXiv:2411.11285},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition},
pubstate = {published},
tppubtype = {misc}
}
Currently, deep learning-based instance segmentation for various applications (e.g., Agriculture) is predominantly performed using a labor-intensive process involving extensive field data collection using sophisticated sensors, followed by careful manual annotation of images, presenting significant logistical and financial challenges to researchers and organizations. The process also slows down the model development and training process. In this study, we presented a novel method for deep learning-based instance segmentation of apples in commercial orchards that eliminates the need for labor-intensive field data collection and manual annotation. Utilizing a Large Language Model (LLM), we synthetically generated orchard images and automatically annotated them using the Segment Anything Model (SAM) integrated with a YOLO11 base model. This method significantly reduces reliance on physical sensors and manual data processing, presenting a major advancement in "Agricultural AI". The synthetic, auto-annotated dataset was used to train the YOLO11 model for Apple instance segmentation, which was then validated on real orchard images. The results showed that the automatically generated annotations achieved a Dice Coefficient of 0.9513 and an IoU of 0.9303, validating the accuracy and overlap of the mask annotations. All YOLO11 configurations, trained solely on these synthetic datasets with automated annotations, accurately recognized and delineated apples, highlighting the method's efficacy. Specifically, the YOLO11m-seg configuration achieved a mask precision of 0.902 and a mask mAP@50 of 0.833 on test images collected from a commercial orchard. Additionally, the YOLO11l-seg configuration outperformed other models in validation on 40 LLM-generated images, achieving the highest mask precision and mAP@50 metrics. Keywords: YOLO, SAM, SAMv2, YOLO11, YOLOv11, Segment Anything, YOLO-SAM
115.
Mohammad Rafid Ul Islam; Prasad Tadepalli; Alan Fern
Self-attention-based Diffusion Model for Time-series Imputation Journal Article
In: Proceedings of the AAAI Symposium Series, vol. 4, no. 1, pp. 424–431, 2024, ISSN: 2994-4317, (Number: 1).
Abstract | Links | BibTeX | Tags:
@article{islam_self-attention-based_2024,
title = {Self-attention-based Diffusion Model for Time-series Imputation},
author = {Mohammad Rafid Ul Islam and Prasad Tadepalli and Alan Fern},
url = {https://ojs.aaai.org/index.php/AAAI-SS/article/view/31827},
doi = {10.1609/aaaiss.v4i1.31827},
issn = {2994-4317},
year = {2024},
date = {2024-11-01},
urldate = {2024-11-01},
journal = {Proceedings of the AAAI Symposium Series},
volume = {4},
number = {1},
pages = {424\textendash431},
abstract = {Time-series modeling is essential for applications in agriculture, weather forecasting, food production, and more. However, missing data due to sensor malfunctions, power outages, and human errors is a common issue, complicating the training of machine learning models. We propose a diffusion-based generative model to
address this problem and fill the gaps in the data. Our approach captures feature and time correlations through a two-stage imputation process. Our model outperforms state-of-the-art imputation methods and is more scalable in GPU resources.},
note = {Number: 1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Time-series modeling is essential for applications in agriculture, weather forecasting, food production, and more. However, missing data due to sensor malfunctions, power outages, and human errors is a common issue, complicating the training of machine learning models. We propose a diffusion-based generative model to
address this problem and fill the gaps in the data. Our approach captures feature and time correlations through a two-stage imputation process. Our model outperforms state-of-the-art imputation methods and is more scalable in GPU resources.
address this problem and fill the gaps in the data. Our approach captures feature and time correlations through a two-stage imputation process. Our model outperforms state-of-the-art imputation methods and is more scalable in GPU resources.
116.
Srikanth Gorthi; Dattatray G. Bhalekar; Lav R. Khot; Markus Keller
Modeling Grape Berry Temperature for Effective Heat Stress Management in Vineyards Proceedings Article
In: 2024 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), pp. 237–241, 2024.
Abstract | Links | BibTeX | Tags: Atmospheric modeling, Berry Temperature, Lasso Regression, Pipelines, Random Forest, Random forests, Real-time systems, Ridge Regression, Soil, Soil measurements, Solar radiation, Stress, Temperature measurement, Water heating
@inproceedings{gorthi_modeling_2024,
title = {Modeling Grape Berry Temperature for Effective Heat Stress Management in Vineyards},
author = {Srikanth Gorthi and Dattatray G. Bhalekar and Lav R. Khot and Markus Keller},
url = {https://ieeexplore.ieee.org/document/10948825},
doi = {10.1109/MetroAgriFor63043.2024.10948825},
year = {2024},
date = {2024-10-01},
urldate = {2024-10-01},
booktitle = {2024 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor)},
pages = {237\textendash241},
abstract = {This study developed a machine learning model to predict berry temperature using localized weather and soil attributes measured during the summer of 2023. Berry temperature was observed to be higher compared to canopy and air temperature during extreme heat events. A tree-based pipeline optimization tool was used to find an optimum machine learning algorithm. Amongst the tested models, Lasso Regression exhibited reasonable accuracy (R2 = 0.98) and root mean squared error of 0.93 °C on the test dataset.},
keywords = {Atmospheric modeling, Berry Temperature, Lasso Regression, Pipelines, Random Forest, Random forests, Real-time systems, Ridge Regression, Soil, Soil measurements, Solar radiation, Stress, Temperature measurement, Water heating},
pubstate = {published},
tppubtype = {inproceedings}
}
This study developed a machine learning model to predict berry temperature using localized weather and soil attributes measured during the summer of 2023. Berry temperature was observed to be higher compared to canopy and air temperature during extreme heat events. A tree-based pipeline optimization tool was used to find an optimum machine learning algorithm. Amongst the tested models, Lasso Regression exhibited reasonable accuracy (R2 = 0.98) and root mean squared error of 0.93 °C on the test dataset.
117.
Basavaraj R. Amogi; Lav R. Khot; Bernardita V. Sallato
Localized Crop Physiology Sensing System Driven Apple Fruit Color Progression Monitoring Proceedings Article
In: 2024 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), pp. 232–236, 2024.
Abstract | Links | BibTeX | Tags: Biomedical monitoring, Crops, cyber physical system, fruit color, hue, Image color analysis, Monitoring, Prevention and mitigation, Real-time systems, Sensors, Solar heating, Solar radiation, Stress
@inproceedings{amogi_localized_2024,
title = {Localized Crop Physiology Sensing System Driven Apple Fruit Color Progression Monitoring},
author = {Basavaraj R. Amogi and Lav R. Khot and Bernardita V. Sallato},
url = {https://ieeexplore.ieee.org/document/10948846},
doi = {10.1109/MetroAgriFor63043.2024.10948846},
year = {2024},
date = {2024-10-01},
urldate = {2024-10-01},
booktitle = {2024 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor)},
pages = {232\textendash236},
abstract = {Fruit color is a critical quality attribute that significantly affects the commercial value of apples. Elevated air temperatures and solar radiation during heat waves can substantially impact fruit coloration, while mitigation techniques such as netting can further compromise fruit color development by trapping heat. Continuous monitoring of fruit color throughout the growing season, particularly under heat stress conditions, is thus essential for informed grower decision-making. This study leverages a previously developed localized crop physiology sensing system (CPSS) to enable real time monitoring of apple fruit color progression. Using visible imagery captured by the CPSS, apple fruit color was quantified by extracting the hue angle (ˆtextbackslashcirctextbackslashmathbfh). The results showed that, the influence of sunlight on measured color accuracy (ΔE) is lower and more stable during mid-day hours (1000 h − 1300 h), whereas notable variations were observed during early morning and late afternoon periods. The ˆtextbackslashcirctextbackslashmathbfh hence calculated for RGB images captured around 1200 h was utilized to track fruit color progression. Its monitoring over the summer 2022 growing season showed variations as an effect of environmental stressors, especially in response to heat wave. The developed approach offers a tool for growers to adjust different heat mitigation techniques during heat wave/events to mitigate any negative implication on fruit coloration.},
keywords = {Biomedical monitoring, Crops, cyber physical system, fruit color, hue, Image color analysis, Monitoring, Prevention and mitigation, Real-time systems, Sensors, Solar heating, Solar radiation, Stress},
pubstate = {published},
tppubtype = {inproceedings}
}
Fruit color is a critical quality attribute that significantly affects the commercial value of apples. Elevated air temperatures and solar radiation during heat waves can substantially impact fruit coloration, while mitigation techniques such as netting can further compromise fruit color development by trapping heat. Continuous monitoring of fruit color throughout the growing season, particularly under heat stress conditions, is thus essential for informed grower decision-making. This study leverages a previously developed localized crop physiology sensing system (CPSS) to enable real time monitoring of apple fruit color progression. Using visible imagery captured by the CPSS, apple fruit color was quantified by extracting the hue angle (ˆtextbackslashcirctextbackslashmathbfh). The results showed that, the influence of sunlight on measured color accuracy (ΔE) is lower and more stable during mid-day hours (1000 h − 1300 h), whereas notable variations were observed during early morning and late afternoon periods. The ˆtextbackslashcirctextbackslashmathbfh hence calculated for RGB images captured around 1200 h was utilized to track fruit color progression. Its monitoring over the summer 2022 growing season showed variations as an effect of environmental stressors, especially in response to heat wave. The developed approach offers a tool for growers to adjust different heat mitigation techniques during heat wave/events to mitigate any negative implication on fruit coloration.
118.
Basavaraj R. Amogi; Lav R. Khot; Bernardita V. Sallato
Localized Crop Physiology Sensing System Driven Apple Fruit Color Progression Monitoring Proceedings Article
In: 2024 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), pp. 232–236, 2024.
Abstract | Links | BibTeX | Tags: AI, Farm Ops
@inproceedings{amogi_localized_2024b,
title = {Localized Crop Physiology Sensing System Driven Apple Fruit Color Progression Monitoring},
author = {Basavaraj R. Amogi and Lav R. Khot and Bernardita V. Sallato},
url = {https://ieeexplore.ieee.org/document/10948846},
doi = {10.1109/MetroAgriFor63043.2024.10948846},
year = {2024},
date = {2024-10-01},
urldate = {2024-10-01},
booktitle = {2024 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor)},
pages = {232\textendash236},
abstract = {Fruit color is a critical quality attribute that significantly affects the commercial value of apples. Elevated air temperatures and solar radiation during heat waves can substantially impact fruit coloration, while mitigation techniques such as netting can further compromise fruit color development by trapping heat. Continuous monitoring of fruit color throughout the growing season, particularly under heat stress conditions, is thus essential for informed grower decision-making. This study leverages a previously developed localized crop physiology sensing system (CPSS) to enable real time monitoring of apple fruit color progression. Using visible imagery captured by the CPSS, apple fruit color was quantified by extracting the hue angle (ˆtextbackslashcirctextbackslashmathbfh). The results showed that, the influence of sunlight on measured color accuracy (ΔE) is lower and more stable during mid-day hours (1000 h − 1300 h), whereas notable variations were observed during early morning and late afternoon periods. The ˆtextbackslashcirctextbackslashmathbfh hence calculated for RGB images captured around 1200 h was utilized to track fruit color progression. Its monitoring over the summer 2022 growing season showed variations as an effect of environmental stressors, especially in response to heat wave. The developed approach offers a tool for growers to adjust different heat mitigation techniques during heat wave/events to mitigate any negative implication on fruit coloration.},
keywords = {AI, Farm Ops},
pubstate = {published},
tppubtype = {inproceedings}
}
Fruit color is a critical quality attribute that significantly affects the commercial value of apples. Elevated air temperatures and solar radiation during heat waves can substantially impact fruit coloration, while mitigation techniques such as netting can further compromise fruit color development by trapping heat. Continuous monitoring of fruit color throughout the growing season, particularly under heat stress conditions, is thus essential for informed grower decision-making. This study leverages a previously developed localized crop physiology sensing system (CPSS) to enable real time monitoring of apple fruit color progression. Using visible imagery captured by the CPSS, apple fruit color was quantified by extracting the hue angle (ˆtextbackslashcirctextbackslashmathbfh). The results showed that, the influence of sunlight on measured color accuracy (ΔE) is lower and more stable during mid-day hours (1000 h − 1300 h), whereas notable variations were observed during early morning and late afternoon periods. The ˆtextbackslashcirctextbackslashmathbfh hence calculated for RGB images captured around 1200 h was utilized to track fruit color progression. Its monitoring over the summer 2022 growing season showed variations as an effect of environmental stressors, especially in response to heat wave. The developed approach offers a tool for growers to adjust different heat mitigation techniques during heat wave/events to mitigate any negative implication on fruit coloration.
119.
Bikram Pandit; Ashutosh Gupta; Mohitvishnu S. Gadde; Addison Johnson; Aayam Kumar Shrestha; Helei Duan; Jeremy Dao; Alan Fern
Learning Decentralized Multi-Biped Control for Payload Transport Proceedings Article
In: 8th Annual Conference on Robot Learning, 2024.
Abstract | Links | BibTeX | Tags: Computer Science - Artificial Intelligence, Computer Science - Robotics
@inproceedings{pandit_learning_2024,
title = {Learning Decentralized Multi-Biped Control for Payload Transport},
author = {Bikram Pandit and Ashutosh Gupta and Mohitvishnu S. Gadde and Addison Johnson and Aayam Kumar Shrestha and Helei Duan and Jeremy Dao and Alan Fern},
url = {https://openreview.net/forum?id=vhGkyWgctu},
doi = {10.48550/arXiv.2406.17279},
year = {2024},
date = {2024-09-05},
urldate = {2024-09-05},
publisher = {8th Annual Conference on Robot Learning},
abstract = {Payload transport over flat terrain via multi-wheel robot carriers is well-understood, highly effective, and configurable. In this paper, our goal is to provide similar effectiveness and configurability for transport over rough terrain that is more suitable for legs rather than wheels. For this purpose, we consider multi-biped robot carriers, where wheels are replaced by multiple bipedal robots attached to the carrier. Our main contribution is to design a decentralized controller for such systems that can be effectively applied to varying numbers and configurations of rigidly attached bipedal robots without retraining. We present a reinforcement learning approach for training the controller in simulation that supports transfer to the real world. Our experiments in simulation provide quantitative metrics showing the effectiveness of the approach over a wide variety of simulated transport scenarios. In addition, we demonstrate the controller in the real-world for systems composed of two and three Cassie robots. To our knowledge, this is the first example of a scalable multi-biped payload transport system.},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Robotics},
pubstate = {published},
tppubtype = {inproceedings}
}
Payload transport over flat terrain via multi-wheel robot carriers is well-understood, highly effective, and configurable. In this paper, our goal is to provide similar effectiveness and configurability for transport over rough terrain that is more suitable for legs rather than wheels. For this purpose, we consider multi-biped robot carriers, where wheels are replaced by multiple bipedal robots attached to the carrier. Our main contribution is to design a decentralized controller for such systems that can be effectively applied to varying numbers and configurations of rigidly attached bipedal robots without retraining. We present a reinforcement learning approach for training the controller in simulation that supports transfer to the real world. Our experiments in simulation provide quantitative metrics showing the effectiveness of the approach over a wide variety of simulated transport scenarios. In addition, we demonstrate the controller in the real-world for systems composed of two and three Cassie robots. To our knowledge, this is the first example of a scalable multi-biped payload transport system.
120.
Ranjan Sapkota; Dawood Ahmed; Manoj Karkee
Comparing YOLOv8 and Mask R-CNN for instance segmentation in complex orchard environments Journal Article
In: Artificial Intelligence in Agriculture, vol. 13, pp. 84–99, 2024, ISSN: 2589-7217.
Abstract | Links | BibTeX | Tags: Artificial intelligence, Automation, Deep learning, Machine Learning, Machine vision, Mask R-CNN, Robotics, YOLOv8
@article{sapkota_comparing_2024,
title = {Comparing YOLOv8 and Mask R-CNN for instance segmentation in complex orchard environments},
author = {Ranjan Sapkota and Dawood Ahmed and Manoj Karkee},
url = {https://www.sciencedirect.com/science/article/pii/S258972172400028X},
doi = {10.1016/j.aiia.2024.07.001},
issn = {2589-7217},
year = {2024},
date = {2024-09-01},
urldate = {2024-09-01},
journal = {Artificial Intelligence in Agriculture},
volume = {13},
pages = {84\textendash99},
abstract = {Instance segmentation, an important image processing operation for automation in agriculture, is used to precisely delineate individual objects of interest within images, which provides foundational information for various automated or robotic tasks such as selective harvesting and precision pruning. This study compares the one-stage YOLOv8 and the two-stage Mask R-CNN machine learning models for instance segmentation under varying orchard conditions across two datasets. Dataset 1, collected in dormant season, includes images of dormant apple trees, which were used to train multi-object segmentation models delineating tree branches and trunks. Dataset 2, collected in the early growing season, includes images of apple tree canopies with green foliage and immature (green) apples (also called fruitlet), which were used to train single-object segmentation models delineating only immature green apples. The results showed that YOLOv8 performed better than Mask R-CNN, achieving good precision and near-perfect recall across both datasets at a confidence threshold of 0.5. Specifically, for Dataset 1, YOLOv8 achieved a precision of 0.90 and a recall of 0.95 for all classes. In comparison, Mask R-CNN demonstrated a precision of 0.81 and a recall of 0.81 for the same dataset. With Dataset 2, YOLOv8 achieved a precision of 0.93 and a recall of 0.97. Mask R-CNN, in this single-class scenario, achieved a precision of 0.85 and a recall of 0.88. Additionally, the inference times for YOLOv8 were 10.9 ms for multi-class segmentation (Dataset 1) and 7.8 ms for single-class segmentation (Dataset 2), compared to 15.6 ms and 12.8 ms achieved by Mask R-CNN\'s, respectively. These findings show YOLOv8\'s superior accuracy and efficiency in machine learning applications compared to two-stage models, specifically Mask-R-CNN, which suggests its suitability in developing smart and automated orchard operations, particularly when real-time applications are necessary in such cases as robotic harvesting and robotic immature green fruit thinning.},
keywords = {Artificial intelligence, Automation, Deep learning, Machine Learning, Machine vision, Mask R-CNN, Robotics, YOLOv8},
pubstate = {published},
tppubtype = {article}
}
Instance segmentation, an important image processing operation for automation in agriculture, is used to precisely delineate individual objects of interest within images, which provides foundational information for various automated or robotic tasks such as selective harvesting and precision pruning. This study compares the one-stage YOLOv8 and the two-stage Mask R-CNN machine learning models for instance segmentation under varying orchard conditions across two datasets. Dataset 1, collected in dormant season, includes images of dormant apple trees, which were used to train multi-object segmentation models delineating tree branches and trunks. Dataset 2, collected in the early growing season, includes images of apple tree canopies with green foliage and immature (green) apples (also called fruitlet), which were used to train single-object segmentation models delineating only immature green apples. The results showed that YOLOv8 performed better than Mask R-CNN, achieving good precision and near-perfect recall across both datasets at a confidence threshold of 0.5. Specifically, for Dataset 1, YOLOv8 achieved a precision of 0.90 and a recall of 0.95 for all classes. In comparison, Mask R-CNN demonstrated a precision of 0.81 and a recall of 0.81 for the same dataset. With Dataset 2, YOLOv8 achieved a precision of 0.93 and a recall of 0.97. Mask R-CNN, in this single-class scenario, achieved a precision of 0.85 and a recall of 0.88. Additionally, the inference times for YOLOv8 were 10.9 ms for multi-class segmentation (Dataset 1) and 7.8 ms for single-class segmentation (Dataset 2), compared to 15.6 ms and 12.8 ms achieved by Mask R-CNN's, respectively. These findings show YOLOv8's superior accuracy and efficiency in machine learning applications compared to two-stage models, specifically Mask-R-CNN, which suggests its suitability in developing smart and automated orchard operations, particularly when real-time applications are necessary in such cases as robotic harvesting and robotic immature green fruit thinning.
