2024
1.
Ranjan Sapkota; Achyut Paudel; Manoj Karkee
2024, (arXiv:2411.11285).
Abstract | Links | BibTeX | Tags: Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition
@misc{sapkota_zero-shot_2024,
title = {Zero-Shot Automatic Annotation and Instance Segmentation using LLM-Generated Datasets: Eliminating Field Imaging and Manual Annotation for Deep Learning Model Development},
author = {Ranjan Sapkota and Achyut Paudel and Manoj Karkee},
url = {http://arxiv.org/abs/2411.11285},
doi = {10.48550/arXiv.2411.11285},
year = {2024},
date = {2024-11-01},
urldate = {2024-11-01},
publisher = {arXiv},
abstract = {Currently, deep learning-based instance segmentation for various applications (e.g., Agriculture) is predominantly performed using a labor-intensive process involving extensive field data collection using sophisticated sensors, followed by careful manual annotation of images, presenting significant logistical and financial challenges to researchers and organizations. The process also slows down the model development and training process. In this study, we presented a novel method for deep learning-based instance segmentation of apples in commercial orchards that eliminates the need for labor-intensive field data collection and manual annotation. Utilizing a Large Language Model (LLM), we synthetically generated orchard images and automatically annotated them using the Segment Anything Model (SAM) integrated with a YOLO11 base model. This method significantly reduces reliance on physical sensors and manual data processing, presenting a major advancement in "Agricultural AI". The synthetic, auto-annotated dataset was used to train the YOLO11 model for Apple instance segmentation, which was then validated on real orchard images. The results showed that the automatically generated annotations achieved a Dice Coefficient of 0.9513 and an IoU of 0.9303, validating the accuracy and overlap of the mask annotations. All YOLO11 configurations, trained solely on these synthetic datasets with automated annotations, accurately recognized and delineated apples, highlighting the method's efficacy. Specifically, the YOLO11m-seg configuration achieved a mask precision of 0.902 and a mask mAP@50 of 0.833 on test images collected from a commercial orchard. Additionally, the YOLO11l-seg configuration outperformed other models in validation on 40 LLM-generated images, achieving the highest mask precision and mAP@50 metrics. Keywords: YOLO, SAM, SAMv2, YOLO11, YOLOv11, Segment Anything, YOLO-SAM},
note = {arXiv:2411.11285},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition},
pubstate = {published},
tppubtype = {misc}
}
Currently, deep learning-based instance segmentation for various applications (e.g., Agriculture) is predominantly performed using a labor-intensive process involving extensive field data collection using sophisticated sensors, followed by careful manual annotation of images, presenting significant logistical and financial challenges to researchers and organizations. The process also slows down the model development and training process. In this study, we presented a novel method for deep learning-based instance segmentation of apples in commercial orchards that eliminates the need for labor-intensive field data collection and manual annotation. Utilizing a Large Language Model (LLM), we synthetically generated orchard images and automatically annotated them using the Segment Anything Model (SAM) integrated with a YOLO11 base model. This method significantly reduces reliance on physical sensors and manual data processing, presenting a major advancement in "Agricultural AI". The synthetic, auto-annotated dataset was used to train the YOLO11 model for Apple instance segmentation, which was then validated on real orchard images. The results showed that the automatically generated annotations achieved a Dice Coefficient of 0.9513 and an IoU of 0.9303, validating the accuracy and overlap of the mask annotations. All YOLO11 configurations, trained solely on these synthetic datasets with automated annotations, accurately recognized and delineated apples, highlighting the method's efficacy. Specifically, the YOLO11m-seg configuration achieved a mask precision of 0.902 and a mask mAP@50 of 0.833 on test images collected from a commercial orchard. Additionally, the YOLO11l-seg configuration outperformed other models in validation on 40 LLM-generated images, achieving the highest mask precision and mAP@50 metrics. Keywords: YOLO, SAM, SAMv2, YOLO11, YOLOv11, Segment Anything, YOLO-SAM
2.
Syrine Belakaria; Benjamin Letham; Janardhan Rao Doppa; Barbara Engelhardt; Stefano Ermon; Eytan Bakshy
Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes Proceedings Article
In: arXiv, 2024, (arXiv:2407.09739).
Abstract | Links | BibTeX | Tags: Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning
@inproceedings{belakaria_active_2024,
title = {Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes},
author = {Syrine Belakaria and Benjamin Letham and Janardhan Rao Doppa and Barbara Engelhardt and Stefano Ermon and Eytan Bakshy},
url = {http://arxiv.org/abs/2407.09739},
doi = {10.48550/arXiv.2407.09739},
year = {2024},
date = {2024-07-01},
urldate = {2024-07-01},
publisher = {arXiv},
abstract = {We consider the problem of active learning for global sensitivity analysis of expensive black-box functions. Our aim is to efficiently learn the importance of different input variables, e.g., in vehicle safety experimentation, we study the impact of the thickness of various components on safety objectives. Since function evaluations are expensive, we use active learning to prioritize experimental resources where they yield the most value. We propose novel active learning acquisition functions that directly target key quantities of derivative-based global sensitivity measures (DGSMs) under Gaussian process surrogate models. We showcase the first application of active learning directly to DGSMs, and develop tractable uncertainty reduction and information gain acquisition functions for these measures. Through comprehensive evaluation on synthetic and real-world problems, our study demonstrates how these active learning acquisition strategies substantially enhance the sample efficiency of DGSM estimation, particularly with limited evaluation budgets. Our work paves the way for more efficient and accurate sensitivity analysis in various scientific and engineering applications.},
note = {arXiv:2407.09739},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
We consider the problem of active learning for global sensitivity analysis of expensive black-box functions. Our aim is to efficiently learn the importance of different input variables, e.g., in vehicle safety experimentation, we study the impact of the thickness of various components on safety objectives. Since function evaluations are expensive, we use active learning to prioritize experimental resources where they yield the most value. We propose novel active learning acquisition functions that directly target key quantities of derivative-based global sensitivity measures (DGSMs) under Gaussian process surrogate models. We showcase the first application of active learning directly to DGSMs, and develop tractable uncertainty reduction and information gain acquisition functions for these measures. Through comprehensive evaluation on synthetic and real-world problems, our study demonstrates how these active learning acquisition strategies substantially enhance the sample efficiency of DGSM estimation, particularly with limited evaluation budgets. Our work paves the way for more efficient and accurate sensitivity analysis in various scientific and engineering applications.
3.
Bikram Pandit; Ashutosh Gupta; Mohitvishnu S. Gadde; Addison Johnson; Aayam Kumar Shrestha; Helei Duan; Jeremy Dao; Alan Fern
Learning Decentralized Multi-Biped Control for Payload Transport Proceedings
arXiv, 2024, (arXiv:2406.17279).
Abstract | Links | BibTeX | Tags: Computer Science - Artificial Intelligence, Computer Science - Robotics
@proceedings{pandit_learning_2024,
title = {Learning Decentralized Multi-Biped Control for Payload Transport},
author = {Bikram Pandit and Ashutosh Gupta and Mohitvishnu S. Gadde and Addison Johnson and Aayam Kumar Shrestha and Helei Duan and Jeremy Dao and Alan Fern},
url = {http://arxiv.org/abs/2406.17279},
doi = {10.48550/arXiv.2406.17279},
year = {2024},
date = {2024-06-01},
urldate = {2024-06-01},
publisher = {arXiv},
abstract = {Payload transport over flat terrain via multi-wheel robot carriers is well-understood, highly effective, and configurable. In this paper, our goal is to provide similar effectiveness and configurability for transport over rough terrain that is more suitable for legs rather than wheels. For this purpose, we consider multi-biped robot carriers, where wheels are replaced by multiple bipedal robots attached to the carrier. Our main contribution is to design a decentralized controller for such systems that can be effectively applied to varying numbers and configurations of rigidly attached bipedal robots without retraining. We present a reinforcement learning approach for training the controller in simulation that supports transfer to the real world. Our experiments in simulation provide quantitative metrics showing the effectiveness of the approach over a wide variety of simulated transport scenarios. In addition, we demonstrate the controller in the real-world for systems composed of two and three Cassie robots. To our knowledge, this is the first example of a scalable multi-biped payload transport system.},
note = {arXiv:2406.17279},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Robotics},
pubstate = {published},
tppubtype = {proceedings}
}
Payload transport over flat terrain via multi-wheel robot carriers is well-understood, highly effective, and configurable. In this paper, our goal is to provide similar effectiveness and configurability for transport over rough terrain that is more suitable for legs rather than wheels. For this purpose, we consider multi-biped robot carriers, where wheels are replaced by multiple bipedal robots attached to the carrier. Our main contribution is to design a decentralized controller for such systems that can be effectively applied to varying numbers and configurations of rigidly attached bipedal robots without retraining. We present a reinforcement learning approach for training the controller in simulation that supports transfer to the real world. Our experiments in simulation provide quantitative metrics showing the effectiveness of the approach over a wide variety of simulated transport scenarios. In addition, we demonstrate the controller in the real-world for systems composed of two and three Cassie robots. To our knowledge, this is the first example of a scalable multi-biped payload transport system.
4.
Mohammed Amine Gharsallaoui; Bhupinderjeet Singh; Supriya Savalkar; Aryan Deshwal; Yan Yan; Ananth Kalyanaraman; Kirti Rajagopalan; Janardhan Rao Doppa
Streamflow Prediction with Uncertainty Quantification for Water Management: A Constrained Reasoning and Learning Approach Miscellaneous
2024, (arXiv:2406.00133 [cs]).
Abstract | Links | BibTeX | Tags: Computer Science - Artificial Intelligence, Computer Science - Machine Learning
@misc{gharsallaoui_streamflow_2024,
title = {Streamflow Prediction with Uncertainty Quantification for Water Management: A Constrained Reasoning and Learning Approach},
author = {Mohammed Amine Gharsallaoui and Bhupinderjeet Singh and Supriya Savalkar and Aryan Deshwal and Yan Yan and Ananth Kalyanaraman and Kirti Rajagopalan and Janardhan Rao Doppa},
url = {http://arxiv.org/abs/2406.00133},
doi = {10.48550/arXiv.2406.00133},
year = {2024},
date = {2024-05-01},
urldate = {2024-05-01},
publisher = {arXiv},
abstract = {Predicting the spatiotemporal variation in streamflow along with uncertainty quantification enables decision-making for sustainable management of scarce water resources. Process-based hydrological models (aka physics-based models) are based on physical laws, but using simplifying assumptions which can lead to poor accuracy. Data-driven approaches offer a powerful alternative, but they require large amount of training data and tend to produce predictions that are inconsistent with physical laws. This paper studies a constrained reasoning and learning (CRL) approach where physical laws represented as logical constraints are integrated as a layer in the deep neural network. To address small data setting, we develop a theoretically-grounded training approach to improve the generalization accuracy of deep models. For uncertainty quantification, we combine the synergistic strengths of Gaussian processes (GPs) and deep temporal models (i.e., deep models for time-series forecasting) by passing the learned latent representation as input to a standard distance-based kernel. Experiments on multiple real-world datasets demonstrate the effectiveness of both CRL and GP with deep kernel approaches over strong baseline methods.},
note = {arXiv:2406.00133 [cs]},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning},
pubstate = {published},
tppubtype = {misc}
}
Predicting the spatiotemporal variation in streamflow along with uncertainty quantification enables decision-making for sustainable management of scarce water resources. Process-based hydrological models (aka physics-based models) are based on physical laws, but using simplifying assumptions which can lead to poor accuracy. Data-driven approaches offer a powerful alternative, but they require large amount of training data and tend to produce predictions that are inconsistent with physical laws. This paper studies a constrained reasoning and learning (CRL) approach where physical laws represented as logical constraints are integrated as a layer in the deep neural network. To address small data setting, we develop a theoretically-grounded training approach to improve the generalization accuracy of deep models. For uncertainty quantification, we combine the synergistic strengths of Gaussian processes (GPs) and deep temporal models (i.e., deep models for time-series forecasting) by passing the learned latent representation as input to a standard distance-based kernel. Experiments on multiple real-world datasets demonstrate the effectiveness of both CRL and GP with deep kernel approaches over strong baseline methods.
