publications

2024

1. Dissertation

   Sample-Efficient and Robust Neurosymbolic Learning From Demonstrations

   Aniruddh Gopinath Puranic

   2024

   Abs Bib HTML PDF

   The thesis introduces a novel neurosymbolic learning framework aimed at enhancing the paradigm of learning-from-demonstrations (LfD) to obtain robot control policies. By integrating neural network-based learning with symbolic reasoning, it addresses the challenges posed by imperfect demonstrations and the concerns regarding safety and interpretability in the learned control policies. This innovative approach utilizes Signal Temporal Logic (STL) for expressing high-level robotic tasks and evaluating the quality of demonstrations. STL allows for the creation of non-Markovian rewards and describe causal dependencies between tasks, such as sequential specifications, making it a powerful tool within this framework. Through the LfD-STL framework, reward functions are inferred from suboptimal demonstrations and STL specifications, subsequently utilized by reinforcement learning (RL) algorithms to derive control policies. Extensive experimental evaluations conducted across diverse environments highlight how this framework outperforms prior state-of-the-art LfD methods. In real-world scenarios, robotic tasks often involve multiple objectives, some of which may inherently conflict, necessitating careful trade-offs. To address this challenge, the thesis extends the LfD-STL framework by introducing performance graphs, which are directed graphs that leverage the quality of demonstrations to provide intuitive explanations regarding performance and trade-offs. These performance graphs not only enhance interpretability but also offer valuable insights into the learning process of RL agents, as confirmed by a user study. Furthermore, they serve as an optimization objective in apprenticeship learning (AL), potentially guiding RL agents to learn policies that outperform the (imperfect) demonstrators. The theoretical analysis within the AL-STL framework provides guarantees in the robustness and performance of RL agents.

   @phdthesis{puranic_usc_dissertation2024,
     author = {Puranic, Aniruddh Gopinath},
     title = {Sample-Efficient and Robust Neurosymbolic Learning From Demonstrations},
     school = {University of Southern California},
     year = {2024},
     type = {PhD Dissertation},
     address = {Los Angeles, CA, USA},
   }

2. Conference

   Signal Temporal Logic-Guided Apprenticeship Learning

   Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, and Stefanos Nikolaidis

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), To Appear, 2024

   Abs Bib

   Apprenticeship learning crucially depends on effectively learning rewards, and hence control policies from user demonstrations. Of particular difficulty is the setting where the desired task consists of a number of sub-goals with temporal dependencies. The quality of inferred rewards and hence policies are typically limited by the quality of demonstrations, and poor inference of these can lead to undesirable outcomes. In this letter, we show how temporal logic specifications that describe high level task objectives, are encoded in a graph to define a temporal-based metric that reasons about behaviors of demonstrators and the learner agent to improve the quality of inferred rewards and policies. Through experiments on a diverse set of robot manipulator simulations, we show how our framework overcomes the drawbacks of prior literature by drastically improving the number of demonstrations required to learn a control policy.

   @inproceedings{PDN_ALSTL,
     title = {Signal Temporal Logic-Guided Apprenticeship Learning},
     author = {Puranic, Aniruddh G. and Deshmukh, Jyotirmoy V. and Nikolaidis, Stefanos},
     booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     year = {2024},
     location = {To Appear}
   }

2023

1. Journal

   Learning Performance Graphs From Demonstrations via Task-Based Evaluations

   Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, and Stefanos Nikolaidis

   IEEE Robotics and Automation Letters (RA-L). Oral presentation at ICRA, 2023

   Abs arXiv Bib HTML Supp

   In the paradigm of robot learning-from-demonstrations (LfD), understanding and evaluating the demonstrated behaviors plays a critical role in extracting control policies for robots. Without this knowledge, a robot may infer incorrect reward functions that lead to undesirable or unsafe control policies. Prior work has used temporal logic specifications, manually ranked by human experts based on their importance, to learn reward functions from imperfect/suboptimal demonstrations. To overcome reliance on expert rankings, we propose a novel algorithm that learns from demonstrations, a partial ordering of provided specifications in the form of a performance graph. Through various experiments, including simulation of industrial mobile robots, we show that extracting reward functions with the learned graph results in robot policies similar to those generated with the manually specified orderings. We also show in a user study that the learned orderings match the orderings or rankings by participants for demonstrations in a simulated driving domain. These results show that we can accurately evaluate demonstrations with respect to provided task specifications from a small set of imperfect data with minimal expert input.

   @article{PDN23,
     author = {Puranic, Aniruddh G. and Deshmukh, Jyotirmoy V. and Nikolaidis, Stefanos},
     journal = {IEEE Robotics and Automation Letters (RA-L). Oral presentation at ICRA},
     title = {Learning Performance Graphs From Demonstrations via Task-Based Evaluations},
     year = {2023},
     volume = {8},
     number = {1},
     pages = {336-343},
     doi = {10.1109/LRA.2022.3226072},
   }

2022

1. Poster

   Poster Abstract: Learning from Demonstrations with Temporal Logics

   Aniruddh Puranic, Jyotirmoy Deshmukh, and Stefanos Nikolaidis

   In 25th ACM International Conference on Hybrid Systems: Computation and Control (HSCC), Milan, Italy, 2022

   Abs

   Learning-from-demonstrations (LfD) is a popular paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is susceptible to imperfections in demonstrations and also raises concerns of safety and interpretability in the learned control policies. To address these issues, we propose to use Signal Temporal Logic (STL) to express high-level robotic tasks and use its quantitative semantics to evaluate and rank the quality of demonstrations. Temporal logic-based specifications allow us to create non-Markovian rewards, and are also capable of defining interesting causal dependencies between tasks such as sequential task specifications. We present our completed work that proposed LfD-STL framework that learns from even suboptimal/imperfect demonstrations and STL specifications to infer rewards for reinforcement learning tasks. We have validated our approach through various experimental setups to show how our method outperforms prior LfD methods. We then discuss future directions for tackling the problem of explainability and interpretability in such learning-based systems.

2021

1. Journal

   Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains

   Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, and Stefanos Nikolaidis

   IEEE Robotics and Automation Letters (RA-L). Presented at IROS, 2021

   Abs Bib HTML PDF

   Learning control policies that are safe, robust and interpretable are prominent challenges in developing robotic systems. Learning-from-demonstrations with formal logic is an arising paradigm in reinforcement learning to estimate rewards and extract robot control policies that seek to overcome these challenges. In this approach, we assume that mission-level specifications for the robotic system are expressed in a suitable temporal logic such as Signal Temporal Logic (STL). The main idea is to automatically infer rewards from user demonstrations (that could be suboptimal or incomplete) by evaluating and ranking them w.r.t. the given STL specifications. In contrast to existing work that focuses on deterministic environments and discrete state spaces, in this letter, we propose significant extensions that tackle stochastic environments and continuous state spaces.

   @article{PDN21,
     author = {Puranic, Aniruddh G. and Deshmukh, Jyotirmoy V. and Nikolaidis, Stefanos},
     journal = {IEEE Robotics and Automation Letters (RA-L). Presented at IROS},
     title = {Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains},
     year = {2021},
     volume = {6},
     number = {4},
     pages = {6250-6257},
     doi = {10.1109/LRA.2021.3092676}
   }

2. Conference

   Learning from Demonstrations using Signal Temporal Logic

   Aniruddh Puranic, Jyotirmoy Deshmukh, and Stefanos Nikolaidis

   In Proceedings of the 2020 Conference on Robot Learning (CoRL), 2021

   Abs Bib HTML PDF

   Learning-from-demonstrations is an emerging paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is susceptible to imperfections in demonstrations and also raises concerns of safety and interpretability in the learned control policies. To address these issues, we use Signal Temporal Logic to evaluate and rank the quality of demonstrations. Temporal logic-based specifications allow us to create non-Markovian rewards, and also define interesting causal dependencies between tasks such as sequential task specifications. We validate our approach through experiments on discrete-world and OpenAI Gym environments, and show that our approach outperforms the state-of-the-art Maximum Causal Entropy Inverse Reinforcement Learning.

   @inproceedings{PDN20,
     title = {Learning from Demonstrations using Signal Temporal Logic},
     author = {Puranic, Aniruddh and Deshmukh, Jyotirmoy and Nikolaidis, Stefanos},
     booktitle = {Proceedings of the 2020 Conference on Robot Learning (CoRL)},
     pages = {2228--2242},
     year = {2021},
     volume = {155},
     series = {Proceedings of Machine Learning Research},
     publisher = {PMLR},
     url = {https://proceedings.mlr.press/v155/puranic21a.html},
   }

2020

1. Conference

   Interpretable Classification of Time-Series Data Using Efficient Enumerative Techniques

   Sara Mohammadinejad, Jyotirmoy V. Deshmukh, Aniruddh G. Puranic, and 2 more authors

   In Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control (HSCC), Sydney, New South Wales, Australia, 2020

   Abs Bib

   Cyber-physical system applications such as autonomous vehicles, wearable devices, and avionic systems generate a large volume of time-series data. Designers often look for tools to help classify and categorize the data. Traditional machine learning techniques for time-series data offer several solutions to solve these problems; however, the artifacts trained by these algorithms often lack interpretability. On the other hand, temporal logic, such as Signal Temporal Logic (STL) have been successfully used in the formal methods community as specifications of time-series behaviors. In this work, we propose a new technique to automatically learn temporal logic formulas that are able to classify real-valued time-series data. Previous work on learning STL formulas from data either assumes a formula-template to be given by the user, or assumes some special fragment of STL that enables exploring the formula structure in a systematic fashion. In our technique, we relax these assumptions, and provide a way to systematically explore the space of all STL formulas. As the space of all STL formulas is very large, and contains many semantically equivalent formulas, we suggest a technique to heuristically prune the space of formulas considered. Finally, we illustrate our technique on various case studies from the automotive and transportation domains.

   @inproceedings{HSCC20,
     author = {Mohammadinejad, Sara and Deshmukh, Jyotirmoy V. and Puranic, Aniruddh G. and Vazquez-Chanlatte, Marcell and Donz\'{e}, Alexandre},
     title = {Interpretable Classification of Time-Series Data Using Efficient Enumerative Techniques},
     year = {2020},
     isbn = {9781450370189},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3365365.3382218},
     doi = {10.1145/3365365.3382218},
     booktitle = {Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control (HSCC)},
     articleno = {9},
     numpages = {10},
     keywords = {signal temporal logic, formal methods, machine learning, cyber-physical systems, time-series data, interpretability},
     location = {Sydney, New South Wales, Australia},
     series = {HSCC '20}
   }

2. Conference

   Mining Environment Assumptions for Cyber-Physical System Models

   Sara Mohammadinejad, Jyotirmoy V. Deshmukh, and Aniruddh G. Puranic

   In 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS), 2020

   Bib

   @inproceedings{ICCPS20,
     author = {Mohammadinejad, Sara and Deshmukh, Jyotirmoy V. and Puranic, Aniruddh G.},
     booktitle = {2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS)},
     title = {Mining Environment Assumptions for Cyber-Physical System Models},
     year = {2020},
     volume = {},
     number = {},
     pages = {87-97},
     doi = {10.1109/ICCPS48487.2020.00016}
   }

2019

1. Poster

   MP35-04 Automated Evaluation of Instrument Force Sensitivity During Robotic Suturing Utilizing Vision-Based Machine Learning

   Aniruddh Puranic, Jian Chen, Jessica Nguyen, and 2 more authors

   Journal of Urology, 2019
2. Conference

   Specifying and Evaluating Quality Metrics for Vision-based Perception Systems

   Anand Balakrishnan, Aniruddh G. Puranic, Xin Qin, and 4 more authors

   In 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2019

   Bib

   @inproceedings{DATE19,
     author = {Balakrishnan, Anand and Puranic, Aniruddh G. and Qin, Xin and Dokhanchi, Adel and Deshmukh, Jyotirmoy V. and Ben Amor, Heni and Fainekos, Georgios},
     booktitle = {2019 Design, Automation \& Test in Europe Conference \& Exhibition (DATE)},
     title = {Specifying and Evaluating Quality Metrics for Vision-based Perception Systems},
     year = {2019},
     volume = {},
     number = {},
     pages = {1433-1438},
     doi = {10.23919/DATE.2019.8715114}
   }

2016

1. Journal

   Vehicle Number Plate Recognition System: A Literature Review and Implementation using Template Matching

   Aniruddh Gopinath Puranic, Kanwar Deepak, and V. Umadevi

   International Journal of Computer Applications (IJCA), 2016