publications | Aniruddh Puranic

2023

arXiv
Signal Temporal Logic-Guided Apprenticeship Learning

Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, and Stefanos Nikolaidis

2023

Abs arXiv Bib

Apprenticeship learning crucially depends on effectively learning rewards, and hence control policies from user demonstrations. Of particular difficulty is the setting where the desired task consists of a number of sub-goals with temporal dependencies. The quality of inferred rewards and hence policies are typically limited by the quality of demonstrations, and poor inference of these can lead to undesirable outcomes. In this letter, we show how temporal logic specifications that describe high level task objectives, are encoded in a graph to define a temporal-based metric that reasons about behaviors of demonstrators and the learner agent to improve the quality of inferred rewards and policies. Through experiments on a diverse set of robot manipulator simulations, we show how our framework overcomes the drawbacks of prior literature by drastically improving the number of demonstrations required to learn a control policy.
@misc{PDN_ALSTL, title = {Signal Temporal Logic-Guided Apprenticeship Learning}, author = {Puranic, Aniruddh G. and Deshmukh, Jyotirmoy V. and Nikolaidis, Stefanos}, year = {2023}, eprint = {2311.05084}, archiveprefix = {arXiv}, primaryclass = {cs.RO} }
RA-L
Learning Performance Graphs From Demonstrations via Task-Based Evaluations

Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, and Stefanos Nikolaidis

IEEE Robotics and Automation Letters (RA-L). Oral presentation at ICRA, 2023

Abs arXiv Bib HTML Supp

In the paradigm of robot learning-from-demonstrations (LfD), understanding and evaluating the demonstrated behaviors plays a critical role in extracting control policies for robots. Without this knowledge, a robot may infer incorrect reward functions that lead to undesirable or unsafe control policies. Prior work has used temporal logic specifications, manually ranked by human experts based on their importance, to learn reward functions from imperfect/suboptimal demonstrations. To overcome reliance on expert rankings, we propose a novel algorithm that learns from demonstrations, a partial ordering of provided specifications in the form of a performance graph. Through various experiments, including simulation of industrial mobile robots, we show that extracting reward functions with the learned graph results in robot policies similar to those generated with the manually specified orderings. We also show in a user study that the learned orderings match the orderings or rankings by participants for demonstrations in a simulated driving domain. These results show that we can accurately evaluate demonstrations with respect to provided task specifications from a small set of imperfect data with minimal expert input.
@article{PDN23, author = {Puranic, Aniruddh G. and Deshmukh, Jyotirmoy V. and Nikolaidis, Stefanos}, journal = {IEEE Robotics and Automation Letters (RA-L). Oral presentation at ICRA}, title = {Learning Performance Graphs From Demonstrations via Task-Based Evaluations}, year = {2023}, volume = {8}, number = {1}, pages = {336-343}, doi = {10.1109/LRA.2022.3226072}, }

2022

HSCC

Poster Abstract: Learning from Demonstrations with Temporal Logics

Aniruddh Puranic, Jyotirmoy Deshmukh, and Stefanos Nikolaidis

In 25th ACM International Conference on Hybrid Systems: Computation and Control, 2022

Abs

Learning-from-demonstrations (LfD) is a popular paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is susceptible to imperfections in demonstrations and also raises concerns of safety and interpretability in the learned control policies. To address these issues, we propose to use Signal Temporal Logic (STL) to express high-level robotic tasks and use its quantitative semantics to evaluate and rank the quality of demonstrations. Temporal logic-based specifications allow us to create non-Markovian rewards, and are also capable of defining interesting causal dependencies between tasks such as sequential task specifications. We present our completed work that proposed LfD-STL framework that learns from even suboptimal/imperfect demonstrations and STL specifications to infer rewards for reinforcement learning tasks. We have validated our approach through various experimental setups to show how our method outperforms prior LfD methods. We then discuss future directions for tackling the problem of explainability and interpretability in such learning-based systems.

2021

RA-L
Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains

Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, and Stefanos Nikolaidis

IEEE Robotics and Automation Letters (RA-L). Presented at IROS, 2021

Abs Bib HTML PDF

Learning control policies that are safe, robust and interpretable are prominent challenges in developing robotic systems. Learning-from-demonstrations with formal logic is an arising paradigm in reinforcement learning to estimate rewards and extract robot control policies that seek to overcome these challenges. In this approach, we assume that mission-level specifications for the robotic system are expressed in a suitable temporal logic such as Signal Temporal Logic (STL). The main idea is to automatically infer rewards from user demonstrations (that could be suboptimal or incomplete) by evaluating and ranking them w.r.t. the given STL specifications. In contrast to existing work that focuses on deterministic environments and discrete state spaces, in this letter, we propose significant extensions that tackle stochastic environments and continuous state spaces.
@article{PDN21, author = {Puranic, Aniruddh G. and Deshmukh, Jyotirmoy V. and Nikolaidis, Stefanos}, journal = {IEEE Robotics and Automation Letters (RA-L). Presented at IROS}, title = {Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains}, year = {2021}, volume = {6}, number = {4}, pages = {6250-6257}, doi = {10.1109/LRA.2021.3092676} }
CoRL
Learning from Demonstrations using Signal Temporal Logic

Aniruddh Puranic, Jyotirmoy Deshmukh, and Stefanos Nikolaidis

In Proceedings of the 2020 Conference on Robot Learning, 2021

Abs Bib HTML PDF

Learning-from-demonstrations is an emerging paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is susceptible to imperfections in demonstrations and also raises concerns of safety and interpretability in the learned control policies. To address these issues, we use Signal Temporal Logic to evaluate and rank the quality of demonstrations. Temporal logic-based specifications allow us to create non-Markovian rewards, and also define interesting causal dependencies between tasks such as sequential task specifications. We validate our approach through experiments on discrete-world and OpenAI Gym environments, and show that our approach outperforms the state-of-the-art Maximum Causal Entropy Inverse Reinforcement Learning.
@inproceedings{PDN20, title = {Learning from Demonstrations using Signal Temporal Logic}, author = {Puranic, Aniruddh and Deshmukh, Jyotirmoy and Nikolaidis, Stefanos}, booktitle = {Proceedings of the 2020 Conference on Robot Learning}, pages = {2228--2242}, year = {2021}, volume = {155}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, url = {https://proceedings.mlr.press/v155/puranic21a.html}, }

2020

HSCC
Interpretable Classification of Time-Series Data Using Efficient Enumerative Techniques

Sara Mohammadinejad, Jyotirmoy V. Deshmukh, Aniruddh G. Puranic, and 2 more authors

In Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control, 2020

Abs Bib

Cyber-physical system applications such as autonomous vehicles, wearable devices, and avionic systems generate a large volume of time-series data. Designers often look for tools to help classify and categorize the data. Traditional machine learning techniques for time-series data offer several solutions to solve these problems; however, the artifacts trained by these algorithms often lack interpretability. On the other hand, temporal logic, such as Signal Temporal Logic (STL) have been successfully used in the formal methods community as specifications of time-series behaviors. In this work, we propose a new technique to automatically learn temporal logic formulas that are able to classify real-valued time-series data. Previous work on learning STL formulas from data either assumes a formula-template to be given by the user, or assumes some special fragment of STL that enables exploring the formula structure in a systematic fashion. In our technique, we relax these assumptions, and provide a way to systematically explore the space of all STL formulas. As the space of all STL formulas is very large, and contains many semantically equivalent formulas, we suggest a technique to heuristically prune the space of formulas considered. Finally, we illustrate our technique on various case studies from the automotive and transportation domains.
@inproceedings{HSCC20, author = {Mohammadinejad, Sara and Deshmukh, Jyotirmoy V. and Puranic, Aniruddh G. and Vazquez-Chanlatte, Marcell and Donz\'{e}, Alexandre}, title = {Interpretable Classification of Time-Series Data Using Efficient Enumerative Techniques}, year = {2020}, isbn = {9781450370189}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3365365.3382218}, doi = {10.1145/3365365.3382218}, booktitle = {Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control}, articleno = {9}, numpages = {10}, keywords = {signal temporal logic, formal methods, machine learning, cyber-physical systems, time-series data, interpretability}, location = {Sydney, New South Wales, Australia}, series = {HSCC '20} }

ICCPS

Mining Environment Assumptions for Cyber-Physical System Models

Sara Mohammadinejad, Jyotirmoy V. Deshmukh, and Aniruddh G. Puranic

In 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS), 2020

Bib

@inproceedings{ICCPS20,
  author = {Mohammadinejad, Sara and Deshmukh, Jyotirmoy V. and Puranic, Aniruddh G.},
  booktitle = {2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS)},
  title = {Mining Environment Assumptions for Cyber-Physical System Models},
  year = {2020},
  volume = {},
  number = {},
  pages = {87-97},
  doi = {10.1109/ICCPS48487.2020.00016}
}

2019

MP35-04 AUTOMATED EVALUATION OF INSTRUMENT FORCE SENSITIVITY DURING ROBOTIC SUTURING UTILIZING VISION-BASED MACHINE LEARNING

Aniruddh Puranic, Jian Chen*, Jessica Nguyen, and 2 more authors

Journal of Urology, 2019

DATE

Specifying and Evaluating Quality Metrics for Vision-based Perception Systems

Anand Balakrishnan, Aniruddh G. Puranic, Xin Qin, and 4 more authors

In 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2019

Bib

@inproceedings{DATE19,
  author = {Balakrishnan, Anand and Puranic, Aniruddh G. and Qin, Xin and Dokhanchi, Adel and Deshmukh, Jyotirmoy V. and Ben Amor, Heni and Fainekos, Georgios},
  booktitle = {2019 Design, Automation \& Test in Europe Conference \& Exhibition (DATE)},
  title = {Specifying and Evaluating Quality Metrics for Vision-based Perception Systems},
  year = {2019},
  volume = {},
  number = {},
  pages = {1433-1438},
  doi = {10.23919/DATE.2019.8715114}
}

2016

Vehicle Number Plate Recognition System: A Literature Review and Implementation using Template Matching

Aniruddh Gopinath Puranic, Kanwar Deepak, and V. Umadevi

International Journal of Computer Applications, 2016