I am a Ph.D. student of Computer Science at University of Texas at Austin. I am very honored to be advised by Prof. Peter Stone. I have general interests in reinforcement learning, meta learning, curriculum learning, and their applicaitons in robotics. My primiary focus is to apply deep reinforcement learning for mobile robot autonomous navigation.
Before I started the CS program, I recieved master’s and bachelor’s degrees in Physics, but was lucky to find my passion in artificial intellegence. Feel free to reach out to me if you are curious about my experience and research!
@article{SES-ZIFAN,bibtex_show={false},preprint={true},title={Learning Real-world Autonomous Navigation by Self-Supervised Environment Synthesis},author={Xu, Zifan and Nair, Anirudh and Xiao, Xuesu and Stone, Peter},year={2022},pdf={learning_ses.pdf},figure={ses.png}}
DynaBARN: Benchmarking Metric Ground Navigation in Dynamic Environments
Anirudh Nair,
Fulin Jiang,
Kang Hou,
Zifan Xu,
Shuozhe Li,
Xuesu Xiao,
and Peter Stone
Submitted to International Symposium on Safety, Security, and Rescue Robotics (SSRR) 2023
@article{DynaBARN-ANI,bibtex_show={false},preprint={true},title={DynaBARN: Benchmarking Metric Ground Navigation in Dynamic Environments},author={Nair, Anirudh and Jiang, Fulin and Hou, Kang and Xu, Zifan and Li, Shuozhe and Xiao, Xuesu and and Peter Stone},journal={Submitted to International Symposium on Safety, Security, and Rescue Robotics (SSRR) 2023},year={2022},pdf={dyna_barn.pdf},figure={dyna_barn.png}}
Benchmarking Reinforcement Learning Techniques for Autonomous Navigation
Zifan Xu,
Bo Liu,
Anirudh Nair,
Xuesu Xiao,
and Peter Stone
Submitted to International Conference on Robot Learning (CoRL) 2022
@article{NavBench-ZIFAN,bibtex_show={false},preprint={true},title={Benchmarking Reinforcement Learning Techniques for Autonomous Navigation},author={Xu, Zifan and Liu, Bo and Nair, Anirudh and Xiao, Xuesu and Stone, Peter},journal={Submitted to International Conference on Robot Learning (CoRL) 2022},year={2022},pdf={navbenchmark.pdf},figure={navbenchmark.png}}
Model-Based Meta Automatic Curriculum Learning
Zifan Xu,
Yulin Zhang,
Shahaf S. Shperberg,
Yuqian Jiang,
Reuth Mirsky,
Bo Liu,
and Peter Stone
Submitted to Association for the Advancement of Artificial Intelligence (AAAI) 2023
@article{MM-ACL-ZIFAN,bibtex_show={false},preprint={true},title={Model-Based Meta Automatic Curriculum Learning},author={Xu, Zifan and Zhang, Yulin and Shperberg, Shahaf S. and Jiang, Yuqian and Mirsky, Reuth and Liu, Bo and Stone, Peter},journal={Submitted to Association for the Advancement of Artificial Intelligence (AAAI) 2023},year={2022},pdf={MM-ACL-ZIFAN.pdf},figure={MM-ACL-ZIFAN.png}}
Journal
APPL: Adaptive Planner Parameter Learning
Xuesu Xiao,
Zizhao Wang,
Zifan Xu,
Bo Liu,
Gauraang Dhamankar,
Anirudh Nair,
Garrett Warnell,
and Peter Stone
While current autonomous navigation systems allow robots to successfully drive themselves from one point to another in specific environments, they typically require extensive manual parameter re-tuning by human robotics experts in order to function in new environments. Furthermore, even for just one complex environment, a single set of fine-tuned parameters may not work well in different regions of that environment. These problems prohibit reliable mobile robot deployment by non-expert users. As a remedy, we propose Adaptive Planner Parameter Learning (APPL), a machine learning framework that can leverage non-expert human interaction via several modalities – including teleoperated demonstrations, corrective interventions, and evaluative feedback – and also unsupervised reinforcement learning to learn a parameter policy that can dynamically adjust the parameters of classical navigation systems in response to changes in the environment. APPL inherits safety and explainability from classical navigation systems while also enjoying the benefits of machine learning, i.e., the ability to adapt and improve from experience. We present a suite of individual appl methods and also a unifying cycle-oflearning scheme that combines all the proposed methods in a framework that can improve navigation performance through continual, iterative human interaction and simulation training.
@article{ras22-xiao,bibtex_show={true},is_journal={true},title={APPL: Adaptive Planner Parameter Learning},author={Xiao, Xuesu and Wang, Zizhao and Xu, Zifan and Liu, Bo and abd Gauraang Dhamankar and Nair, Anirudh and Warnell, Garrett and Stone, Peter},journal={Robotics and Autonomous Systems},year={2022},figure={appl.png},pdf={https://www.cs.utexas.edu/users/pstone/Papers/bib2html-links/ras22-xiao.pdf},month=may}
Conference
Causal Dynamics Learning for Task-Independent State Abstraction
Zizhao Wang,
Xuesu Xiao,
Zifan Xu,
Yuke Zhu,
and Peter Stone
In Proceedings of the 39th International Conference on Machine Learning (ICML2022),
Learning dynamics models accurately is an important goal for Model-Based Reinforcement Learning (MBRL), but most MBRL methods learn a dense dynamics model which is vulnerable to spurious correlations and therefore generalizes poorly to unseen states. In this paper, we introduce Causal Dynamics Learning for Task-Independent State Abstraction (CDL), which first learns a theoretically proved causal dynamics model that removes unnecessary dependencies between state variables and the action, thus generalizing well to unseen states. A state abstraction can then be derived from the learned dynamics, which not only improves sample efficiency but also applies to a wider range of tasks than existing state abstraction methods. Evaluated on two simulated environments and downstream tasks, both the dynamics model and policies learned by the proposed method generalize well to unseen states and the derived state abstraction improves sample efficiency compared to learning without it.
@inproceedings{ICML22-wang,bibtex_show={true},conference={true},author={Wang, Zizhao and Xiao, Xuesu and Xu, Zifan and Zhu, Yuke and Stone, Peter},title={Causal Dynamics Learning for Task-Independent State Abstraction},booktitle={Proceedings of the 39th International Conference on Machine Learning (ICML2022)},location={Baltimore, USA},month=jul,year={2022},figure={wang-icml22-cdl.png},pdf={https://www.cs.utexas.edu/users/pstone/Papers/bib2html-links/ICML22-wang.pdf}}
APPLR: Adaptive Planner Parameter Learning from Reinforcement
Zifan Xu,
Gauraang Dhamankar,
Anirudh Nair,
Xuesu Xiao,
Garrett Warnell,
Bo Liu,
Zizhao Wang,
and Peter Stone
In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA 2021),
Classical navigation systems typically operate using a fixed set of hand-picked parameters (e.g. maximum speed, sampling rate, inflation radius, etc.) and require heavy expert re-tuning in order to work in new environments. To mitigate this requirement, it has been proposed to learn parameters for different contexts in a new environment using human demonstrations collected via teleoperation. However, learning from human demonstration limits deployment to the training environment, and limits overall performance to that of a potentially-suboptimal demonstrator. In this paper, we introduce APPLR, Adaptive Planner Parameter Learning from Reinforcement, which allows existing navigation systems to adapt to new scenarios by using a parameter selection scheme discovered via reinforcement learning (RL) in a wide variety of simulation environments. We evaluate APPLR on a robot in both simulated and physical experiments, and show that it can outperform both a fixed set of hand-tuned parameters and also a dynamic parameter tuning scheme learned from human demonstration.
@inproceedings{icra21-xu,bibtex_show={true},conference={true},author={Xu, Zifan and Dhamankar, Gauraang and Nair, Anirudh and Xiao, Xuesu and Warnell, Garrett and Liu, Bo and Wang, Zizhao and Stone, Peter},title={APPLR: Adaptive Planner Parameter Learning from Reinforcement},booktitle={Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA 2021)},location={Xi'an, China},month=jun,year={2021},figure={applr.png},code={https://github.com/Daffan/ros_jackal},video={https://www.youtube.com/watch?v=JKHTAowdGUk&t=26s},pdf={https://www.cs.utexas.edu/users/pstone/Papers/bib2html-links/icra21-xu.pdf},slides={https://www.cs.utexas.edu/users/pstone/Papers/bib2html-links/icra21-xu.slides.pptx},website={https://www.cs.utexas.edu/users/xiao/Research/APPL/APPL.html}}
Machine Learning Methods for Local Motion Planning: A Study of End-to-End vs. Parameter Learning
Zifan Xu,
Xuesu Xiao,
Garrett Warnell,
Anirudh Nair,
and Peter Stone
In Proceedings of the 2021 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR 2021),
While decades of research efforts have been devoted to developing classical autonomous navigation systems to move robots from one point to another in a collision-free manner, machine learning approaches to navigation have been recently proposed to learn navigation behaviors from data. Two representative paradigms are end-to-end learning (directly from perception to motion) and parameter learning (from perception to parameters used by a classical underlying planner). These two types of methods are believed to have complementary pros and cons: parameter learning is expected to be robust to different scenarios, have provable guarantees, and exhibit explainable behaviors; end-to-end learning does not require extensive engineering and has the potential to outperform approaches that rely on classical systems. However, these beliefs have not been verified through real-world experiments in a comprehensive way. In this paper, we report on an extensive study to compare end-to-end and parameter learning for local motion planners in a large suite of simulated and physical experiments. In particular, we test the performance of end-to-end motion policies, which directly compute raw motor commands, and parameter policies, which compute parameters to be used by classical planners, with different inputs (e.g., raw sensor data, costmaps), and provide an analysis of the results.
@inproceedings{ssrr2021-xu,bibtex_show={true},conference={true},author={Xu, Zifan and Xiao, Xuesu and Warnell, Garrett and Nair, Anirudh and Stone, Peter},title={Machine Learning Methods for Local Motion Planning: A Study of End-to-End vs. Parameter Learning},booktitle={Proceedings of the 2021 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR 2021)},location={New York, USA},month=oct,year={2021},figure={ssrr.png},code={https://github.com/Daffan/ros_jackal},video={https://www.youtube.com/watch?v=U592HQ30TsY&t=6s},pdf={https://www.cs.utexas.edu/users/pstone/Papers/bib2html-links/ssrr2021-xu.pdf}}
A Scavenger Hunt for Service Robots
Harel Yedidsion,
Jennifer Suriadinata,
Zifan Xu,
Stefan Debruyn,
and Peter Stone
In Proceedings of the 2021 International Conference on Robotics and Automation (ICRA 2021),
Creating robots that can perform general-purpose service tasks in a human-populated environment has been a longstanding grand challenge for AI and Robotics research.
One particularly valuable skill that is relevant to a wide variety of tasks is the ability to locate and retrieve objects upon request.
This paper models this skill as a Scavenger Hunt (SH) game, which we formulate as a variation of the NP-hard stochastic traveling purchaser problem. In this problem, the goal is to find a set of objects as quickly as possible, given probability distributions of where they may be found.
We investigate the performance of several solution algorithms for the SH problem, both in simulation and on a real mobile robot. We use Reinforcement Learning (RL) to train an agent to plan a minimal cost path, and show that the RL agent can outperform a range of heuristic algorithms, achieving near optimal performance.
In order to stimulate research on this problem, we introduce a publicly available software stack and associated website that enable users to upload scavenger hunts which robots can download, perform, and learn from to continually improve their performance on future hunts.
@inproceedings{ICRA21-Yedidsion,bibtex_show={true},conference={true},author={Yedidsion, Harel and Suriadinata, Jennifer and Xu, Zifan and Debruyn, Stefan and Stone, Peter},title={A Scavenger Hunt for Service Robots},booktitle={Proceedings of the 2021 International Conference on Robotics and Automation (ICRA 2021)},location={Xi'an China},month=may,year={2021},figure={scavenger_hunt.png},code={https://github.com/Daffan/scavenger_hunt_sim},video={http://scavenger-hunt.cs.utexas.edu/},pdf={https://www.cs.utexas.edu/users/pstone/Papers/bib2html-links/ICRA21-Yedidsion.pdf},website={http://scavenger-hunt.cs.utexas.edu/}}
Workshop / Extended Abstract / Technical Report
Task Factorization in Curriculum Learning
Reuth Mirsky,
Shahaf S. Shperberg,
Yulin Zhang,
Zifan Xu,
Yuqian Jiang,
Jiaxun Cui,
and Peter Stone
In Decision Awareness in Reinforcement Learning (DARL) workshop at the 39th International Conference on Machine Learning (ICML),
A common challenge for learning when applied to a complex “target” task is that learning that task all at once can be too difficult due to inefficient exploration given a sparse reward signal. Curriculum Learning addresses this challenge by sequencing training tasks for a learner to facilitate gradual learning. One of the crucial steps in finding a suitable curriculum learning approach is to understand the dimensions along which the domain can be factorized. In this paper, we identify different types of factorizations common in the literature of curriculum learning for reinforcement learning tasks: factorizations that involve the agent, the environment, or the mission. For each factorization category, we identify the relevant algorithms and techniques that leverage that factorization and present several case studies to showcase how leveraging an appropriate factorization can boost learning using a simple curriculum.
@inproceedings{DARL22-REUTH,is_other={true},bibtex_show={true},author={Mirsky, Reuth and Shperberg, Shahaf S. and Zhang, Yulin and Xu, Zifan and Jiang, Yuqian and Cui, Jiaxun and Stone, Peter},title={Task Factorization in Curriculum Learning},booktitle={Decision Awareness in Reinforcement Learning (DARL) workshop at the 39th International Conference on Machine Learning (ICML)},location={Baltimore, Maryland, USA},month=jul,year={2022},figure={DARL22-REUTH.png},pdf={https://www.cs.utexas.edu/users/pstone/Papers/bib2html-links/DARL22-REUTH.pdf}}
GOLD-FACTUAL: Learning to Generate Faithful Summaries from Models’ Generations
@article{GOLD-FACTUAL-LIYAN,is_other={true},bibtex_show={true},author={Xu, Zifan and Tang, Liyan},title={GOLD-FACTUAL: Learning to Generate Faithful Summaries from Models’ Generations},journal={CS394R Final Project},figure={GOLD-FACTUAL-LIYAN.png},pdf={GOLD-FACTUAL-LIYAN.pdf}}
Model-Based Meta Automatic Curriculum Learning
Zifan Xu,
Yulin Zhang,
Shahaf S. Shperberg,
Reuth Mirsky,
Yulin Zhan,
Yuqian Jiang,
Bo Liu,
and Peter Stone
In Decision Awareness in Reinforcement Learning (DARL) workshop at the 39th International Conference on Machine Learning (ICML),
When an agent trains for one target task, its experience is expected to be useful for training on another target task. This paper formulates the meta curriculum learning problem that builds a sequence of intermediate training tasks, called a curriculum, which will assist the learner to train toward any given target task in general. We propose a model-based meta automatic curriculum learning algorithm (MM-ACL) that learns to predict the performance improvement on one task when the policy is trained on another, given contextual information such as the history of training tasks, loss functions, rollout state-action trajectories from the policy, etc. This predictor facilitates the generation of curricula that optimizes the performance of the learner on different target tasks. Our empirical results demonstrate that MM-ACL outperforms a random curriculum, a manually created curriculum, and a commonly used non-stationary bandit algorithm in a GridWorld domain.
@inproceedings{DARL22-ZIFAN,is_other={true},bibtex_show={true},author={Xu, Zifan and Zhang, Yulin and Shperberg, Shahaf S. and Mirsky, Reuth and Zhan, Yulin and Jiang, Yuqian and Liu, Bo and Stone, Peter},title={Model-Based Meta Automatic Curriculum Learning},booktitle={Decision Awareness in Reinforcement Learning (DARL) workshop at the 39th International Conference on Machine Learning (ICML)},location={Baltimore, Maryland, USA},month=jul,year={2022},figure={DARL22-ZIFAN.png},pdf={https://www.cs.utexas.edu/users/pstone/Papers/bib2html-links/DARL22-ZIFAN.pdf}}
