Zuxin Liu

Email: zuxin1997 [at] gmail [dot] com

I am a Research Scientist at Salesforce AI Research, working on LLM Agent. I received my PhD and MS degrees from Carnegie Mellon University, focusing on machine learning and robotics. During my PhD, I work closely with Google DeepMind, AWS AI, and Nuro. Before that, I finished my bachelor's degree with honor (President's Award) from Beihang University.

News & Updates

[2024/10] Our APIGen paper is accepted by NeurIPS 2024!
[2024/09] Check out our xLAM blog post and Technical Report Paper for insights into our Salesforce's Large Action Models.
[2024/08] We are thrilled to announce the release of the entire xLAM family, our suite of Large Action Models! From the "tiny giant" 1B model to industrial powerhouses 8x22B model. These models have achieved impressive rankings, placing #1 and #6 on the Berkeley Function-Calling Leaderboard. Explore our Hugging Face collection for more details.
[2024/07] We are excited to announce the release of our two function-calling models: xLAM-1b-fc-r and xLAM-7b-fc-r. These models have achieved impressive rankings, placing #3 and #25 on the Berkeley Function-Calling Leaderboard, outperforming many significantly larger models.
[2024/06] Check our latest work APIGen, the best open-sourced models for function calling. Our dataset is currently among the Top-3 trending datasets on HuggingFace as of July 4, 2024. See also the Twitter by Salesforce CEO, VentureBeat and 新智元.
[2024/05] Our paper for efficient and safe RL is accepted by ICML 2024!
[2024/04] Our RL dataset and benchmark paper is accepted by DMLR Journal! Checkout the website for details!
[2024/02] We release our multi-LLM-Agent framework AgentLite library and paper!
[2024/01] I joined Salesforce AI Research as a Research Scientist! Looking forward to working with the amazing team members on LLM Agent!
[2024/01] Our two papers, one about efficient foundation model adaptation, and one about offline RL, are accepted by ICLR 2024!
[2024/01] Our paper about robustness certification is accepted by AISTATS 2024!
[2023/09] Our two papers for safe RL, one about versatile policy learning, and one about inverse constraint learning, are accepted by NeurIPS 2023!
[2023/06] Our comprehensive datasets, benchmarks, and algorithms for offline safe learning are released! Checkout our website for details!
[2023/05] A fast safe reinforcement learning framework is released! Checkout our GitHub repo for details!
[2023/04] Our two papers for safe RL, one about robustness and one about offline learning, are accepted by ICML 2023!
[2023/01] Our paper about observational robustness in safe RL is accepted by ICLR 2023!
[2022/12] Our paper about robustness in safe RL win the AI Risk Analysis Award at the 2022 NeurIPS ML Safety Workshop!
[2022/09] Our work about robustness certification in visual perception is accepted by CoRL 2022.
[2022/09] Our work about safety evaluation for self-driving vehicles is accepted by NeurIPS 2022.
[2022/07] Our paper about robustness in safe RL win the best paper runner-up in the SL4AD Workshop at ICML 2022!
[2022/07] I am glad to present my work about safe RL at Google DeepMind robotics team.
[2022/05] Our paper about variational inference approach for off-policy safe RL is accepted by ICML 2022!
[2022/05] I give a talk about recent advances in safe RL at Prof. Fei Fang's lab.
[2022/04] Our work about safe learning for delivery robot is featured on the front page of CMU news!
[2022/03] Our paper about LiDAR sensing in autonomous vehicle is accepted by CVPR 2022!
[2021/11] The autonomous delivery robot that we have built for one year is featured by CMU Engineering.
[2021/07] We win the Hackathon during my intern at Nuro! Really enjoyed to solve challenging real-world problems for self-driving.

Research Interests

My long-term ambition is to develop AI agents capable of achieving and surpassing human-level performance in various daily tasks, ultimately freeing humans from repetitive work and enhancing productivity. Beyond the commonly recognized capabilities like reasoning and planning, I believe that continual self-evolution (in terms of training) and self-reflection (during deployment) are also essential traits of truly intelligent agents. This aligns with the core principles of reinforcement learning (RL), which I view as a guiding philosophical framework for thinking and studying AI agents.

My research aims to apply the foundational principles—not just the methods—of RL to large language model (LLM)-based agents, contributing to the promising future of AI Agent systems that that evolve, learn, and interact in ways that complement and enhance human capabilities. Currently, I am developing scalable approaches, such as utilizing synthetic data, to improve models' agentic abilities, and leveraging environmental feedback to better enable self-learning and reflection, as exemplified like the Software Engineering Agent.

Publications ( show selected / show all by date / show all by topic )

(* indicates equal contribution.)

Topics: LLM Agent / Foundation Model / RL Algorithms
Past topics: Embodied AI & Robotics / Computer Vision & Autonomous Vehicles / Fun Undergrads Projects

APIGen: Automated PIpeline for Generating Verifiable and Diverse Function-calling Datasets

Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh Murthy, Liangwei Yang, Silvio Savarese, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, Caiming Xiong

NeurIPS 2024 Dataset and Benchmark Track | Paper | Website | Dataset | Model

xLAM: A Family of Large Action Models to Empower AI Agent Systems

Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang*, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, Zhiwei Liu, Yihao Feng, et. al.

Paper | Website | Model

Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents

Kexun Zhang, Weiran Yao, Zuxin Liu, Yihao Feng, Zhiwei Liu, Rithesh Murthy, Tian Lan, Lei Li, Renze Lou, Jiacheng Xu, Bo Pang, Yingbo Zhou, Shelby Heinecke, Silvio Savarese, Huan Wang, Caiming Xiong

Paper | Website | Code

TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models

Zuxin Liu, Jesse Zhang, Kavosh Asadi, Yao Liu, Ding Zhao, Shoham Sabach, Rasool Fakoor.

ICLR 2024 | Paper

AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System

Zhiwei Liu, Weiran Yao, Jianguo Zhang, Liangwei Yang, Zuxin Liu, Juntao Tan, Prafulla K Choubey, Tian Lan, Jason Wu, Huan Wang, Shelby Heinecke, Caiming Xiong, Silvio Savarese

Paper | Code

AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning

Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Juntao Tan, Thai Hoang, Liangwei Yang, Yihao Feng, Zuxin Liu, Tulika Awalgaonkar, Juan Carlos Niebles, Silvio Savarese, Shelby Heinecke, Huan Wang, Caiming Xiong

Paper | Code

Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving

Haohong Lin, Wenhao Ding, Zuxin Liu, Yaru Niu, Jiacheng Zhu, Yuming Niu, Ding Zhao

IEEE Robotics and Automation Letters | Paper | Website

Gradient Shaping for Multi-Constraint Safe Reinforcement Learning

Yihang Yao, Zuxin Liu, Zhepeng Cen, Peide Huang, Tingnan Zhang, Wenhao Yu, Ding Zhao

L4DC 2024 | Paper

Feasibility Consistent Representation Learning for Safe Reinforcement Learning

Zhepeng Cen, Yihang Yao, Zuxin Liu, Ding Zhao

ICML 2024 | Paper | Website | Code

Learning from Sparse Offline Datasets via Conservative Density Estimation

Zhepeng Cen, Zuxin Liu, Zitong Wang, Yihang Yao, Henry Lam, Ding Zhao.

ICLR 2024 | Paper

EXTRACT: Efficient Policy Learning by Extracting Transferrable Robot Skills from Offline Data

Jesse Zhang, Minho Heo, Zuxin Liu, Erdem Biyik, Joseph J Lim, Yao Liu, Rasool Fakoor

CoRL 2024 | Paper | Website

Reinforcement Learning in a Safety-Embedded MDP with Trajectory Optimization

Fan Yang, Wenxuan Zhou, Zuxin Liu, Ding Zhao, David Held

ICRA 2024 | Paper | Website | Code

Datasets and Benchmarks for Offline Safe Reinforcement Learning

Zuxin Liu, Zijian Guo, Haohong Lin, Yihang Yao, Jiacheng Zhu, Zhepeng Cen, Hanjiang Hu, Wenhao Yu, Tingnan Zhang, Jie Tan, Ding Zhao.

Constrained Decision Transformer for Offline Safe Reinforcement Learning

Zuxin Liu, Zijian Guo, Yihang Yao, Zhepeng Cen, Wenhao Yu, Tingnan Zhang, Ding Zhao.

ICML 2023 | Paper | Code

Towards Robust and Safe Reinforcement Learning with Benign Off-policy Data

Zuxin Liu, Zijian Guo, Zhepeng Cen, Yihang Yao, Hanjiang Hu, Ding Zhao.

ICML 2023 | Paper

Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learning

Yihang Yao, Zuxin Liu, Zhepeng Cen, Jiacheng Zhu, Wenhao Yu, Tingnan Zhang, Ding Zhao.

NeurIPS 2023 | Paper

Learning Shared Safety Constraints from Multi-task Demonstrations

Konwoo Kim, Gokul Swamy, Zuxin Liu, Ding Zhao, Sanjiban Choudhury, Zhiwei Steven Wu.

NeurIPS 2023 | Paper

Seasondepth: Cross-season Monocular Depth Prediction Dataset and Benchmark under Multiple Environments

Hanjiang Hu, Baoquan Yang, Zhijian Qiao, Shiqi Liu, Jiacheng Zhu, Zuxin Liu, Wenhao Ding, Ding Zhao, Hesheng Wang

IROS 2023 | Paper | Website | Code

Pixel-wise Smoothing for Certified Robustness against Camera Motion Perturbations

Hanjiang Hu, Zuxin Liu, Linyi Li, Jiacheng Zhu, Ding Zhao

AISTATS 2023 | Paper | Code

On the Robustness of Safe Reinforcement Learning under Observational Perturbations

Zuxin Liu, Zijian Guo, Zhepeng Cen, Huan Zhang, Jie Tan, Bo Li, Ding Zhao.

ICLR 2023 | Paper
2022 ICML SL4AD Workshop (Best Paper Runner-up)
2022 NeurIPS ML Safety Workshop (AI Risk Analysis Award)
Paper | Website | Code

Learning to Explore (L2E): Deep Reinforcement Learning-based Autonomous Exploration for Household Robot

Zuxin Liu, Mohit Deshpande, Xuewei Qi, Ding Zhao, Rajasimman Madhivanan, Arnie Sen.

RSS Robot Representations Workshop | Paper

Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalization

Mengdi Xu, Zuxin Liu, Peide Huang*, Wenhao Ding, Zhepeng Cen, Bo Li, Ding Zhao.

Survey | Paper

Constrained Variational Policy Optimization for Safe Reinforcement Learning

Zuxin Liu, Zhepeng Cen, Vladislav Isenbaev, Wei Liu, Steven Wu, Bo Li, Ding Zhao.

ICML 2022 | Paper | Code

Robustness Certification of Visual Perception Models via Camera Motion Smoothing

Hanjiang Hu, Zuxin Liu, Linyi Li, Jiacheng Zhu, Ding Zhao.

CoRL 2022 | Paper | Code

SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehicles

Chejian Xu, Wenhao Ding, Weijie Lyu, Zuxin Liu, Shuai Wang, Yihan He, Hanjiang Hu, Ding Zhao, Bo Li.

NeurIPS 2022 | Paper | Website

Investigating the Impact of Multi-LiDAR Placement on Object Detection for Autonomous Driving

Hanjiang Hu^, Zuxin Liu^, Sharad Chitlangia, Akhil Agnihotri, Ding Zhao.

CVPR 2022 | Paper | Code

Context-Aware Safe Reinforcement Learning in Non-Stationary Environments

Baiming Chen, Zuxin Liu, Jiacheng Zhu, Mengdi Xu, Wenhao Ding, Ding Zhao.

ICRA 2021 | Paper | Code

MAPPER: Multi-Agent Path Planning with Evolutionary Reinforcement Learning in Mixed Dynamic Environments

Zuxin Liu, Baiming Chen, Hongyi Zhou, Guru Koushik, Martial Hebert, Ding Zhao.

IROS 2020 | Paper

Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes

Mengdi Xu, Wenhao Ding, Jiacheng Zhu, Zuxin Liu, Baiming Chen, Ding Zhao.

NeurIPS 2020 | Paper | Code

Where Should We Place LiDARs on the Autonomous Vehicle? - An Optimal Design Approach

Zuxin Liu, Mansur Arief, Ding Zhao.

ICRA 2019 | Paper | Code

DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments

Chao Yu, Zuxin Liu, Fei Qiao, et al.

IROS 2018 (ranked as the most popular presentation by INFOVAYA)
Paper | Code

Robot’s Eyes and Brain: Visual Semantic SLAM System, 2018

Our visual semantic SLAM system empowers robots to construct detailed semantic maps of their surroundings, identifying and remembering the locations of objects. With voice command capabilities, the robot can efficiently locate and autonomously navigate to specific items. This project secured the top prize at the 2018 ICOPEN 3D Sensor Application Design Competition, standing out among 20 teams worldwide. | Video

VR Multicopter System, 2017

Ever wanted to experience the sensation of flying? Our VR Multicopter system offers an immersive flying experience. By using a VR device, users can control the orientation of a gimbal mounted on our multicopter. The stereo camera on the gimbal streams real-time video back to the VR equipment, allowing you to simply move your head and enjoy breathtaking aerial views. This project earned the 1st prize at the 2017 International Design and Innovation Competition among 14 global teams. | Video

Autonomous Navigation Robot, 2017

I led a team to build a mobile robot platform which could achieve autonomous navigation and obstacle avoidance based on RTAB-Map SLAM and ROS Navigation Stack. Just with one-click, the robot can autonomously navigate to wherever you want.
Video

Automatic AI Robot System, 2017

This project is designed for ICRA DJI Robotmaster AI challenge. The robots are required to autonomously find enemy robots and hit them (shoot rubber ball). More exciting information and videos about this robot platform and the relevant robot competition can be found here.

Arduino-based Interactive Facial Expression Robot, 2016

This cute robot could make different expressions according to user’s voice command. The movements of its eyes, eyebrow, and mouth etc are fully controlled by servo motors. Most of the materials are 3D printed.