Assistant Professor
IIIS, Tsinghua University
MMW Building S-221
100084, Beijing, China

+8610-62773713 Ext. 6221
chongjie at tsinghua.edu.cn

I am an Assistant Professor in the Institute for Interdisciplinary Information Sciences (IIIS) at Tsinghua University. I am leading the Machine Intelligence Group.

Before joining the faculty, I was a postdoctoral associate at MIT working with Julie Shah and the Interactive Robotics group in Computer Science and Artificial Intelligence Lab (CSAIL) on studying robotics with applications on manufacturing. I received my Ph.D. in Computer Science under the supervision of Victor Lesser from University of Massachusetts at Amherst in 2011.

My group is now recruiting self-motivated Ph.D. and Master students. We also have openings for research interns and post-docs in the areas related to Deep Reinforcement Learning, Multi-Agent Systems, and Robotics.

My research focuses on a fundamental aspect of artificial intelligence -- how intelligent agents learn to make decisions and perform actions in order to achieve their goals.

To develop a general-purpose learning framework, I am interested in studying topics including, but not limited to, representation learning, world-model learning and reasoning, policy learning and planning, multi-agent learning.

Specifically, my goal is to build autonomous agents equipped with the capability of extracting information from perceptions, learning and reasoning world models that are generalized over tasks, predicting consequences of actions, envisioning and setting achievable goals, planning actions to achieve these goals, collaborating with other agents or humans to perform complex tasks, as well as generalizing learned knowledge and sharing learning experiences with other agents.

To approach my goal, my current research is building novel efficient models and methods of deep reinforcement learning, multi-agent systems, human-AI interaction, and particularly of their intersections to enable agents to learn to collaborate with other agents or humans and accomplish beyond what they can do alone..

• Flow to Control: Offline Reinforcement Learning with Lossless Primitive Discovery.
  Yiqin Yang, Hao Hu, Wenzhe Li, Siyuan Li, Jun Yang, Qianchuan Zhao, Chongjie Zhang.
  AAAI Conference on Artificial Intelligence (AAAI), 2023.
  
• Low-Rank Modular Reinforcement Learning via Muscle Synergy.
  Heng Dong*, Tonghan Wang*, Jiayuan Liu, Chongjie Zhang.
  Advances in Neural Information Processing Systems (NeurIPS), 2022.
  
• CUP: Critic-Guided Policy Reuse.
  Jin Zhang, Siyuan Li, Chongjie Zhang.
  Advances in Neural Information Processing Systems (NeurIPS), 2022. [Spotlight Paper]
  
• Latent-Variable Advantage-Weighted Policy Optimization for Offline Reinforcement Learning.
  Xi Chen, Ali Ghadirzadeh, Tianhe Yu, Yuan Gao, Jianhao Wang, Wenzhe Li, Liang Bin, Chelsea Finn, Chongjie Zhang.
  Advances in Neural Information Processing Systems (NeurIPS), 2022.
  
• RORL: Robust Offline Reinforcement Learning via Conservative Smoothing.
  Rui Yang, Chenjia Bai, Xiaoteng Ma, Zhaoran Wang, Chongjie Zhang, Lei Han.
  Advances in Neural Information Processing Systems (NeurIPS), 2022. [Spotlight Paper]
  
• Non-Linear Coordination Graphs.
  Yipeng Kang*, Tonghan Wang*, Qianlan Yang, Xiaoran Wu, Chongjie Zhang.
  Advances in Neural Information Processing Systems (NeurIPS), 2022. [Spotlight Paper]
  
• Safe Opponent-Exploitation Subgame Refinement.
  Mingyang Liu*, Chengjie Wu*, Qihan Liu, Yansen Jing, Jun Yang, Pingzhong Tang, Chongjie Zhang.
  Advances in Neural Information Processing Systems (NeurIPS), 2022.
  
• On the Role of Discount Factor in Offline Reinforcement Learning.
  Hao Hu*, Yiqin Yang*, Qianchuan Zhao, Chongjie Zhang.
  International Conference on Machine Learning (ICML), 2022.
  
• Self-Organized Polynomial-Time Coordination Graphs.
  Qianlan Yang*, Weijun Dong*, Zhizhou Ren*, Jianhao Wang, Tonghan Wang, Chongjie Zhang.
  International Conference on Machine Learning (ICML), 2022.
  
• Individual Reward Assisted Multi-Agent Reinforcement Learning.
  Li Wang, Yujing Hu, Yupeng Zhang, Weixun Wang, Chongjie Zhang, Yang Gao, Jianye Hao, Tangjie Lv, Changjie Fan.
  International Conference on Machine Learning (ICML), 2022.
  
• Multi-Agent Concentrative Coordination with Decentralized Task Representation.
  Lei Yuan*, Chenghe Wang*, Jianhao Wang, Fuxiang Zhang, Feng Chen, Cong Guan, Zongzhang Zhang, Chongjie Zhang, Yang Yu.
  International Joint Conference on Artificial Intelligence (IJCAI), 2022.
  
• LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammates.
  iahan Cao*, Lei Yuan*, Jianhao Wang, Shaowei Zhang, Chongjie Zhang, Yang Yu, De-Chuan Zhan.
  SCIENCE CHINA Information Sciences, 2022.
  
• Safe Opponent-Exploitation Subgame Refinement.
  Mingyang Liu, Chengjie Wu, Qihan Liu, Yansen Jing, Jun Yang, Pingzhong Tang, Chongjie Zhang.
  ICLR Workshop on Gamification and Multiagent Solutions, 2022.
  [PDF]
• Active Hierarchical Exploration with Stable Subgoal Representation Learning.
  Siyuan Li, Jin Zhang, Jianhao Wang, Yang Yu, Chongjie Zhang.
  International Conference on Learning Representations (ICLR) , 2022.
  [PDF]
• Context-Aware Sparse Deep Coordination Graphs.
  Tonghan Wang*, Liang Zeng*, Weijun Dong, Qianlan Yang, Yang Yu, Chongjie Zhang.
  International Conference on Learning Representations (ICLR) , 2022. [Spotlight Paper]
  [PDF]
• Rethinking Goal-Conditioned Supervised Learning and Its Connection to Offline RL.
  Rui Yang, Yiming Lu, Wenzhe Li, Hao Sun, Meng Fang, Yali Du, Xiu Li, Lei Han, Chongjie Zhang.
  International Conference on Learning Representations (ICLR) , 2022.
  [Code]
• Offline Reinforcement Learning with Value-based Episodic Memory.
  Xiaoteng Ma* , Yiqin Yang* , Hao Hu* , Qihan Liu , Jun Yang , Chongjie Zhang, Qianchuan Zhao, Bin Liang.
  International Conference on Learning Representations (ICLR) , 2022.
  [PDF]
• Multi-Agent Incentive Communication via Decentralized Teammate Modeling.
  Lei Yuan*， Jianhao Wang*, Fuxiang Zhang, Chenghe Wang, Zongzhang Zhang, Yang Yu, Chongjie Zhang.
  AAAI Conference on Artificial Intelligence (AAAI), 2022.
  
• Towards Understanding Cooperative Multi-Agent Q-Learning with Value Factorization.
  Jianhao Wang*, Zhizhou Ren*, Beining Han, Jianing Ye, Chongjie Zhang.
  Advances on Neural Information Processing Systems (NeurIPS), 2021.
  [PDF]
• Celebrating Diversity in Shared Multi-Agent Reinforcement Learning.
  Chenghao Li, Chengjie Wu, Tonghan Wang, Jun Yang, Qianchuan Zhao, Chongjie Zhang.
  Advances on Neural Information Processing Systems (NeurIPS), 2021.
  [PDF] [Code]
• On the Estimation Bias in Double Q-Learning
  Zhizhou Ren, Guangxiang Zhu, Hao Hu, Beining Han, Jianglun Chen, Chongjie Zhang.
  Advances on Neural Information Processing Systems (NeurIPS), 2021.
  [Code] [PDF]
• Offline Reinforcement Learning with Reverse Model-based Imagination
  Jianhao Wang*, Wenzhe Li*, Haozhe Jiang, Guangxiang Zhu, Siyuan Li, Chongjie Zhang.
  Advances on Neural Information Processing Systems (NeurIPS), 2021.
  [Website] [Code] [PDF]
• Episodic Multi-agent Reinforcement Learning with Curiosity-driven Exploration
  Lulu Zheng*, Jiarui Chen*, Jianhao Wang, Jiamin He, Yujing Hu, Yingfeng Chen, Changjie Fan, Yang Gao, Chongjie Zhang.
  Advances on Neural Information Processing Systems (NeurIPS), 2021.
  [Code] [PDF]
• Model-Based Reinforcement Learning via Imagination with Derived Memory
  Yao Mu, Yuzheng Zhuang, Bin Wang, Guangxiang Zhu, Wulong Liu, Jianyu Chen, Ping Luo, Shengbo Eben Li, Chongjie Zhang, Jianye HAO.
  Advances on Neural Information Processing Systems (NeurIPS), 2021.
  [PDF]
• MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Exploration.
  Jin Zhang*, Jianhao Wang*, Hao Hu, Tong Chen, Yingfeng Chen, Changjie Fan and Chongjie Zhang.
  International Conference on Machine Learning (ICML), 2021.
  [PDF] [Code]
• Generalizable Episodic Memory for Deep Reinforcement Learning.
  Hao Hu, Jianing Ye, Zhizhou Ren, Guangxiang Zhu, and Chongjie Zhang.
  International Conference on Machine Learning (ICML), 2021.
  [PDF] [Code]
• Reward-Constrained Behavior Cloning.
  Zhaorong Wang, Meng Wang, Jingqi Zhang, Yingfeng Chen, Chongjie Zhang.
  International Joint Conference on Artificial Intelligence (IJCAI), 2021.
  
• RODE: Learning Roles to Decompose Multi-Agent Tasks.
  Tonghan Wang, Tarun Gupta, Anuj Mahajan, Bei Peng, Shimon Whiteson and Chongjie Zhang.
  International Conference on Learning Representations (ICLR), 2021.
  [Website][Code][PDF]
• QPLEX: Duplex Dueling Multi-Agent Q-Learning
  Jianhao Wang*, Zhizhou Ren*, Terry Liu, Yang Yu and Chongjie Zhang.
  International Conference on Learning Representations (ICLR), 2021.
  [Website][Code][PDF]
• DOP: Off-Policy Multi-Agent Decomposed Policy Gradients.
  Yihan Wang*, Beining Han*, Tonghan Wang*, Heng Dong and Chongjie Zhang.
  International Conference on Learning Representations (ICLR), 2021.
  [Website][Code][PDF]
• Learning Subgoal Representations with Slow Dynamics.
  Siyuan Li*, Lulu Zheng*, Jianhao Wang and Chongjie Zhang.
  International Conference on Learning Representations (ICLR), 2021.
  [Website][Code][PDF]
• Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
  Guangxiang Zhu*, Minghao Zhang*, Honglak Lee and Chongjie Zhang
  Advances on Neural Information Processing Systems (NeurIPS), 2020.
  [PDF]
• ROMA: Multi-Agent Reinforcement Learning with Emergent Roles
  Tonghan Wang, Heng Dong, Victor Lesser, and Chongjie Zhang
  International Conference on Machine Learning (ICML), 2020.
  [Website][Code][PDF] [Arxiv]
• Learning Nearly Decomposable Value Functions via Communication Minimization
  Tonghan Wang*, Jianhao Wang*, Chongyi Zheng, Chongjie Zhang
  International Conference on Learning Representations (ICLR), 2020. [Spotlight Paper]
  [Website][Code][PDF] [Arxiv]
• Influence-Based Multi-Agent Exploration
  Tonghan Wang*, Jianhao Wang*, Yi Wu, Chongjie Zhang
  International Conference on Learning Representations (ICLR), 2020.
  [Website][Code][PDF] [Arxiv]
• Episodic Reinforcement Learning with Associated Memory
  Guangxiang Zhu*, Zichuan Lin*, Guangwen Yang, Chongjie Zhang
  International Conference on Learning Representations (ICLR), 2020.
  [PDF]
• Object-Oriented Dynamics Learning through Multi-Level Abstraction
  Guangxiang Zhu*, Jianhao Wang*, Zhizhou Ren*, Chongjie Zhang
  AAAI Conference on Artificial Intelligence (AAAI), 2020.
  [PDF] [Arxiv]
• Modeling Trust Dynamics in Human-robot Teaming: A Bayesian Inference Approach.
  Yaohui Guo, Chongjie Zhang and X. Jessie Yang.
  CHI Conference on Human Factors in Computing Systems, 2020.
  [pdf] [PDF]
• Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewards
  Siyuan Li*, Rui Wang*, Minxue Tang, Chongjie Zhang
  Advances in Neural Information Processing Systems (NeurIPS), 2019.
  [pdf] [Arxiv]
• Efficiently Detecting and Optimally Responding Towards Sophisticated Opponents
  Tianpei Yang, Jianye Hao, Zhaopeng Meng, Chongjie Zhang, Yan Zheng, Ze Zheng
  Proc. of the 28th International Joint Conference on Artificial Intelligence (IJCAI) , 2019.
  [pdf]
• Convergence of Multi-Agent Learning with a Finite Step Size in General-Sum Games
  Xinliang Song, Tonghan Wang, Chongjie Zhang
  Proc. of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS) , 2019.
  [pdf]
• Context-Aware Policy Reuse
  Siyuan Li, Fangda Gu, Guangxiang Zhu, Chongjie Zhang
  Proc. of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS) , 2019.
  [pdf]
• Bayes-ToMoP: A Fast Detection and Best Response Algorithm Towards Sophisticated Opponents
  Tianpei Yang, Jianye Hao, Zhaopeng Meng, Chongjie Zhang, Yan Zheng, Ze Zheng
  Proc. of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS) , 2019.
  [pdf (Extended Abstract)] [ arXiv (Longer Version)]
• Object-Oriented Dynamics Learning through Multi-Level Abstraction
  Guangxiang Zhu*, Jianhao Wang*, Zhizhou Ren*, Chongjie Zhang
  Deep Reinforcement Learning Workshop, NeurIPS , 2018.
  [pdf]
• Object-Oriented Dynamics Predictor
  Guangxiang Zhu, Zhiao Huang, Chongjie Zhang
  Advances in Neural Information Processing Systems (NeurIPS), 2018.
  [pdf] [supplementary] [ arXiv ] [ Code ]
• Automation Reliability and Trust: A Bayesian Inference Approach
  Chenlan Wang, Chongjie Zhang, X. Jessie Yang
  Proceedings of the 62nd Human Factors and Ergonomics Society Annual Meeting (HFES) , 202-206, 2018. [Best Student Paper 3rd Prize, HFES CEDM-TG]
  [pdf]
• Trust Dynamics in Sequential Decision Making
  Changhoon Kim, Mengyuan Zhang, Chongjie Zhang, X. Jessie Yang
  Proceedings of the 62nd Human Factors and Ergonomics Society Annual Meeting (HFES), 165-166, 2018.
  [pdf]
• An Optimal Online Method of Selecting Source Policies for Reinforcement Learning
  Siyuan Li, Chongjie Zhang
  AAAI Conference on Artificial Intelligence (AAAI), 2018.
  [pdf]
• Perturbation Training for Human-Robot Teams
  Ramya Ramakrishnan, Chongjie Zhang, Julie A. Shah
  Journal of Artificial Intelligence Research (JAIR), 2017.
  [pdf]
• Context-Based Concurrent Experience Sharing in Multiagent Systems
  Dan Garant, Bruno Da Silva, Victor Lesser, Chongjie Zhang
  International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2017.
  [pdf (Extended Abstract)][arXiv (longer version)]
• Co-Optimizing Task and Motion Planning
  Chongjie Zhang, Julie A. Shah
  IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.
  [pdf]
• Co-Optimization Multi-Agent Placement with Task Assignment and Scheduling
  Chongjie Zhang, Julie A. Shah
  International Joint Conference on Artificial Intelligence (IJCAI), 2016.
  [pdf]
• On Fairness in Decision-Making under Uncertainty: Definitions, Computation, and Comparison
  Chongjie Zhang, Julie A. Shah
  AAAI Conference on Artificial Intelligence (AAAI), 2015.
  [pdf]
• Fairness in Multi-Agent Sequential Decision-Making
  Chongjie Zhang, Julie A. Shah
  Advances in Neural Information Processing Systems (NIPS), 2014.
  [pdf]
• Decentralized Multi-Agent Reinforcement Learning in Average-Reward Dynamic DCOPs
  Nguyen Duc Thien, William Yeoh, Hoong Chuin Lau, Shlomo Zilberstein, Chongjie Zhang
  AAAI Conference on Artificial Intelligence (AAAI), 2014.
  [pdf]
• Decentralized Multi-Agent Reinforcement Learning in Average-Reward Dynamic DCOPs (Extended Abstract)
  Nguyen, Duc Thien, William Yeoh, Hoong Chuin Lau, Shlomo Zilberstein, Chongjie Zhang
  International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2014.
  [pdf]
• Coordinating Multi-Agent Reinforcement Learning with Limited Communication
  Chongjie Zhang, Victor Lesser
  International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2013.
  [pdf]
• Combining Dynamic Reward Shaping and Action Shaping for Coordinating Multi-Agent Learning
  Xiangbin Zhu, Chongjie Zhang, Victor Lesser
  IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT), 2013.
  [pdf]
• Biasing the Behavior of Organizationally Adept Agents (Extended Abstract)
  Dan Corkill, Chongjie Zhang, Bruno da Silva, Yoonheui Kim, Xiaoqin (Shelley) Zhang, Victor Lesser
  International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2013.
  [pdf]
• Coordinating Multi-Agent Learning for Decentralized POMDPs
  Chongjie Zhang, Victor Lesser
  International Workshop on Multiagent Sequential Decision Making Under Uncertainty (MSDM), 2012.
  [pdf]
• Using Annotated Guidelines to Influence the Behavior of Organizationally Adept Agents
  Dan Corkill, Chongjie Zhang, Bruno da Silva, Yoonheui Kim, Xiaoqin (Shelley) Zhang, Victor Lesser
  International Workshop on Coordination, Organization, Institutions, and Norms (COIN@AAMAS), 2012.
  [pdf]
• An Ensemble Architecture for Learning Complex Problem-Solving Techniques from Demonstration
  Xiaoqin Shelley Zhang, Sungwook Yoon, Phillip DiBona, Darren Scott Appling, Li Ding, Janardhan Rao Doppa, Derek Green, Jinhong K Guo, Ugur Kuter, Geoff Levine, Reid L MacTavish, Daniel McFarlane, James R Michaelis, Hala Mostafa, Bhavesh Shrestha, Zhexuan Song, Ethan B Trewhitt, Huzaifa Zafar, Chongjie Zhang, et al.
  ACM Transactions on Intelligent Systems and Technology (TIST), 2012.
  [pdf]
• Coordinated Multi-Agent Reinforcement Learning in Networked Distributed POMDPs
  Chongjie Zhang, Victor Lesser
  AAAI Conference on Artificial Intelligence (AAAI), 2011.
  [pdf]
• Organizationally Adept Agents
  Daniel Corkill, Edmund Durfee, Victor Lesser, Huzaifa Zafar, Chongjie Zhang
  International Workshop on Coordination, Organization, Institutions and Norms in Agent Systems (COIN@AAMAS 2011), 2011.
  [pdf]
• Scaling Multi-Agent Learning in Complex Environments
  Chongjie Zhang
  Ph.D. Dissertation, 2011.
  [pdf]
• Multi-Agent Learning with Policy Prediction
  Chongjie Zhang, Victor Lesser
  AAAI Conference on Artificial Intelligence (AAAI), 2010.
  [pdf]
• Self-Organization for Coordinating Decentralized Reinforcement Learning
  Chongjie Zhang, Victor Lesser, Sherief Abdallah
  International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2010.
  [pdf]
• A Multi-Agent Learning Approach to Online Distributed Resource Allocation
  Chongjie Zhang and Victor Lesser, Prashant Shenoy
  International Joint Conference on Artificial Intelligence (IJCAI), 2009.
  [pdf]
• Integrating Organizational Control into Multi-Agent Learning
  Chongjie Zhang, Victor Lesser, Sherief Abdallah
  International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2009.
  [pdf]
• An Ensemble Learning and Problem-Solving Architecture for Airspace Management
  Xiaoqin Shelley Zhang, Sungwook Yoon, Phillip DiBona, Darren Scott Appling, Li Ding, Janardhan Rao Doppa, Derek Green, Jinhong K Guo, Ugur Kuter, Geoff Levine, Reid L MacTavish, Daniel McFarlane, James R Michaelis, Hala Mostafa, Bhavesh Shrestha, Zhexuan Song, Ethan B Trewhitt, Huzaifa Zafar, Chongjie Zhang, et al.
  Annual Conference on Innovative Applications of Artificial Intelligence (IAAI), 2009.
  [pdf]
• Efficient Multi-Agent Reinforcement Learning through Automated Supervision (Short Paper)
  Chongjie Zhang, Victor Lesser, Sherief Abdallah
  International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2008.
  [pdf]
• An Application Portal for Collaborative Coastal Modeling
  Chongjie Zhang, Chirag Dekate, Gabrielle Allen, Ian Kelley, Jon MacLaren
  Concurrency and Computation: Practice and Experience, 2007.
  [pdf]
• Grid Portal Solutions: A Comparison of GridPortlets and OGCE
  Chongjie Zhang, Ian Kelley, Gabrielle Allen
  Concurrency and Computation: Practice and Experience, 2007.
  [pdf]
• A Workflow Approach to Designed Reservoir Study
  Gabrielle Allen, Promita Chakraborty, Dayong Huang, Zhou Lei, John Lewis, Christopher White, Xiaoxi Xu, Chongjie Zhang
  Workshop on Workflows in Support of Large-Scale Science (WORKS), 2007.
  [pdf]
• Shelter from the Storm: Building a Safe Archive in a Hostile World
  Jon MacLaren, Gabrielle Allen, Chirag Dekate, Dayong Huang, Andrei Hutanu, Chongjie Zhang
  Lecture Notes in Computer Science, 2005.
  [pdf]
• An Application Portal for Collaborative Coastal Modeling
  Chongjie Zhang, Chirag Dekate, Gabrielle Allen, Ian Kelley, Jon MacLaren
  International Workshop on Grid Computing Environments (GCE), 2005. (Best Paper Award)
  
• Grid Portal Solutions: A Comparison of GridPortlets and OGCE
  Chongjie Zhang, Ian Kelley, Gabrielle Allen
  International Workshop on Grid Computing Environments (GCE), 2005.