Hi! I am Zichuan Lin. I am a researcher working on game AI at Tencent AI Lab. I got my Ph.D. degree in June, 2021 in the Department of Computer Science and Technology, Tsinghua University, advised by Prof. Guangwen Yang. Previously, I completed my B.S. in Computer Science at Tsinghua University.

I was a research intern at Microsoft Research Asia (MSRA) from Sep, 2016 to Mar, 2018 (advised by Lintao Zhang) and from May, 2018 to Jun, 2019 (advised by Tao Qin and Li Zhao). I was a visiting student researcher at Stanford University from Feb, 2020, advised by Prof. Tengyu Ma.

My research interests include reinforcement learning and deep learning. My goal is to develop sample-efficient reinforcement learning algorithms with strong robustness and interpretability. My research projects mainly involve episodic control, reward decomposition, disentangled representation learning, model-based RL and meta-RL. I am also interested in RL applications such as task-oriented dialogue systems.

Manuscripts

• Pretraining in Deep Reinforcement Learning: A Survey
  Zhihui Xie, Zichuan Lin, Junyou Li, Shuai Li, Deheng Ye
  arxiv 2022

• Revisiting Discrete Soft Actor-Critic
  Haibin Zhou, Zichuan Lin, Junyou Li, Deheng Ye, Qiang Fu, Wei Yang
  arxiv 2022

• Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System
  Zichuan Lin, Jing Huang, Bowen Zhou, Xiaodong He, Tengyu Ma
  arxiv 2021

Publications

(* represents equal contribution)

• A Survey on Transformers in Reinforcement Learning
  Wenzhe Li*, Hao Luo*, Zichuan Lin*, Chongjie Zhang, Zongqing Lu, Deheng Ye
  TMLR 2023

• Future-conditioned Unsupervised Pretraining for Decision Transformer
  Zhihui Xie, Zichuan Lin, Deheng Ye, Qiang Fu, Wei Yang, Shuai Li
  ICML 2023

• Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization
  Tiantian Zhang, Zichuan Lin, Yuxing Wang, Deheng Ye, Qiang Fu, Wei Yang, Xueqian Wang, Bin Liang, Bo Yuan, Xiu Li
  TNNLS 2023

• JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning
  Zichuan Lin*, Junyou Li*, Jianing Shi*, Deheng Ye, Qiang Fu, Wei Yang
  IJCAI 2022 (Long Oral top3%) (The champion solution of NeurIPS 2021 MineRL research competition)

• Model-based Adversarial Meta-Reinforcement Learning [code]
  Zichuan Lin, Garrett Thomas, Guangwen Yang, Tengyu Ma
  NeurIPS 2020

• RD$^2$: Reward Decomposition with Representation Decomposition
  Zichuan Lin*, Derek Yang*, Li Zhao, Tao Qin, Guangwen Yang, Tie-yan Liu
  NeurIPS 2020

• Episodic Reinforcement Learning with Associative Memory
  Guangxiang Zhu*, Zichuan Lin*, Guangwen Yang, and Chongjie Zhang
  ICLR 2020

• Object-Oriented Dynamics Learning through Multi-Level Abstraction
  Guangxiang Zhu*, Jianhao Wang*, Zhizhou Ren*, Zichuan Lin and Chongjie Zhang
  AAAI 2020

• Distributional Reward Decomposition for Reinforcement Learning
  Zichuan Lin, Li Zhao, Derek Yang, Tao Qin, Guangwen Yang, and Tie-yan Liu
  NeurIPS 2019

• Fully Parameterized Quantile Function for Distributional Reinforcement Learning [code]
  Derek Yang, Li Zhao, Zichuan Lin, Jiang Bian, Tao Qin, and Tie-yan Liu
  NeurIPS 2019

• Unified Policy Optimization for Robust Reinforcement Learning
  Zichuan Lin, Li Zhao, Jiang Bian, Tao Qin, and Guangwen Yang
  ACML 2019 (Oral)

• Episodic Memory Deep Q-Networks [code]
  Zichuan Lin, Tianqi Zhao, Guangwen Yang, and Lintao Zhang
  IJCAI 2018

Talks

• JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning
  IJCAI, Long Oral, 2022

• Model-based Adversarial Meta-Reinforcement Learning
  ICML, BIG Workshop, 2020

• Unified Policy Optimization for Reinforcement Learning [slides]
  Nagoya, Japan, 2019

• Towards Sample-efficient, Interpretable and Robust Reinforcement Learning [slides]
  Wuxi, China, 2019

• Episodic Memory Deep Q-Networks [slides]
  Stockholm, Sweden, 2018

• Reinforcement Learning: Episodic Memory and Learning to Run [slides]
  MSRA, Beijing, 2017

Awards & Honors

• Beijing Outstanding PhD Graduate, 2021.
• 84 Innovative Future Scholarship (Top 1%) at department of Computer Science at Tsinghua University, 2020.
• National Scholarship (Top 1%) at Tsinghua University, 2020.
• Person of the Year nominee, Tsinghua University, 2019. [Media Coverage]
• Tsinghua Scholarship for Overseas Graduate Study, 2019.
• Excellence in the Microsoft Research Asia Internship Program, 2018. [Media Coverage]
• WeChat Public Account Article, Tsinghua University, 2018. [Media Coverage]
• National Scholarship (Top 1%) at Tsinghua University, 2018.
• Top 10 Outstanding Athletes at Tsinghua University, 2017. [Media Coverage]
• Excellent Undergraduate Thesis Award (Top 10%) at Tsinghua University, 2016.
• Top 2 in Tsinghua Mathematical Contest in Modeling, 2016.
• 1st Place in College Table Tennis Student Championships in Beijing, 2013-2016.
• Table Tennis National First-Level Athlete, 2012.
• 1st Place in World Table Tennis Student Championships, Liaoning, China, 2011.

Experience and Services

• Reviewer: NeurlPS, ICML, ICLR, AAAI
• Teaching Assistant: Software Engineering (Undergraduate Course, Spring 2016)

Contact

• Email: lastname + zc16 at mails dot tsinghua dot edu dot cn