About me

I am a final-year PhD student from the Department of Computer Science and Technology, Tsinghua University. I am fortunately supervised by Professor Maosong Sun and Zhiyuan Liu. In the summer of 2019, I visited MILA and conducted research under Professor Jian Tang.

My current passion revolves around building SCALABLE solutions to AGI, which means the solutions will bring improvement simply with more resources on computation and data. This include:

  1. Scaling Pretrain. Ensuring the growth of language models’ ability is measureable and predictable.

  2. Scaling RL. This includes developing scaling principles for scalable oversight, PPO, MCTS, etc.

  3. Scaling World Model Unifing modality and training objectives, then scaling with pretrain and RL.

In fact, I believe that we will not have achieved AGI until the model is capable of conducting scientific research independently. All the aforementioned points contribute to this objective.

News!

🔥 2024.4 We release LEGENT. An Open Platform for Embodied Agents, The main contributor Zhili Cheng is awesome at unity programming!

🔥 2024.4 We release the paper of MiniCPM in Arxiv. A small LLM with 2.4B non-embedding parameters that rivals Llama-13B or Mistral-7B.

Selected Publications

For other papers, please refer to my google scholar

Selected Projects

Readme Card

Readme Card

Readme Card

Readme Card

Readme Card

Readme Card

Awards

Master&PhD

  • National Natural Science Foundation of China (NSFC) Doctoral Project Leader (~1/100 among all subjects and all institutes in China)

  • Siebel Scholar of Class 2023

    1/83 across the world, Price $30,000

  • National Scholarship 2021.

    One of the highest award in Tsinghua University.

Bachelor

  • Academic Excellence Award in 2016-2017 , 2017-2018, 2018-2019.
  • Good reading scholarship in 2018-2019 year.
  • Zhuzhou Scholarship in 2017-2018 year

Earlier

  • Silver Medal in the 32nd National Middle School Physics Competition