awesome-rl

A curated list of awesome reinforcement learning resources in the context of LLMs and multimodal models. Doesn't cover robotics or other domains that are equally cool.

📕 Papers and technical writeups

• DeepSeek-R1 by The WHALE!

• DeepScaleR

• SFT Memorizes, RL Generalizes

• An Unexpected RL Renaissance

• simplerl-reason

• Process Reinforcement through Implicit Rewards

• Search-o1 paper

• R-star Math

• LIMR

• SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution

• Process Reinforcement through Implicit Rewards - Notion

• Search-o1 paper

• Maybe https://arxiv.org/pdf/2412.09413 (or is this distill)??

• Teaching Large Language Models to Reason with Reinforcement Learning

• Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs

• RAGEN: Training Agents by Reinforcing Reasoning

• SimpleRL-reason

  • GitHub
  • Notion

💼 Projects (blogs, repos, Twitter threads etc.)

• Train your own R1 reasoning model with Unsloth (GRPO)
• Open-R1

📚 Libraries

• verl
• GRPO from Unsloth
• EasyR1 from hiyouga@ of LLaMA factory
  • GitHub repo
  • Twitter thread
• HuggingFace TRL
  • https://huggingface.co/docs/trl/en/index
• Verifiers (github repo)
• Search-r1
  • Twitter thread
  • GitHub repo

📊 Datasets

• open-thoughts/OpenThoughts-114k
• open-r1/OpenR1-Math-220k
• GeneralReasoning/GeneralThought-195K
• PrimeIntellect/SYNTHETIC-1
• facebook/Natural-reasoning
• SynthLabsAI/Big-Math-RL-Verified
• Congliu/Chinese-DeepSeek-R1-Distill-data-110k
• FreedomIntelligence/medical-o1-reasoning-SFT
• FreedomIntelligence/Medical-R1-Distill-Data-Chinese

📚 Other resources and classic RL

• reinforcement-learning resources (GitHub repo)
• The classic book on RL: Reinforcement Learning: An Introduction
• Playing Atari with Deep Reinforcement Learning
• RL Course by David Silver