Ideas

Research directions, hypotheses, experiment proposals, and half-baked thoughts.

Use _template.qmd as a starting point for new entries.

Date Idea Tags Summary
Mar 7, 2026 Population-Based GRPO / Reward Models rl helpful chat: https://claude.ai/chat/a844d6c1-7048-4340-a1dd-9a43892d2413
Mar 6, 2026 Contrastive Learning on GRPO Rollouts rl CLIP-style contrastive objective over the G rollouts per prompt — positive/negative pairs from reward signal.
Mar 6, 2026 Meta Reward Models rl Learn to autonomously construct reward functions for arbitrary tasks, then instantiate RLVR loops — a meta-level system that figures out what to reward, not just how to optimize.
Mar 5, 2026 Continual Learning for RLVR rl See slide decks of some new ideas in this space:
Mar 5, 2026 Mixture of Distillations rl See slide deck here.
Mar 5, 2026 Beyond Reweighting: Can RLVR Discover? rl Does RLVR only reweight and sharpen pre-existing solutios, or can it discover new solutions?
Mar 5, 2026 Self-Distilling Reward Models rl, favorite Policy as its own judge. Self-distill teacher as a reward model into the student
Mar 4, 2026 On-Policy Self-Distillation for Cross-Embodiment Transfer in World Action Models robot-learning Self-distill different embodiments through demo in a world model
Mar 2, 2026 Search Self-Distillation rl, llm Use MCTS on-policy
Mar 2, 2026 The Policy as Its Own Judge: Self-Rewarding Language Models through Self-Distillation Grounded by Asymmetric Environment Feedback rl Use self-distillation to make self-teacher own LLM-as-a-Judge. Conditioned on environment feedback.
Mar 1, 2026 Meta-Reasoning via Search: Integrating Monte Carlo Tree Search with Group Relative Policy Optimization for Efficient LLM Post-Training rl Adding in MCTS into GRPO and then distilling this into the base model
Mar 1, 2026 VLAF: Video-Language-Action Fusion for Robot Learning Backbone vla, robot learning Condition policy on additional context, use to detect reward hacking, self-distill into student policy, all online, preventing distribution mismatch.
Feb 26, 2026 Adversarial Co-Training for Online Reward Hacking Mitigation (ARHD) rl Co-evolve a critic alongside the policy to detect and mitigate reward hacking online
Feb 26, 2026 Self-Distillation for Reward Hacking Mitigation via Contextualized Self-Judgment rl Condition policy on additional context, use to detect reward hacking, self-distill into student policy, all online, preventing distribution mismatch.
No matching items