Paper Notes
Summaries, key takeaways, and critiques of papers Iām reading.
Use _template.qmd as a starting point for new entries.
| Date | Title | Categories | Description |
|---|
No matching items
| Date | Title | status | Categories | Description |
|---|---|---|---|---|
| Mar 6, 2026 | RLAC: RL with Adversarial Critic for Free-Form Generation | rl, llm, post-training, reading | Adversarial game between a generator and learned critic that dynamically proposes failure modes verified by external validators, replacing exhaustive rubric checking. | |
| Mar 5, 2026 | POPE: Privileged On-Policy Exploration for Hard Reasoning | rl, llm, post-training, reading | Uses oracle solution prefixes as privileged guidance during RL to enable exploration on hard problems where standard on-policy RL yields zero reward. | |
| Mar 5, 2026 | RARO: Reasoning via Adversarial Inverse RL from Demonstrations | rl, llm, post-training, reading | Adversarial IRL framework that trains LLM reasoning from expert demonstrations without task-specific verifiers, using a relativistic critic to distinguish expert from policy outputs. | |
| Mar 3, 2026 | Midtraining Bridges Pretraining and Posttraining Distributions | rl, post-training | Systematic study of midtraining as distributional bridging between pretraining and posttraining; benefits code/math domains, mitigates forgetting, and timing-weight interaction matters. | |
| Mar 3, 2026 | PipelineRL: Faster On-policy RL for Long Sequence Generation | rl, llm, reading | Asynchronous RL training with in-flight weight updates achieves ~2x faster learning while maintaining on-policy data quality for LLM reasoning. | |
| Mar 1, 2026 | DreamZero: World Action Models are Zero-shot Policies | robotics, foundation-models, world-models, reading | 14B WAM built on pretrained video diffusion that jointly predicts video and actions, enabling zero-shot generalization, heterogeneous data learning, and cross-embodiment transfer. | |
| Mar 1, 2026 | LLMs Can Learn to Reason Via Off-Policy RL (OAPL) | rl, llm, reading | Optimal Advantage-based Policy Optimization with Lagged Inference ā off-policy RL for LLM post-training without importance sampling. | |
| Feb 26, 2026 | DeepSeekMath: Pushing the Limits of Mathematical Reasoning (GRPO) | rl, llm, reading | Notes on Group Relative Policy Optimization and its applicability to clinical agent training. |
No matching items