CDS Researchers Make Strong Showing at ICLR 2025

The International Conference on Learning Representations (ICLR) 2025 has accepted numerous papers from our researchers, showcasing the Center’s continued impact on machine learning research. Thirty-seven researchers from across faculty, postdoctoral researchers, faculty fellows, and PhD students have had their work accepted to this prestigious conference, with contributions spanning diverse areas including language models, multimodal learning, representation learning, and causality.

Several papers received special recognition, with CDS Silver Professor Julia Kempe having two Spotlighted works, on distributionally robust pruning and — with CDS PhD Student Yunzhen Feng — how synthetic data degrades performance and interacts with model scaling. CDS Professor Andrew Gordon Wilson and CDS PhD student Yilun Kuang have a Spotlighted paper on using Bayesian optimization informed by generative models of evolving antibody sequences to more efficiently design effective therapeutic antibodies. Finally, CDS PhD student Jianyu Zhang received Spotlight recognition for his innovative hashtable technique that leverages both CPU and GPU resources to optimize transformer inference.

Congratulations to all CDS researchers whose work was accepted to ICLR 2025. Their contributions continue to advance the field of machine learning and represent the collaborative and innovative research environment at the Center.

• Rico Angell (CDS Postdoctoral Researcher)
  — “Monitoring LLM Agents for Sequentially Contextual Harm” (Building Trust Workshop Paper)
• Sam Bowman (CDS Associate Professor of Linguistics and Data Science)
  — “Language Models Learn to Mislead Humans via RLHF” (Poster)
  — “Inverse Scaling: When Bigger Isn’t Better” (Poster)
  — “Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models” (Poster)
• Joan Bruna (CDS Professor of Computer Science and Data Science)
  —“Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers” (Poster)
  — “Quality over Quantity in Attention Layers: When Adding More Heads Hurts” (Poster)
• Angelica Chen (CDS PhD Student)
  — “Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models” (Poster)
• Yulin Chen (CDS PhD Student)
  — “Monitoring LLM Agents for Sequentially Contextual Harm” (Building Trust Workshop Paper)
• Kyunghyun Cho (CDS Professor of Computer Science and Data Science)
  — “Aioli: A Unified Optimization Framework for Language Model Data Mixing” (Poster)
  — “Multi-modal Learning: A Look Back and the Road Ahead” (Blog Post)
  — “Concept Bottleneck Language Models For Protein Design” (Poster)
  — “X-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs” (Poster)
• Eunsol Choi (CDS Assistant Professor of Computer Science and Data Science)
  — “Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions” (Poster)
• Yunzhen Feng (CDS PhD Student)
  — “Strong Model Collapse” (Paper — Spotlight)
  — “Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification” (Poster)
• Juliana Freire (CDS Professor of Computer Science, Engineering, and Data Science)
  — “Matrix Product Sketching via Coordinated Sampling” (Poster)
• Yuzhou Gu (CDS Faculty Fellow)
  — “Faster Algorithms for Structured Linear and Kernel Support Vector Machines” (Poster)
• Florentin Guth (CDS Faculty Fellow)
  — “A Rainbow in Deep Network Black Boxes” (Poster)
• He He (CDS Assistant Professor of Computer Science and Data Science)
  — “Monitoring LLM Agents for Sequentially Contextual Harm” (Building Trust Workshop Paper)
  — “Language Models Learn to Mislead Humans via RLHF” (Poster)
  — “Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats” (Poster)
  — “Transformers Struggle to Learn to Search” (Poster)
• Sophie Hao (CDS Faculty Fellow)
  — “Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models” (Poster)
• Michael Hu (CDS PhD Student)
  — “Aioli: A Unified Optimization Framework for Language Model Data Mixing” (Poster)
  — “How to visualize training dynamics in neural networks” (Blog Post)
• Sanyam Kapoor (CDS PhD Student)
  — “Compute-Optimal LLMs Provably Generalize Better with Scale” (Poster)
• Julia Kempe (CDS Silver Professor of Computer Science, Mathematics, and Data Science)
  — “DRoP: Distributionally Robust Pruning” (Poster — Spotlight)
  — “Strong Model Collapse” (Paper — Spotlight)
  — “Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification” (Poster)
  — “Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networks” (Poster)
• Yilun Kuang (CDS PhD Student)
  — “Bayesian Optimization of Antibodies Informed by a Generative Model of Evolving Sequences” (Poster — Spotlight)
• Yann LeCun (CDS Founding Director and Professor)
  — “Hierarchical World Models as Visual Whole-Body Humanoid Controllers” (Poster)
  — “LiveBench: A Challenging, Contamination-Limited LLM Benchmark” (Poster)
  — “PooDLe: Pooled and dense self-supervised learning from naturalistic videos” (Poster)
  — “Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning” (Poster)
  — “URLOST: Unsupervised Representation Learning without Stationarity or Topology” (Poster)
  — “X-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs” (Poster)
• Qi Lei (CDS Assistant Professor of Mathematics and Data Science)
  — “Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness” (Poster)
• Tal Linzen (CDS Associate Professor of Linguistics and Data Science)
  — “Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models” (Poster)
• Vishakh Padmakumar (CDS PhD Student)
  — “Transformers Struggle to Learn to Search” (Poster)
  — “Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models” (Poster)
• Xiang Pan (CDS PhD Student)
  — “Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness” (Poster)
• Jacob Pfau (CDS PhD Student)
  — “Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback” (Poster)
• Andres Potapczynski (CDS PhD Student)
  — “Flexible Models of Functional Annotations to Variant Effects using Accelerated Linear Algebra” (MLGenX 2025)
• Rajesh Ranganath (CDS Associate Professor of Computer Science and Data Science)
  — “Time After Time: Deep-Q Effect Estimation for Interventions on When and What to Do” (Poster)
• Shauli Ravfogel (CDS Faculty Fellow)
  — “Gumbel Counterfactual Generation From Language Models” (Poster)
• Mengye Ren (CDS Assistant Professor of Computer Science and Data Science)
  — “PooDLe: Pooled and dense self-supervised learning from naturalistic videos” (Poster)
• Cristina Savin (CDS Associate Professor of Neural Science and Data Science)
  — “Nonlinear multiregion neural dynamics with parametric impulse response communication channels” (Poster)
• Ravid Shwartz-Ziv (CDS Research Scientist)
  — “Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning” (Poster)
  — “Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs” (Poster)
  — “LiveBench: A Challenging, Contamination-Limited LLM Benchmark” (Poster)
• Vlad Sobal (CDS PhD Student)
  — “X-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs” (Poster)
  — “Hierarchical World Models as Visual Whole-Body Humanoid Controllers” (Poster)
• Ilia Sucholutsky (CDS Faculty Fellow)
  — “Large Language Models Assume People are More Rational than We Really are” (Poster)
• Ryan Teehan (CDS PhD Student)
  — “Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models” (Poster)
• Nikolaos Tsilivis (CDS PhD Student)
  — “Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networks” (Poster)
• Yoav Wald (CDS Faculty Fellow)
  — “Time After Time: Deep-Q Effect Estimation for Interventions on When and What to Do” (Poster)
• Andrew Gordon Wilson (CDS Professor of Computer Science and Data Science)
  — “Bayesian Optimization of Antibodies Informed by a Generative Model of Evolving Sequences” (Poster — Spotlight)
  — “Compute-Optimal LLMs Provably Generalize Better with Scale” (Poster)
  — “Flexible Models of Functional Annotations to Variant Effects using Accelerated Linear Algebra” (MLGenX 2025)
• Denny Wu (CDS Faculty Fellow)
  — “Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics” (Poster)
• Jianyu Zhang (CDS PhD Student)
  — “MagicPIG: LSH Sampling for Efficient LLM Generation” (Paper — Spotlight)
  — “Memory Mosaics” (Poster)