All talks listed in Pacific Time. Schedule subject to change.

–
Coffee and Check-In
–
Opening Remarks
–
Transformers, parallel computation, and logarithmic depth

Daniel Hsu (Columbia University)
Video
–
Break
–
Capabilities and limitations of Transformer in sequential reasoning

Bingbin Liu (Carnegie Mellon University)
Video
–
Break
–
The Parallelism Tradeoff: Understanding Transformer Expressivity Through Circuit Complexity

Will Merrill (New York University)
Video
–
Lunch (on your own)
–
Emergence and grokking in "simple" architectures

Misha Belkin (UCSD)
Video
–
Break
–
Iterated Models: Expressive Power, Learning, and Chain of Thought

Nati Srerbo (Toyota Technological Institute at Chicago)
Video
–
Reception
–
Coffee and Check-In
–
The emergence of clusters in self-attention dynamics

Philippe Rigollet (MIT)
Video
–
Break
–
Computational Benefits and Limitations of Transformers and State-Space Models

Eran Malach (Kempner Institute, Harvard University)
Video
–
Break
–
Transformer Expressivity and Formal Logic

David Chiang (University of Notre Dame)
Video
–
Lunch (on your own)
–
Interpretability Agents

Sarah Schwettmann (MIT)
Video
–
Break
–
Recent Efficiency Improvements to Transformers

David Woodruff (Carnegie Mellon University)
–
Break
–
Panel Discussion: What can theory offer to the design and use of LLMs?

Jon Kleinberg, Philippe Rigollet, Nati Srebro, and Naomi Saphra
–
Coffee and Check-In
–
Language Generation in the Limit

Jon Kleinberg (Cornell University)
Video
–
Break
–
Do Large Language Models Perform Latent Reasoning? (Remote Talk)

Mor Geva (Tel Aviv University)
Video
–
Break
–
Language Acquisition in Language Models

Naomi Saphra (Kempner Institute at Harvard University)
Video
–
Lunch (on your own)
–
What was Revolutionized by the "Transformer Revolution?

Stella Biderman (ElutherAI)
Video
–
Break
–
Poster Session (2nd Floor Interactive Area)
–
Coffee and Check-In
–
Learning to Reason with LLMs

Noam Brown (OpenAI)
Video
–
Break
–
Understanding and Improving Efficient Language Models

Simran Arora (Stanford University)
Video
–
Break
–
Using recurrence to achieve weak to strong generalization

Tom Goldstein (University of Maryland)
Video
–
Lunch (on your own)
–
Towards Understanding Modern Alchemy

Ekin Akyurek (MIT)
Video
–
Break
–
A Retrieval-based Language Model at Scale (Remote Talk)

Sewon Min (UC Berkeley & AI2)
Video
–
Break
–
Panel Discussion: Are Transformers the end game?

Simran Arora, Stella Biderman, Jitendra Malik, Andrew Wilson
–
Coffee and Check-In
–
On the Tradeoffs of State Space Models

Albert Gu (Carnegie Mellon University)
Video
–
Break
–
Re-thinking Transformers: Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices

Andrew Gordon Wilson (New York University)
Video
–
Break
–
Exact solutions to the geometric dynamics of signal propagation through transformers predict their trainability

Surya Ganguli (Stanford University)
Video
–
Lunch (on your own)
–
Using Algorithms to Understand Transformers (and Using Transformers to Understand Algorithms)

Vatsal Sharan (University of Southern California)
Video
–
Break
–
Associative memories as a building block in Transformers

Alberto Bietti (Flatiron Institute)
Video
–
Hike