I am a PhD student in Computer Science at Boston University, where I am advised by Venkatesh Saligrama. My research lies at the intersection of theory and empirics in deep learning.

My current work studies in-context learning in transformers: when and how end-to-end trained transformers acquire algorithmic behavior, and how these behaviors can be interpreted. I use controlled tasks, such as classification and regression, together with structured transformer architectures to obtain explicit evidence on the algorithmic role of attention. More broadly, I aim to uncover algorithmic motifs implemented by attention and use them to study in-context learning in large-scale LLMs.

Before joining Boston University, I obtained a MS in Mathematics at Sorbonne University in Paris, France. I grew up in Weiden, Germany.

Publications

Layerwise Dynamics for In-Context Classification in Transformers
P. Lutz, T. Haris, A. Chandra, A. Gangrade, V. Saligrama
Preprint, 2026
[pdf]

Linear Transformers Implicitly Discover Unified Numerical Algorithms
P. Lutz, A. Gangrade, H. Daneshmand, V. Saligrama
NeurIPS, 2025
[pdf]

Sparse Tree-based Initialization for Neural Networks
P. Lutz, L. Arnauld, C. Boyer, E. Scornet
ICLR, 2023
[pdf] [code]