Preetum Nakkiran

Machine Learning Theory

• What Algorithms can Transformers Learn? A Study in Length Generalization [tweet]
  Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Josh Susskind, Samy Bengio, Preetum Nakkiran
  ICLR 2024.
• When Does Optimizing a Proper Loss Yield Calibration? [tweet]
  Jarosław Błasiok, Parikshit Gopalan, Lunjia Hu, Preetum Nakkiran
  NeurIPS 2023.
• Loss Minimization Yields Multicalibration for Large Neural Networks
  Jarosław Błasiok, Parikshit Gopalan, Lunjia Hu, Adam Tauman Kalai, Preetum Nakkiran
  ITCS 2024.
• A Unifying Theory of Distance from Calibration [tweet]
  Jarosław Błasiok, Parikshit Gopalan, Lunjia Hu, Preetum Nakkiran
  STOC 2023.
• The Calibration Generalization Gap
  A. Michael Carrell, Neil Mallinar, James Lucas, Preetum Nakkiran
  ICML 2022 DFUQ Workshop.
• Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting
  Neil Mallinar*, Jamie Simon*, Amirhesam Abedsoltan, Parthe Pandit, Mikhail Belkin, Preetum Nakkiran
  NeurIPS 2022.
• Deconstructing Distributions: A Pointwise Framework of Learning [demo]
  Gal Kaplun*, Nikhil Ghosh*, Saurabh Garg, Boaz Barak, Preetum Nakkiran
  ICLR 2023.
• What You See is What You Get: Distributional Generalization for Algorithm Design in Deep Learning
  Bogdan Kulynych*, Yao-Yuan Yang*, Yaodong Yu, Jarosław Błasiok, Preetum Nakkiran
  NeurIPS 2022.
• Knowledge Distillation: Bad Models Can Be Good Role Models
  Gal Kaplun, Eran Malach, Preetum Nakkiran, Shai Shalev-Shwartz
  NeurIPS 2022.
• Limitations of the NTK for Understanding Generalization in Deep Learning
  Nikhil Vyas, Yamini Bansal, Preetum Nakkiran
  Preprint. 2022.
• Limitations of Neural Collapse for Understanding Generalization in Deep Learning [tweet]
  Like Hui, Mikhail Belkin, Preetum Nakkiran
  Preprint. 2022.
• Turing-Universal Learners with Optimal Scaling Laws
  Preetum Nakkiran
  Manuscript. 2021.
• Revisiting Model Stitching to Compare Neural Representations [tweet]
  Yamini Bansal, Preetum Nakkiran, Boaz Barak
  NeurIPS 2021.
• The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers [slides] [tweet]
  Preetum Nakkiran, Behnam Neyshabur, Hanie Sedghi
  ICLR 2021.
• Distributional Generalization: A New Kind of Generalization [10m talk] [slides]
  Preetum Nakkiran*, Yamini Bansal*
  - Desk-Rejected from NeurIPS 2020.
  - Rejected from ICLR 2021. [reviews]
  - Rejected from ICML 2021. [reviews] [rebuttal] [meta]
  - Rejected from NeurIPS 2021. [reviews]
  - Rejected from ICLR 2022. [reviews]
• Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems [tweet]
  Preetum Nakkiran
  OPT2020 Workshop (Best Student Paper)
• Optimal Regularization Can Mitigate Double Descent
  Preetum Nakkiran, Prayaag Venkat, Sham Kakade, Tengyu Ma
  ICLR 2021.
• Deep Double Descent: Where Bigger Models and More Data Hurt
  Preetum Nakkiran, Gal Kaplun*, Yamini Bansal*, Tristan Yang, Boaz Barak, Ilya Sutskever
  ICLR 2020.
• More Data Can Hurt for Linear Regression: Sample-wise Double Descent
  Preetum Nakkiran
  Manuscript. 2019.
• SGD on Neural Networks Learns Functions of Increasing Complexity
  Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L. Edelman, Fred Zhang, Boaz Barak
  NeurIPS 2019 (Spotlight).
• Adversarial Examples are Just Bugs, Too
  Preetum Nakkiran
  Distill 2019.
• Adversarial Robustness May Be at Odds With Simplicity
  Preetum Nakkiran
  (Merged appears in COLT 2019).
• The Generic Holdout: Preventing False-Discoveries in Adaptive Data Science
  Preetum Nakkiran, Jarosław Błasiok
  Manuscript. 2018.

Theory

• Algorithmic Polarization for Hidden Markov Models
  Venkatesan Guruswami, Preetum Nakkiran, Madhu Sudan
  ITCS 2019.
• General Strong Polarization
  Jarosław Błasiok, Venkatesan Guruswami, Preetum Nakkiran, Atri Rudra, Madhu Sudan
  STOC 2018.
• Tracking the L2 Norm with Constant Update Time
  Chi-Ning Chou, Zhixian Lei, Preetum Nakkiran
  APPROX-RANDOM 2018.
• Near-Optimal UGC-hardness of Approximating Max k-CSP_R
  Pasin Manurangsi, Preetum Nakkiran, Luca Trevisan
  APPROX-RANDOM 2016.

Machine Learning

• Compressing Deep Neural Networks Using a Rank-Constrained Topology
  Preetum Nakkiran, Raziel Alvarez, Rohit Prabhavalkar, Carolina Parada
  INTERSPEECH 2015.
• Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks
  Rohit Prabhavalkar, Raziel Alvarez, Carolina Parada, Preetum Nakkiran, and Tara Sainath
  ICASSP 2015.