Preetum Nakkiran

Research

I take a scientific approach to machine learning— trying to advance understanding through basic experiments and foundational theory.

See [publications] for full list of papers.

Machine Learning Theory

• The Deep Bootstrap: Good Online Learners are Good Offline Generalizers
  Preetum Nakkiran, Behnam Neyshabur, Hanie Sedghi
  ICLR 2021.
• Distributional Generalization: A New Kind of Generalization
  Preetum Nakkiran*, Yamini Bansal*
  In submission. 2020. [talk]
• Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems
  Preetum Nakkiran
  OPT2020 Workshop (Best Student Paper)
• Optimal Regularization Can Mitigate Double Descent
  Preetum Nakkiran, Prayaag Venkat, Sham Kakade, Tengyu Ma
  ICLR 2021.
• Deep Double Descent: Where Bigger Models and More Data Hurt
  Preetum Nakkiran, Gal Kaplun*, Yamini Bansal*, Tristan Yang, Boaz Barak, Ilya Sutskever
  ICLR 2020.
• More Data Can Hurt for Linear Regression: Sample-wise Double Descent
  Preetum Nakkiran
  Manuscript. 2019.
• SGD on Neural Networks Learns Functions of Increasing Complexity
  Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L. Edelman, Fred Zhang, Boaz Barak
  NeurIPS 2019 (Spotlight).
• Adversarial Examples are Just Bugs, Too
  Preetum Nakkiran
  Distill 2019.
• Adversarial Robustness May Be at Odds With Simplicity
  Preetum Nakkiran
  (Merged appears in COLT 2019).
• The Generic Holdout: Preventing False-Discoveries in Adaptive Data Science
  Preetum Nakkiran, Jarosław Błasiok
  Manuscript. 2018.

Theory

• Algorithmic Polarization for Hidden Markov Models
  Venkatesan Guruswami, Preetum Nakkiran, Madhu Sudan
  ITCS 2019.
• General Strong Polarization
  Jarosław Błasiok, Venkatesan Guruswami, Preetum Nakkiran, Atri Rudra, Madhu Sudan
  STOC 2018.
• Tracking the L2 Norm with Constant Update Time
  Chi-Ning Chou, Zhixian Lei, Preetum Nakkiran
  APPROX-RANDOM 2018.
• Near-Optimal UGC-hardness of Approximating Max k-CSP_R
  Pasin Manurangsi, Preetum Nakkiran, Luca Trevisan
  APPROX-RANDOM 2016.

Machine Learning

• Compressing Deep Neural Networks Using a Rank-Constrained Topology
  Preetum Nakkiran, Raziel Alvarez, Rohit Prabhavalkar, Carolina Parada
  INTERSPEECH 2015.
• Automatic Gain Control and Multi-style Training for Robust Small-Footprint Keyword Spotting with Deep Neural Networks
  Rohit Prabhavalkar, Raziel Alvarez, Carolina Parada, Preetum Nakkiran, and Tara Sainath
  ICASSP 2015.