Preetum Nakkiran
I'm a 5thyear PhD student at Harvard, in the Theory of Computation and ML Theory Groups, advised by Madhu Sudan and Boaz Barak.I'm currently trying to understand why deep learning works. I do theory by doing experiments.
[publications]
[CV]
[twitter]
preetum@cs.harvard.edu
News:
 Sept 2021: I will be starting a postdoc with Misha Belkin at UCSD, as part of the NSF/Simons Collaboration on the Theoretical Foundations of Deep Learning.
 Summer 2020: Interned with Hanie Sedghi and Behnam Neyshabur at Google Brain.
 Spring 2020: Visited Jacob Steinhardt at UC Berkeley.
 Summer 2019: Interned with Ilya Sutskever at OpenAI.
Recent Invited Talks
I'm happy to speak about my works and interests. Currently, I will likely speak about The Deep Bootstrap, Distributional Generalization, or musings about scaling. Apr 2021: Guest Lecture in ML Theory Course (Boaz Barak), speaking on scaling laws. [slides]
 Apr 2021: UPenn Seminar (Weijie Su group), speaking on Distributional Generalization. [slides]
 Aug 2020: UCLA Big Data and Machine Learning Seminar, speaking on Distributional Generalization.
 Feb 2021: Simons Collaboration Monthly Meeting, speaking on The Deep Bootstrap. [slides]
 Aug 2020: Max Planck+UCLA Math ML Seminar, speaking on Distributional Generalization. [video]
Research
I take a scientific approach to machine learning— trying to advance understanding through basic experiments and foundational theory.See [publications] for full list of papers.
Machine Learning Theory

The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers
[slides]
Preetum Nakkiran, Behnam Neyshabur, Hanie Sedghi
ICLR 2021. 
Distributional Generalization: A New Kind of Generalization
[talk]
[slides]
Preetum Nakkiran*, Yamini Bansal*
 DeskRejected from NeurIPS 2020.
 Rejected from ICLR 2021. [reviews]
 Rejected from ICML 2021. [reviews] [rebuttal] [meta]
 In submission to NeurIPS 2021. 
Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems
Preetum Nakkiran
OPT2020 Workshop (Best Student Paper) 
Optimal Regularization Can Mitigate Double Descent
Preetum Nakkiran, Prayaag Venkat, Sham Kakade, Tengyu Ma
ICLR 2021. 
Deep Double Descent: Where Bigger Models and More Data Hurt
Preetum Nakkiran, Gal Kaplun*, Yamini Bansal*, Tristan Yang, Boaz Barak, Ilya Sutskever
ICLR 2020. 
More Data Can Hurt for Linear Regression:
Samplewise Double Descent
Preetum Nakkiran
Manuscript. 2019. 
SGD on Neural Networks Learns Functions of Increasing Complexity
Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L. Edelman, Fred Zhang, Boaz Barak
NeurIPS 2019 (Spotlight). 
Adversarial Examples are Just Bugs, Too
Preetum Nakkiran
Distill 2019. 
Adversarial Robustness May Be at Odds With Simplicity
Preetum Nakkiran
(Merged appears in COLT 2019). 
The Generic Holdout:
Preventing FalseDiscoveries in Adaptive Data Science
Preetum Nakkiran, Jarosław Błasiok
Manuscript. 2018.
Theory

Algorithmic Polarization for Hidden Markov Models
Venkatesan Guruswami, Preetum Nakkiran, Madhu Sudan
ITCS 2019. 
General Strong Polarization
Jarosław Błasiok, Venkatesan Guruswami, Preetum Nakkiran, Atri Rudra, Madhu Sudan
STOC 2018. 
Tracking the L2 Norm with Constant Update Time
ChiNing Chou, Zhixian Lei, Preetum Nakkiran
APPROXRANDOM 2018. 
NearOptimal UGChardness of Approximating Max kCSP_R
Pasin Manurangsi, Preetum Nakkiran, Luca Trevisan
APPROXRANDOM 2016.
Machine Learning

Compressing Deep Neural Networks Using a RankConstrained Topology
Preetum Nakkiran, Raziel Alvarez, Rohit Prabhavalkar, Carolina Parada
INTERSPEECH 2015. 
Automatic Gain Control and Multistyle Training for Robust SmallFootprint Keyword Spotting
with Deep Neural Networks
Rohit Prabhavalkar, Raziel Alvarez, Carolina Parada, Preetum Nakkiran, and Tara Sainath
ICASSP 2015.
About Me
I did my undergrad in EECS at UC Berkeley. I'm broadly interested in theory and science.In the past, I have interned at OpenAI (with Ilya Sutskever) Google Research (with Raziel Alvarez), Google Brain (with Behnam Neyshabur, Hanie Sedghi), and have also done research in errorcorrecting codes, distributed storage, and cryptography. I am partially supported by a Google PhD Fellowship, and I am grateful for past support from NSF GRFP.
See also my old website for more. This version borrowed in part from Luca Trevisan and Jon Barron.
What People are Saying
a "highlevel" scientist —colleague (ML)
makes plots and draws lines through them
—colleague (TCS)
has merits that outweigh flaws —reviewer 2