Statistical Learning Reading Group | Department of Statistics

2024 Summer Reading

May 20, 2024July 12, 2024

by Yoonkyung Lee at 11:25amJuly 12, 2024

A group of students will read papers on case influence, model sensitivity and related topics.

Case Influence and Model Complexity in Regression and Classification by Shanshan Tu
Influence analysis based on the case sensitivity function by Critchley et al.
General conditions for predictivity in learning theory by Poggio et al.
The unreasonable effectiveness of deep learning in artificial intelligence by Terry Sejnowski
Understanding Black-box Predictions via Influence Function by Koh and Liang
Data Shapley: Equitable Valuation of Data for Machine Learning by Ghorbani and Zou
Sobol’ Indices and Shapley Value by Art Owen

2023 Autumn Statistical Learning Reading Group

September 4, 2023October 15, 2023

by Yoonkyung Lee at 9:14pmOctober 15, 2023

Statistical Learning reading group will meet this semester from 11:00am to 12:30 pm in Scott Lab N056 every other Tuesday starting from September 5th. Students may register for course credit by enrolling in STAT 8750.01. If you are not officially enrolled but want to be added to the reading group email list, please contact Haozhen Yu at yu.2823@osu.edu.

This semester we will read the book, Reinforcement Learning: an Introduction by Sutton and Barto.

September 5: Haozhen Yu, Torey Hilbert and Qian Zhou will discuss Chapters 1 and 2.
September 19: Fangyi Wang and Xuerong Wang will discuss Chapter 3.
October 3: Wenxin Du and Rezoanoor Rahman will discuss Chapter 4.
October 17: Arkajyoti Bhattacharjee and Ningyi Liu will discuss Chapter 5.
October 31: Yingyu Cheng and Meghna Kalra will discuss Chapter 5.
November 14: Yue Ma and Zhenbang Jiao will discuss Chapter 6.
November 28: Biqing Yang and Xinyu Zhang will discuss Chapter 6.

Weekly Topics and Presenters (SP23 Stat 8750.01, Part II)

March 23, 2023March 23, 2023

by vu.104 at 2:21pmMarch 23, 2023

In the second part of the term we will learn a little bit about Conformal Prediction. Here is the schedule of presentations.

March 23 – Vincent Vu: Introduction to Conformal Prediction
March 30 – Haozhen Yu: Lei et al. (2018) “Distribution-Free Predictive Inference for Regression“
April 6 – Xuerong Wang: Classification with conformal predictors (based on Chapter 3 of Algorithmic Learning in a Random World and Angelopoulos & Bates (2022) “A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification“)
April 13 – Torey Hilbert & Meghna Karl: Barber et al. (2022) “Conformal Prediction Beyond Exchangeability“

Weekly Topics and Presenters (SP23 Stat 8750.01, Part I)

February 2, 2023October 15, 2023

by Yuan Zhang at 2:25pmOctober 15, 2023

Time	Presenter(s)	Topic
19-Jan	Meijia Shao	Motivation, concepts, definitions, basic examples
26-Jan	Fangyi Wang	Randomized responses
2-Feb	Zhizhen Zhao	Laplace mechanism
9-Feb	Xuerong Wang	Post-processing, group privacy, composition
16-Feb	Chenze Li	Exponential mechanism
23-Feb	Zhenbang Jiao	Report noisy max
2-Mar	Fangyi Wang	Gaussian Differential Privacy (JRSS-B discussion paper)
9-Mar	Yuan Zhang	Gaussian Differential Privacy (JRSS-B discussion paper)

2023 Spring Statistical Learning Reading Group (Part I)

January 18, 2023

by Yuan Zhang at 3:48pm

The first part of the reading group (8 meetings before spring break) will focus on the topic of differential privacy (DP). Here are some materials for your reference (mostly “top-k” Google results).

Textbooks

Dwork & Roth book

Salil Vadhan tutorial

Websites

Marco Gaboardi course

We will meet weekly in Enarson 258 on Thursdays, 11:30–12:25.

Students may register for course credit by enrolling in STAT 8750.01. If you are not officially enrolled but want to be added to the reading group email list, please contact Haozhen Yu at yu.2823@osu.edu.

2022 Autumn Reading

September 13, 2022November 22, 2022

by Yoonkyung Lee at 9:02pmNovember 22, 2022

Statistical Learning reading group will meet this semester after a long hiatus! We will meet from 11:30am to 12:30 pm in Cockins Hall Room 212 every other Tuesday starting from September 6th. Students may register for course credit by enrolling in STAT 8750.01. If you are not officially enrolled but want to be added to the reading group email list, please contact Haozhen Yu at yu.2823@osu.edu.

September 6: Welcome, organization, and discussion of
Prediction, estimation, and attribution by Brad Efron (2020);
Watch his recorded presentation for the 62nd ISI World Statistics Congress in Kuala Lumpur
September 20: Torey Hilbert and Xuerong Wang will discuss Sections 1-3 of the paper: Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation by Mikhail Belkin (2021)
October 4: Haozhen Yu and Zhenbang Jiao will discuss Sections 4-6 of the paper: Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation by Mikhail Belkin (2021)
October 18: Lun Li and Ningyi Liu will discuss Sections 1-2.2.1 of the following paper focusing on the concept of kernels: Kernel methods in machine learning by Thomas Hofmann, Bernhard Schölkopf, and Alexander J. Smola (2008)
November 1: Yue Ma and Jiahao Ping will discuss the paper: Deep neural networks as Gaussian processes by Jaehoon Lee, Yasaman Bahri, Roman Novak , Samuel S. Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein (2018)
November 15: Chenze Li and Cheng-Yu Hung will discuss the paper: Wide neural networks of any depth evolve as linear models under gradient descent by Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington (2019)
November 29: Arko Gupta and Nick Mandarano will discuss the following papers on word embedding: Distributed representations of words and phrases and their compositionality by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean (2013) and Efficient estimation of word representations in vector space by Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean (2013).

September 13: Seminar – Arnab Auddy

September 12, 2022September 12, 2022

by Yoonkyung Lee at 12:42amSeptember 12, 2022

Time and Location: September 13 (Tuesday) 11:30am-12:30pm in CH 212

Speaker: Arnab Auddy (Columbia University)

Title: Why and how to use orthogonally decomposable tensors for statistical learning

Abstract: As we encounter more and more complex data generating mechanisms, it becomes necessary to model higher order interactions among the observed variables. Orthogonally decomposable tensors provide a unified framework for such modeling in a number of interesting statistical problems. While this is a natural extension of matrix SVD to tensors, they automatically provide much better identifiability properties. Moreover, a small perturbation affects each singular vector in isolation, and hence their recovery does not depend on the gap between consecutive singular values. In addition to the attractive statistical properties, the tensor decomposition problem in this case presents us with intriguing computational challenges. To understand these better, we will explore some statistical-computational tradeoffs, and also describe tractable methods that provide rate optimal estimators for the tensor singular vectors.

2022 Summer Reading

July 18, 2022August 1, 2022

by Yoonkyung Lee at 5:28pmAugust 1, 2022

A group of students will read papers on interesting risk behaviors of prediction models in modern overparameterized regimes (e.g. deep learning).

July 18 (M): Reconciling modern machine-learning practice and the classical bias–variance trade-off by Belkin et al.
July 21 (R): Understanding Deep Learning (Still) Requires Rethinking Generalization by Zhang et al. (Discussion led by Lloyd Goldstein and Xuerong Wang)
July 25 (M): The Implicit Bias of Gradient Descent on Separable Data by Soudry et al. (Discussion led by Arkobrato Gupta and Yumo Peng)
July 28 (R): Two Models of Double Descent for Weak Features by Belkin et al. (Discussion led by Haotian Xie and Sayoni Roychowdhury)
August 1 (M): How Many Variables Should be Entered in a Regression Equation? by Breiman and Freedman (Read and discussed by all)

April 24: Seminar – Yongdai Kim

April 16, 2019September 12, 2022

by Yoonkyung Lee at 5:13pmSeptember 12, 2022

Time and Location: 3-4pm in CH 212

Speaker: Yongdai Kim (Seoul National University, Korea)

Title: Fast learning with deep learning architectures for classification

Abstract: We derive the fast convergence rates of a deep neural network (DNN) classifier with the rectified linear unit (ReLU) activation function learned using the hinge loss. We consider three cases for a true model: (1) a smooth decision boundary, (2) smooth conditional class probability, and (3) the margin condition (i.e., the probability of inputs near the decision boundary is small). We show that the DNN classifier learned using the hinge loss achieves fast convergence rates for all three cases provided that the architecture (i.e., the number of layers, number of nodes and sparsity) is carefully selected. An important implication is that DNN architectures are very flexible for use in various cases without much modification. In addition, we consider a DNN classifier learned by minimizing the cross-entropy, and give conditions for fast convergence rates. If time is allowed, computational algorithms to achieve a right size of deep architectures for fast convergence rates is discussed.

This is joint work with Ph.D. students Ilsang Ohn and Dongha Kim.

November 27: Shanshan Tu

November 13, 2018November 22, 2018

by zhou.779 at 4:03pmNovember 22, 2018

Speaker: Shanshan Tu

Title: The Estimation of Prediction Error: Covariance Penalties and Cross-Validation

Reference: The Estimation of Prediction Error: Covariance Penalties and Cross-Validation by Bradley Efron

Posts