Time and Location: 3-4pm in CH 212
Speaker: Yongdai Kim (Seoul National University, Korea)
Title: Fast learning with deep learning architectures for classification
Abstract: We derive the fast convergence rates of a deep neural network (DNN) classifier with the rectified linear unit (ReLU) activation function learned using the hinge loss. We consider three cases for a true model: (1) a smooth decision boundary, (2) smooth conditional class probability, and (3) the margin condition (i.e., the probability of inputs near the decision boundary is small). We show that the DNN classifier learned using the hinge loss achieves fast convergence rates for all three cases provided that the architecture (i.e., the number of layers, number of nodes and sparsity) is carefully selected. An important implication is that DNN architectures are very flexible for use in various cases without much modification. In addition, we consider a DNN classifier learned by minimizing the cross-entropy, and give conditions for fast convergence rates. If time is allowed, computational algorithms to achieve a right size of deep architectures for fast convergence rates is discussed.
This is joint work with Ph.D. students Ilsang Ohn and Dongha Kim.
Speaker: Shanshan Tu
Title: The Estimation of Prediction Error: Covariance Penalties and Cross-Validation
Reference: The Estimation of Prediction Error: Covariance Penalties and Cross-Validation by Bradley Efron
Speaker: Jianhao Zhang
Title: Optimization with Orthogonality Constraints
A feasible method for optimization with orthogonality constraints by Zaiwen Wen and Wotao Yin (2013)
Speaker: Chenxi Zhou
Title: Density Estimation in Reproducing Kernel Hilbert Space (RKHS)
Reference: Smoothing Spline Density Estimation: Theory (Gu and Qiu, 1993) and Smoothing Spline Density Estimation: A Dimensionless Automatic Algorithm (Gu, 1993)
Speaker: Jiae Kim
Title: Random Features for Kernel Functions
Reference: Random Features for Large-scale Kernel Machines
Speaker: Prateek Sasan
Title: Anchor Regression: heterogenous data meets causality
Reference: Anchor Regression: heterogenous data meets causality
Speaker: Abhijoy Saha
Title: An Introduction to Variational Bayesian Inference
Reference: Variational Inference: A Review for Statisticians
Statistical Learning and Data Mining reading group will meet this semester. Regular meetings are scheduled every other Tuesday starting from September 4th from 12:30 to 1:30 pm in Cockins Hall Room 212.
Students may register for course credit by enrolling in STAT 8750.01 – Research Group in Statistical Learning and Data Mining. If you want to be added to the reading group email list, contact Chenxi Zhou at firstname.lastname@example.org.