**Prerequisites**:

Background in applied mathematics, probability, and statistics

**Instructor**:

Robert Nowak

E-mail: nowak@engr.wisc.edu

Web: http://nowak.ece.wisc.edu

Phone: 608 265 3914

3627 Engineering Hall

Office Hours: email for appointment

**Lectures**:

May 26, 2009-Jul 19, 2009

Time/Place: 9am-12:30pm Wednesdays / MECH ENGR 1152

**Course Format**:

The course will meet once per week for 3.5 hours. Each meeting

period will be divided into two subperiods, each approximately 1.5 hours

in duration. We will take a short break between the subperiods. Each

subperiod will focus on one of the lectures below.

**Lectures**:

Lecture 1 A Probabilistic Approach to Pattern Recognition

Lecture 2 Introduction to Classification and Regression

Lecture 3 Introduction to Complexity Regularization

Lecture 4 Denoising in Smooth Function Spaces

Lecture 5 Plug-in Rules and Histogram Classifiers

Lecture 6 Probably Approximately Correct (PAC) Learning

Lecture 7 Chernoff’s Bound and Hoeffding’s Inequality

Lecture 8 Classification Error Bounds

Lecture 9 Error Bounds in Countably Infinite Models Spaces

Lecture 10 Complexity Regularization

Lecture 11 Decision Trees

Lecture 12 Complexity Regularization for Squared Error Loss

Lecture 13 Maximum Likelihood Estimation

Lecture 14 Maximum Likelihood and Complexity Regularization

Lecture 15 Denoising II: Adapting to Unknown Smoothness

Lecture 16 Wavelet Approximation Theory

Lecture 17 Denoising III: Spatial Adaptivity

Lecture 18 Introduction to VC Theory

Lecture 19 The VC Inequality

Lecture 20 Applications of VC Theory

**Homework Problems**: TBA

**Readings**: TBA

**Textbooks and References**:

A textbook will not be followed in this course. A collection of666

notes, relevant papers and materials will be prepared and distributed.

Textbooks recommended for further reading are listed below.

A probabilistic theory of pattern recognition, Devroye, Gyorfi, Lugosi, Springer

Nonparameteric Estimation Theory, Iain Johnstone, unpublished monograph

The Elements of Statistical Learning, Hastie, et al, Springer

An introduction to support vector machines, Cristianini and Shawe-Taylor, Cambridge Press

Combinatorial methods in density estimation, Devroye and Lugosi, Springer

Statistical Learning Theory, Vapnik, Wiley

An Introduction to Computational Learning Theory, Kearns and Vazirani, MIT Press

Empirical Processes in M-Estimation, van de Geer, Cambridge Press

**Grading and Evaluation**:

Grades will be based on course participation and lecture presentations.