Computational Statistics, Machine Learning, et. al.

ESL: The Elements of Statistical Learning

I am currently working through "The Elements of Statistical Learning" (ESL). I thought that I might take the time to write some notes as I go through the book. In particular, I will try to reproduce most of analysis in the text using R. As a point of comparison, I might also comment on how sections compare to other common statistical/machine learning texts, especially to Christopher Bishop's "Pattern Recognition and Machine Learning" (PRML). Lastly, I will draw comparisons to coursework in this area at major universities. ESL is taught as part of STATS 315A and 315B at Stanford, which can be taken remotely as part of the SCPD certificate in data mining.

Some of the subjects covered: Regression and Classification, Shrinkage and Feature Selection, Support Vector and Kernel Methodology, Principal Components and Variations, Boosting, Random Forests and Ensemble Methods, Cross-Validation and Bootstrap. One theme that I find consistently when comparing statistical learning vs. machine learning coursework is that modern machine learning curriculum place a much greater emphasis on bayesian methods, especially graphical models. So, depending on a number of factors, I may digress to cover some Bayesian methods.


You can download a free PDF copy of "The Elements of Statistical Learning" (Hastie, Tibshirani and Friedman 2008) from the book website. For those of us using an iPad, this PDF looks very nice in iBooks. If possible, I also recommend buying the hard copy both to support the authors and because I'm old fashioned and like real books (see below).

The authors are all statistics professors at Stanford University and are all famous in their own right:

Hastie and Tibshirani are both involved in biostatistics research, which means that they are especially exposed to problem with very high dimensionality.

3 thoughts on “ESL: The Elements of Statistical Learning

Leave a Reply

Your email address will not be published. Required fields are marked *