We considered the problem of overfitting as model complexity increase in the prior post. Now we look at one way to control for this problem: regularization. The basic idea is to penalize each the model, essentially saying that we don't entirely believe the fit that falls out of our optimization. Since we are fitting to …
Tag Archives: stanford-cs229
Stanford ML 5.1: Learning Theory and the Bias/Variance Trade-off
Data analysis is part science, part art. It is part algorithm and part heuristic. Of the various approaches to data analysis, machine learning falls more on the side of purely algorithmic, but even here we have many decisions to make which don't have well-defined answers (e.g. which learning algorithm to use, how to divide the …
Stanford ML 4: Logistic Regression and Classification
The initial lectures in Stanford CS229a were concerned with regression problems where the predicted value was a continuous number. Another class of problems is concerned with discrete problems, where values are divided into groups (e.g. on or off; red, green, or blue). This builds on all the material from the previous linear regression lectures. The …
Stanford ML 3: Multivariate Regression, Gradient Descent, and the Normal Equation
The next set of lectures in CS229 covers "Linear Regression with Multiple Variables", also known as Multivariate Regression. This builds on the univariate linear regression material and results in a more general procedure. As part of this, Professor Ng also provides more guidance on how to use Gradient Descent, and introduces the most widely used …
Stanford ML 2: Linear Algebra Review
Machine learning makes extensive usage of linear algebra, probability, and calculus. CS229 reviews basic linear algebra early on. If you're new to linear algebra, it's certainly worth spending time on; I use it extensively in my professional life. I might expand on this subject more over time, but for now I would just highlight a …
Stanford ML 1.2: Gradient Descent
For the first part of Stanford CS229a, we saw a simple linear model and how we could characterize the loss function as the mean-squared error. Professor Ng tried to build an intuition for the loss function by testing various different lines (varying and ) and seeing the subsequent shape of the loss. How can we …
Stanford ML 1.1: Introduction and Univariate Linear Regression
The first few lectures follow roughly section 1 of notes 1 from CS229 (section 1 and 2 in the video lectures). These lectures provide a brief overview with examples of machine learning (supervised and unsupervised) and then describes univariate linear regression as the first model. Machine Learning What is machine learning? Ng quotes Arthur Samuel …
Stanford ML: Code to Accompany the Lectures
As I mentioned previously, Stanford is offering an open course on Machine Learning which follows the CS229 curriculum. The online course (http://www.ml-class.org/) is actually not following the original CS229 "Machine Learning", but is more closely following the newly created CS229a "Applied Machine Learning". CS229a focuses more on applications and less on theory and mathematics. I …