Machine Learning

Linear Regression

Table of Contents Introduction Probabilistic Interpretation Solving with Normal Equations Another Approach to Normal Equations Fitting Polynomials Linear Basis Functions Slides for these notes are available here. Introduction Given a dataset of observations \(\mathbf{X} \in \mathbb{R}^{n \times d}\), where \(n\) is the number of samples and \(d\) represents the number of features per sample, and corresponding target values \(\mathbf{Y} \in \mathbb{R}^n\), create a simple prediction model which predicts the target value \(\mathbf{y}\) given a new observation \(\mathbf{x}\). The classic example in this case is a linear model, a function that is a linear combination of the input features and some weights \(\mathbf{w}\).