PCA Without the Fundamental Theorem of Algebra
A route to PCA that starts with geometry: compactness and Lagrange multipliers lead to the real spectral theorem, SVD, and optimal low-rank approximation.
The notebook
Extended lecture notes.
I sometimes expand lecture notes into a more systematic treatment, filling in proofs and connecting ideas. This is a selection of the notes I’ve extended.
Mathematics of machine learning
Gradient Descent: What Converges, and How Fast?
What can gradient descent guarantee, and what changes when gradients are noisy? Proofs trace how smoothness and convexity determine convergence and its speed.
Spectral Clustering Through Courant–Fischer
Why do eigenvectors help reveal clusters? An optimization view connects k-means to spectral clustering, with Courant–Fischer and random walks explaining the construction.
Ridge Regression and the Influence of a Single Point
How much can one observation change a regression fit? A geometric treatment of ridge regression leads to Cook’s distance through the Sherman–Morrison formula.
Reinforcement Learning I
Why does an optimal policy exist, and how can we find it? A fixed-point argument for finite discounted Markov decision processes leads to value iteration and policy iteration.
Probability and stochastic processes
From Martingales to Diffusion Generators
How can we describe when and where a diffusion leaves a region? Starting with conditional expectation and Itô calculus, these notes connect stochastic paths to boundary-value problems.
Mathematical finance
Black–Scholes Through Replication
How can a trading strategy determine an option’s price? Replication and risk-neutral valuation lead to Black–Scholes, followed by delta hedging and the question of early exercise.
These are personal notes, not official course materials. Sources are acknowledged in each piece. Corrections are welcome.