Gradient AtlasML Materials

Machine Learning: a free course

Every classic machine learning algorithm built from scratch in NumPy, then checked against scikit-learn. 26 chapters, about 17 hours of reading. Part of the data science roadmap.

Orientation

  1. How to Use This MaterialScope, the per-concept template, numbering rules, datasets and determinism, reading order

Foundations

  1. Machine Learning FundamentalsWhat machine learning is, how it differs from ordinary programming, and the vocabulary and workflow every later chapter assumes.
  2. Mathematical FoundationsThe linear algebra, calculus, probability and statistics the algorithm chapters actually use — every idea computed in NumPy rather than only stated.
  3. The Python Machine Learning StackThe four libraries every later chapter imports, and the scikit-learn API contract that every estimator in this material obeys.

Data

  1. Data and Exploratory Data AnalysisWhere data comes from, what kinds of variables it holds, and how to interrogate it before a single model is fitted.
  2. Data PreprocessingTurning a raw table into something a model can be fitted on — without letting the test set influence a single decision.
  3. Feature EngineeringConstructing the representation the model actually learns from — encodings, transformations and derived features.
  4. Feature Selection and Dimensionality ReductionChoosing which features to keep, and constructing fewer features that carry the same information.

The Learning Framework

  1. Model Evaluation and ValidationMeasuring how well a model will do on data it has never seen — the machinery every later chapter reports its numbers through.
  2. Bias, Variance and RegularizationWhy models fail in two opposite directions, how to tell which is happening, and the general principle that fixes one of them.
  3. Optimization and Gradient DescentThe engine underneath most of the supervised chapters: a loss function, a gradient, and a rule for taking the next step.

Supervised Learning

  1. Linear RegressionThe first supervised model, derived from its objective, implemented twice from scratch, then matched against scikit-learn.
  2. Regularized Linear ModelsRidge, lasso and elastic net — the same linear model with a penalty term, and what that term does to the coefficients.
  3. Logistic RegressionThe linear model adapted to classification: a squashing function, a likelihood, and a decision boundary.
  4. k-Nearest Neighbors and Distance MetricsThe model that does no training at all — and the distance function that therefore decides everything.
  5. Naive BayesA probabilistic classifier built directly on Bayes' theorem, and the deliberately wrong assumption that makes it fast.
  6. Support Vector MachinesClassification as a geometry problem: find the widest possible separating band, and bend the space when no straight band exists.
  7. Decision TreesA model you can read: recursive splits chosen by impurity, and every knob that stops it memorizing the training set.
  8. Ensembles I: Bagging and Random ForestsMany high-variance models averaged into one low-variance model — why that works, and the two ingredients random forests add.
  9. Ensembles II: Boosting and StackingModels trained in sequence, each one correcting its predecessor — and the libraries that win most tabular competitions.

Unsupervised Learning

  1. ClusteringFinding structure with no labels to check against — four algorithms with genuinely different notions of what a cluster is.
  2. Anomaly Detection and Association RulesTwo unsupervised searches for rare joint events - which rows do not belong, and which items occur together more than chance allows - and the ways both mislead.

Practice

  1. Imbalanced Data and Data LeakageTwo causes of a score that is better than the model - the wrong metric, and information the model will not have at prediction time - each reproduced with its inflated number, then repaired.
  2. Pipelines and Hyperparameter TuningWiring every preprocessing step and the estimator into one object - then searching over the whole thing at once.
  3. Interpretability and DeploymentExplaining what a fitted model actually learned, then getting it out of the notebook and keeping it honest over time.

Appendix

  1. AppendixBookkeeping about the material itself, not machine-learning content.