1.1 What Machine Learning Is
Ordinary programming is the business of writing down a rule. You know the rule — charge 18% tax, reject a password under 8 characters — and your job is to express it precisely enough for a machine to follow.
Machine learning starts from the opposite position: the rule exists but nobody can write it down. No one can articulate the exact function that turns a photograph into "cat", or a customer's account history into "will cancel next month". What we can produce is examples of the function's behaviour — thousands of photographs already labelled, thousands of customers who already did or didn't cancel. Machine learning is the set of techniques for recovering the rule from those examples.
Tom Mitchell's 1997 definition is still the most precise one, and it is worth memorizing because it forces three separate questions to be answered before any code is written:
A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.
| Problem | Task (T) | Experience (E) | Performance (P) |
|---|---|---|---|
| Spam filtering | Label an email spam or not | A corpus of emails already marked by users | Fraction correctly labelled, weighted by the cost of a false positive |
| House pricing | Predict a sale price | Historical sales with their attributes | Average error in currency units |
| Customer churn | Predict who cancels next month | Past customers and whether they left | Recall on the customers who actually left |
A project that cannot answer all three has not yet been specified. "Use ML on our data" names E and nothing else, which is why it never converges.
The shift is easiest to see in code. Below, nobody writes down the relationship between x and y — the
data is generated from a rule the model is never shown, and the model recovers it anyway.
import numpy as np
from sklearn.linear_model import LinearRegression
RANDOM_STATE = 42
rng = np.random.default_rng(RANDOM_STATE)
# The "rule" - known to us, never shown to the model.
x = rng.uniform(0, 10, size=2000)
y = 3.0 * x + 7.0 + rng.normal(0, 1.5, size=2000) # true slope 3.0, true intercept 7.0
model = LinearRegression().fit(x.reshape(-1, 1), y)
print(f"learned slope {model.coef_[0]:.3f}") # 3.005 -> recovered from data alone
print(f"learned intercept {model.intercept_:.3f}") # 7.008
learned slope 3.005 learned intercept 7.008
3.0 and 7.0 appear nowhere in the call to fit(). They were inferred. That inference is the whole
subject — everything in the remaining 23 chapters is about doing it for rules more complicated than a
straight line, and about knowing whether the recovered rule is any good.
The catch that defines the entire field. A rule inferred from examples is only useful if it also holds for examples it has never seen. Recovering a rule that fits the training data perfectly is trivial — memorize it. The hard part, called generalization, is recovering one that works on new data. Chapter 8 is about measuring generalization and Chapter 9 is about what goes wrong with it; nearly every technique in this material exists because of it.
1.2 Machine Learning versus Rule-Based Programming
Both approaches produce a function from input to output. They differ in who writes it.
| Rule-based program | Machine learning model | |
|---|---|---|
| Logic comes from | A human, stated explicitly | Data, inferred by an algorithm |
| To change behaviour | Edit the code | Retrain on different data |
| Behaviour on unforeseen input | Undefined, or falls through to a default | Produces something — plausible or not |
| Correctness | Provable by reading the code | Estimated statistically, never proven |
| Debugging | Read the failing branch | Inspect data, features and metrics (Chapter 24) |
| Cost of a new edge case | One more if |
Usually nothing — if similar examples exist |
| Needs data | No | Yes, and its quality caps everything |
The trade is explicitness for coverage. A rule-based filter does exactly what it says and nothing else; a learned filter covers cases nobody enumerated, and occasionally does something inexplicable.
The failure mode of rules is not that they are wrong — it is that they are incomplete. Below, a keyword rule catches the spam it was designed for and misses a message that no one thought to enumerate.
def rule_based_is_spam(text):
"""Hand-written rule: three keywords a human thought of."""
keywords = ("free", "winner", "prize")
return any(word in text.lower() for word in keywords)
messages = [
"You are a WINNER! Claim your prize", # spam, and covered by the rule
"Free trial, click here", # spam, and covered by the rule
"Urgent: verify your account immediately", # spam, but nobody listed these words
"Are you free for a call tomorrow?", # legitimate - but contains "free"
]
for m in messages:
print(f"{rule_based_is_spam(m)!s:<6} {m}")
True You are a WINNER! Claim your prize True Free trial, click here False Urgent: verify your account immediately True Are you free for a call tomorrow?
Two distinct failures in four messages: a miss (no listed keyword) and a false alarm (a listed keyword used innocently). Patching both means adding rules, which adds new edge cases, which adds more rules. A learned classifier is fitted on labelled examples instead, so "urgent + verify + account" becomes evidence without anyone deciding it should be — that is Chapter 15's text-classification example.
Common mistake — reaching for a model where a rule is correct and cheaper. If the logic is known,
stable, and expressible in a few lines, write the few lines. A model trained to reproduce age >= 18 will
be slower, occasionally wrong, and impossible to audit — and it will need a data pipeline, a training job
and monitoring to keep doing something an if statement does exactly. Machine learning earns its
complexity when the rule is unknown or too intricate to state, not merely when data happens to be
available.
1.3 Types of Machine Learning
The taxonomy is organized around one question: what does the training data contain besides the inputs?
| Type | Training data | What is learned | Typical problems | Covered in |
|---|---|---|---|---|
| Supervised | Inputs X and targets y |
A mapping X → y |
Price prediction, churn, spam, diagnosis | Chapters 11–19 |
| Unsupervised | Inputs X only |
Structure within X |
Segmentation, anomaly detection, basket analysis | Chapters 20–21 |
| Semi-supervised | A little labelled X, y plus a lot of unlabelled X |
A mapping, using the unlabelled data to shape it | Anything where labelling is expensive | 1.3.3 |
| Self-supervised | Inputs only, with targets derived from them | Representations, by predicting part of the input from the rest | Language and vision pre-training | Out of scope (1.7) |
| Reinforcement | No fixed dataset — an environment returning rewards | A policy: which action to take in which state | Game playing, control, robotics | Out of scope (1.3.4) |
The boundary that matters most in practice is the first one, because it decides whether you can even measure success. With labels there is a right answer to score against (Chapter 8). Without them, evaluation is a genuinely open question — 20.8 covers why cluster quality metrics are so much weaker than a test-set score.
1.3.1 Supervised learning: regression and classification
Supervised learning splits again on the type of the target, and this split decides the model, the loss function and every metric downstream.
| Regression | Classification | |
|---|---|---|
Target y |
Continuous number | One of a finite set of classes |
| "How wrong" means | A distance (Chapter 11) | A mistake, possibly with unequal costs (8.5) |
| Example question | How much will this house sell for? | Will this customer cancel? |
| Chapters | 11, 12 | 13–16 |
The distinction lives entirely in y — the same feature matrix X can serve either.
from sklearn.datasets import make_regression, make_classification
X_reg, y_reg = make_regression(n_samples=100, n_features=3, noise=10.0,
random_state=RANDOM_STATE)
X_clf, y_clf = make_classification(n_samples=100, n_features=3, n_informative=3,
n_redundant=0, random_state=RANDOM_STATE)
print("regression y dtype:", y_reg.dtype, "| first 3:", y_reg[:3].round(1))
print("classification y dtype:", y_clf.dtype, "| unique:", np.unique(y_clf))
print("both X shapes:", X_reg.shape, X_clf.shape) # identical - only y differs
regression y dtype: float64 | first 3: [ 12.8 -21.9 91.1] classification y dtype: int64 | unique: [0 1] both X shapes: (100, 3) (100, 3)
1.3.2 Unsupervised learning
There is no y at all. The algorithm is asked to describe the structure it finds, and there is no answer
key to check it against.
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
X_blob, y_true = make_blobs(n_samples=150, centers=3, cluster_std=1.0,
random_state=RANDOM_STATE)
# y_true exists only so we can check the result here; the model never receives it.
labels = KMeans(n_clusters=3, n_init=10, random_state=RANDOM_STATE).fit_predict(X_blob)
print("labels found:", np.unique(labels)) # [0 1 2]
print("group sizes: ", np.bincount(labels)) # ~50 each - the three real groups
labels found: [0 1 2] group sizes: [50 50 50]
Note what fit_predict returned: group numbers, not names. Cluster 0 is not "the high-value segment" —
it is simply the first group the algorithm happened to number. Interpreting clusters is human work done
after the fact, and cluster identities are not stable across runs unless the seed is fixed. 20.8 covers how
to evaluate a clustering when there is nothing to compare it to.
1.3.3 Semi-supervised learning
Labels are usually the expensive part: unlabelled data accumulates for free, while labelling it requires a person. Semi-supervised methods use a small labelled set together with a large unlabelled one — the unlabelled points reveal where the data is dense, which constrains where a sensible boundary can run.
from sklearn.linear_model import LogisticRegression
from sklearn.semi_supervised import SelfTrainingClassifier
y_semi = y_clf.copy()
y_semi[20:] = -1 # -1 is scikit-learn's marker for "unlabelled": only 20 of 100 labels survive
supervised = LogisticRegression().fit(X_clf[:20], y_clf[:20]) # the 20 labels alone
semi = SelfTrainingClassifier(LogisticRegression()).fit(X_clf, y_semi) # 20 labels + 80 unlabelled
print(f"20 labels, supervised only : {supervised.score(X_clf, y_clf):.3f}")
print(f"20 labels + 80 unlabelled : {semi.score(X_clf, y_clf):.3f}")
20 labels, supervised only : 0.870 20 labels + 80 unlabelled : 0.890
1.3.4 Reinforcement learning
An agent takes actions in an environment and receives rewards; it learns a policy — a mapping from state to action — that maximizes reward accumulated over time. There is no dataset in the sense used everywhere else in this material: the agent's own actions determine which data it ever sees, which is what makes the problem so different.
It is named here for completeness and does not appear again. This material is about learning from a fixed dataset.
1.4 Core Terminology
The same object goes by several names depending on which textbook, library or team is talking. This table is the vocabulary used consistently for the rest of the material; the notation table in §25.3 of the index gives the mathematical symbols.
| Term | Also called | What it is |
|---|---|---|
| Instance | sample, observation, record, row, example | One thing being predicted about — one customer, one email |
| Feature | attribute, predictor, independent variable, column, covariate | One measured property of an instance |
Feature matrix X |
design matrix | All instances × all features, shape (n_samples, n_features) |
Target y |
label, response, dependent variable, ground truth, outcome | What the model predicts, shape (n_samples,) |
Prediction ŷ |
fitted value, output | What the model actually produced for an instance |
| Model | estimator, hypothesis, learner | The object that maps X to ŷ after fitting |
| Parameter | weight, coefficient | A value learned from data during fit() |
| Hyperparameter | setting, knob | A value chosen before fit(), never learned from the training data |
| Training set | — | The data the model fits on |
| Test set | hold-out set | Data withheld entirely, used once to estimate real-world performance (8.2) |
scikit-learn enforces the X / y shape convention everywhere: X is always 2-D, y is always 1-D.
from sklearn.datasets import load_iris
iris = load_iris()
X, y = iris.data, iris.target
print("X shape:", X.shape) # (150, 4) -> 150 instances, 4 features
print("y shape:", y.shape) # (150,) -> one target per instance
print("features:", iris.feature_names)
print("classes: ", iris.target_names)
X shape: (150, 4) y shape: (150,) features: ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'] classes: ['setosa' 'versicolor' 'virginica']
Common mistake — confusing parameters with hyperparameters. Both are numbers attached to a model, and
the words get used interchangeably in conversation, but they come from opposite directions: a
hyperparameter is an input you supply, a parameter is an output the fitting produces. In scikit-learn the
distinction is visible in the naming — attributes you set have plain names, and attributes learned during
fit() end with a trailing underscore.
from sklearn.linear_model import Ridge
ridge = Ridge(alpha=2.5) # alpha: HYPERparameter, chosen by us
print("before fit, alpha =", ridge.alpha)
print("before fit, has coef_?", hasattr(ridge, "coef_")) # False - nothing learned yet
ridge.fit(X_reg, y_reg)
print("after fit, coef_ =", ridge.coef_.round(2)) # parameters, LEARNED from the data
print("alpha is unchanged:", ridge.alpha) # 2.5 - fitting never touches it
before fit, alpha = 2.5 before fit, has coef_? False after fit, coef_ = [27. 72.32 18.03] alpha is unchanged: 2.5
The practical consequence: parameters are found by fitting, hyperparameters are found by searching — fitting the model repeatedly under different settings and comparing validation scores. That search is Chapter 23, and the reason it needs its own chapter is that doing it naively leaks the test set.
1.5 The End-to-End Machine Learning Workflow
Every project traverses the same stages. The arrows point forward, but real projects loop backwards constantly — a disappointing evaluation usually sends you back to features or data, not to a different algorithm.
business question
|
v
[1] frame the problem -----------------> what is T, E and P? (1.1)
|
v
[2] get and explore data ---------------> Chapter 4
|
v
[3] SPLIT ------------------------------> 5.1 - before anything is learned from the data
|
v
[4] clean and preprocess ---------------> Chapter 5 <--+
| |
v |
[5] engineer and select features -------> Chapters 6, 7 | loop until the
| | evaluation is
v | good enough
[6] choose and train a model -----------> Chapters 11-21 |
| |
v |
[7] evaluate and tune ------------------> Chapters 8, 23 --+
|
v
[8] interpret, deploy, monitor ---------> Chapter 24
| Stage | The characteristic failure | Chapter |
|---|---|---|
| Frame | Optimizing a metric that does not match the business cost | 8.5 |
| Explore | Not noticing that a column encodes the answer | 4.8, 22.7 |
| Split | Splitting after preprocessing, so the test set influenced training | 5.1, 22.7 |
| Preprocess | Fitting a scaler on all the data instead of just the training data | 5.5, 6.8 |
| Engineer | Building a feature that would not exist at prediction time | 22.7 |
| Train | Choosing the algorithm first and the problem second | 9.8 |
| Evaluate | Reporting the best of many scores as if it were an estimate | 8.8, 23.8 |
| Deploy | Assuming next year's data looks like last year's | 24.9 |
Note where the loop does not extend: back to step 3. The test set is touched once, at the end. Every time it informs a decision it stops being an estimate of unseen performance.
The whole workflow, at its smallest, is about fifteen lines. Every later chapter is an expansion of one of them.
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
# [2] get data
X, y = load_iris(return_X_y=True)
# [3] split FIRST - the test set is now untouchable until the last line
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, stratify=y, random_state=RANDOM_STATE
)
# [4]+[6] preprocess and train, wired together so the scaler only ever sees training data
model = make_pipeline(StandardScaler(), LogisticRegression())
model.fit(X_train, y_train)
# [7] evaluate, once
print(f"train accuracy: {model.score(X_train, y_train):.3f}") # 0.964
print(f"test accuracy: {model.score(X_test, y_test):.3f}") # 0.921 -> the only number that counts
train accuracy: 0.964 test accuracy: 0.921
In practice — the stages are nothing like equal in cost. Steps 2 through 5, the data work, routinely consume the large majority of a project's time; step 6, choosing and training the model, is often an afternoon. This is the opposite of how the subject is usually taught, where algorithms get the attention and data preparation gets a footnote. The chapter allocation here reflects the real ratio: four chapters on data (4–7) before a single algorithm is fitted in anger, and three more (22–24) on the things that make a result trustworthy afterwards.
1.6 When Machine Learning Is the Wrong Tool
Machine learning is a poor choice more often than its popularity suggests. The honest checklist:
| Condition | Why it disqualifies ML | Do this instead |
|---|---|---|
| The rule is already known and stable | A model can only approximate what an if states exactly |
Write the rule |
| Too few examples | Nothing to generalize from; the model fits noise (9.1) | Collect data, or use a heuristic |
| Labels do not exist and cannot be obtained | No supervised signal | Unsupervised framing (Ch. 20), or a rule |
| Every decision must be explained exactly | Most accurate models are not fully auditable (24.1) | A linear model, a shallow tree, or rules |
| Errors are catastrophic and unrecoverable | Models are wrong at some rate, always | Rules, with a human in the loop |
| The inputs at prediction time differ from training | The learned mapping does not apply (24.9) | Fix the data pipeline first |
| A constant guess is nearly as good | The features carry no usable signal | Ship the constant; revisit the features |
The last row is the one that catches real projects, and it is cheap to test: fit a model that ignores the
features entirely and see whether the real model beats it. DummyRegressor and DummyClassifier exist for
exactly this.
from sklearn.dummy import DummyRegressor
# 40 samples, 30 features, and NO relationship between them - pure noise.
noise_rng = np.random.default_rng(RANDOM_STATE) # its own generator, so this cell is self-contained
X_noise = noise_rng.normal(size=(40, 30))
y_noise = noise_rng.normal(size=40)
Xn_tr, Xn_te, yn_tr, yn_te = train_test_split(X_noise, y_noise, test_size=0.3,
random_state=RANDOM_STATE)
real = LinearRegression().fit(Xn_tr, yn_tr)
dummy = DummyRegressor(strategy="mean").fit(Xn_tr, yn_tr)
print(f"linear regression, TRAIN R^2: {real.score(Xn_tr, yn_tr):.3f}") # 1.000 - memorized exactly
print(f"linear regression, TEST R^2: {real.score(Xn_te, yn_te):.3f}") # -38.284 - catastrophic
print(f"predict-the-mean, TEST R^2: {dummy.score(Xn_te, yn_te):.3f}") # -0.052 - essentially baseline
linear regression, TRAIN R^2: 1.000 linear regression, TEST R^2: -38.284 predict-the-mean, TEST R^2: -0.052
The training score is perfect and the model is worthless: with 30 features and only 28 training rows there
is more than enough freedom to fit noise exactly. Predicting the mean — a model with no features at all —
scores -0.05 on unseen data against the fitted model's -38.3.
A negative R² is not a rounding artefact. R² is defined against the constant predictor: 0 means "no better
than predicting the target's mean", so anything below 0 is worse than a constant. Even the dummy scores
slightly below 0 here, because it predicts the mean of the training set, which is not exactly the mean
of the test set. -64.7 is in a different category entirely.
This is why 8.8 insists on a baseline. A score with nothing to compare it to is uninterpretable. "87% accuracy" is excellent if the baseline is 50% and embarrassing if 92% of the rows belong to one class — which is exactly the situation Chapter 22 is about.
1.7 Classical Machine Learning and the Deep Learning Boundary
"Classical" (or "traditional", or "shallow") machine learning means everything in this material: linear models, distance-based models, probabilistic models, kernel methods, trees and ensembles of trees. Deep learning means neural networks with many layers, trained end to end by backpropagation.
They are not competitors so much as tools for different data shapes:
| Classical ML | Deep learning | |
|---|---|---|
| Data size that works | Hundreds to a few million rows | Typically tens of thousands upward, ideally far more |
| Features | Engineered by a human (Chapter 6) | Learned by the network from raw input |
| Data it suits | Tabular, mixed types, structured | Images, audio, text, video — high-dimensional raw signal |
| Compute to train | Seconds to minutes, on a laptop | Hours to weeks, usually on GPUs |
| Interpretability | Ranges from complete to workable (Chapter 24) | Hard, and an active research area |
| Small-data behaviour | Degrades gracefully | Overfits badly without heavy regularization or pre-training |
| Tabular performance | Usually still the best choice | Rarely beats gradient boosting on tabular data |
The last row is the reason this material exists. For the structured, tabular problems that dominate commercial data work — churn, credit risk, demand forecasting, fraud, pricing — gradient-boosted trees (Chapter 19) remain the strongest default, and a well-specified linear model is often close behind at a fraction of the complexity. Deep learning's decisive wins are in perception: vision, speech and language, where the raw input has no natural tabular form and the useful features are exactly what nobody can specify by hand.
What this material excludes, explicitly: neural networks and backpropagation, convolutional and recurrent architectures, transformers and language models, embeddings, GPU training, and every framework built for them. Where a deep-learning method is genuinely the right answer, the text says so and stops rather than pretending a classical model is better.
What transfers if you go on to deep learning: almost all of Chapters 4–10. Train/test discipline, cross- validation, the bias–variance framing, regularization, gradient descent, loss functions, evaluation metrics and leakage are not classical-ML concepts — they are machine-learning concepts. The architecture changes; the surrounding discipline does not.
1.8 In Practice
Every chapter ends here: the chapter's material used the way it would be used at work. This chapter has no algorithm to apply yet, so the practical skill it teaches is the one that comes before any algorithm — turning a business request into a specified machine learning problem. Getting this wrong is the most expensive mistake in the workflow, because nothing downstream can correct it.
1.8.1 Where this shows up: framing a real request
The request, as it actually arrives:
"Our subscription business is losing customers. Can you use machine learning to fix churn?"
That sentence specifies nothing. Work it into 1.1's three answers before writing any code.
import pandas as pd
customers = pd.read_csv("data/customers.csv")
print(customers.shape) # (1012, 14)
print(list(customers.columns))
(1012, 14) ['customer_id', 'signup_date', 'age', 'gender', 'city', 'plan', 'tenure_months', 'monthly_charges', 'total_charges', 'support_tickets', 'last_login_days', 'satisfaction_score', 'churned', 'cancellation_reason']
Question 1 — what is an instance? One row here is one customer. That is a choice, not a fact: the same business could model one customer-month, which would give far more rows and a "will they leave in the next 30 days" target instead of a "have they left" one. Decide it explicitly; the label definition depends on it.
Question 2 — what is the target (T)? churned already exists, so this is supervised classification
(1.3.1). Check its balance immediately, because the balance decides which metric is honest.
counts = customers["churned"].value_counts()
print(counts)
print(f"churn rate: {customers['churned'].mean():.1%}") # 14.0% -> imbalanced
churned 0 870 1 142 Name: count, dtype: int64 churn rate: 14.0%
Question 3 — what is the performance measure (P)? Now the imbalance bites. A model that predicts "never churns" for every single customer is right 86% of the time and useless — it identifies nobody to intervene with. Accuracy is the wrong measure here, which is exactly 1.6's baseline argument arriving in a real problem.
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, recall_score
y_churn = customers["churned"]
never_churns = DummyClassifier(strategy="most_frequent").fit(customers[["tenure_months"]], y_churn)
pred = never_churns.predict(customers[["tenure_months"]])
print(f"accuracy of 'nobody ever churns': {accuracy_score(y_churn, pred):.3f}") # 0.860 - looks fine
print(f"recall of 'nobody ever churns': {recall_score(y_churn, pred):.3f}") # 0.000 - finds no one
accuracy of 'nobody ever churns': 0.860 recall of 'nobody ever churns': 0.000
The business cares about the second number: of the customers who were going to leave, how many did we identify in time to do something about it? That makes recall on the churning class the starting metric, traded against how many retention offers the company can afford to send to people who were never going to leave. 8.5 makes that trade precise and 22.2 returns to this exact table.
Question 4 — what is the experience (E), and is any of it cheating? This is where framing most often fails silently. A column can be perfectly predictive and completely useless.
# cancellation_reason is only ever populated for customers who already left.
print(customers.groupby("churned")["cancellation_reason"].apply(lambda s: s.notna().mean()))
churned 0 0.0 1 1.0 Name: cancellation_reason, dtype: float64
cancellation_reason is filled for 100% of churners and 0% of everyone else. A model given this column
scores near-perfectly and predicts nothing: the reason a customer cancelled is recorded because they
cancelled, so at prediction time — when you are trying to find customers who have not left yet — the column
is empty for everyone. This is target leakage, and it is the single most common cause of a model that
looks excellent in a notebook and does nothing in production. 22.7 is the full taxonomy.
The same question has to be asked of every column: would this value be known, in this form, at the moment
the prediction is needed? total_charges is borderline — it accumulates over a customer's life, so using
its final value for a churned customer bakes in how long they lasted.
The framing, written down. All four answers together:
| Decision | |
|---|---|
| Instance | One customer |
| Task (T) | Binary classification: will this customer churn? |
| Experience (E) | Historical customers, excluding cancellation_reason and any column not known at prediction time |
| Performance (P) | Recall on churners, at a false-positive rate the retention budget can absorb |
| Baseline to beat | "Nobody churns" — 86.0% accuracy, 0.0% recall |
Five lines, no modelling. A project that starts here converges; one that starts at "which algorithm should we use" does not.
1.8.2 When to reach for machine learning — and when not to
1.6 listed the disqualifying conditions. This is the complementary decision: given that ML is appropriate, which framing does the problem take?
| The situation | Framing | Chapters |
|---|---|---|
| Labelled examples of the exact outcome exist | Supervised — classification if the target is a category, regression if it is a number | 11–19 |
| The outcome is a number but only its rank matters | Still regression; evaluate on rank, not error | 8.4 |
| No labels, and the question is "what groups exist?" | Clustering | 20 |
| No labels, and the question is "what is unusual?" | Anomaly detection | 21 |
| No labels, and the question is "what goes together?" | Association rules | 21 |
| Labels exist but are scarce and expensive | Semi-supervised, or label a stratified sample first | 1.3.3 |
| Labels exist but are wrong or inconsistent | Stop. Fix the labels — no algorithm recovers from a corrupt target | — |
| The outcome depends on actions the system itself takes | Reinforcement learning — out of scope here | 1.3.4 |
| The decision must be fully explainable to a regulator | Supervised, restricted to interpretable models | 24.1–24.2 |
The seventh row is the one worth internalizing. Every other problem in this table is solvable with the material that follows; that one is not solvable at all, and recognizing it early saves months.
1.8.3 What goes wrong after deployment
The failures that a good notebook does not catch, because they happen later:
| Failure | What it looks like | Covered in |
|---|---|---|
| The metric did not match the business cost | Excellent scores, no measurable effect on revenue | 8.5, 8.8 |
| A feature was not available at prediction time | Great offline scores, the production pipeline cannot even populate the input | 22.7 |
| The label meant something different than assumed | "Churn" in the training data meant "closed account"; the business meant "stopped paying" | 1.8.1 |
| The world moved | Accuracy decays quietly over months as customer behaviour shifts | 24.9 |
| The model changed the thing it predicts | Retention offers sent to predicted churners change who churns; the next training set is contaminated by the model's own effects | 24.9 |
| Nobody is watching | The model fails silently and keeps returning confident predictions | 24.9 |
The last row is the difference between a model that is deployed and a model that is in production. A prediction service that has stopped being correct does not raise an exception — it keeps answering. What to monitor, and what to do when the monitors fire, is 24.9.