Regular Expressions

15.1 Regex Fundamentals

A regular expression (regex) is a mini pattern-language — itself just a string — that describes the shape of text rather than one exact piece of text. Where s == "A12" only matches that one literal string, the pattern [A-Z]\d{2} (Note: \d{2} = two digits — covered fully in 15.4–15.5) describes an entire family of strings: "one uppercase letter, then exactly two digits" — "A12", "B07", "Z99", and every other string with that shape, all at once.

General applications — the same small pattern language covers a surprising range of everyday text-processing tasks:

Application	What it looks like
Validation	does an input match an expected shape (email, phone number, product code)?
Searching / extraction	find every date, URL, or number buried in a block of text
Replacement / reformatting	rewrite matches (redact digits, normalize whitespace)
Splitting / tokenizing	break text apart on a pattern instead of one fixed delimiter

The win over plain string methods is expressing a shape in one declarative pattern, instead of several separate manual checks:

In [1]10 lines

import re

def is_valid_code_plain(s):   # plain string methods - two separate checks, both required
    return len(s) == 3 and s[0].isupper() and s[1:].isdigit()

def is_valid_code_regex(s):    # one pattern expresses the entire shape at once
    return bool(re.fullmatch(r"[A-Z]\d{2}", s))

print(is_valid_code_plain("A12"), is_valid_code_regex("A12"))   # True True
print(is_valid_code_plain("12A"), is_valid_code_regex("12A"))   # False False

True True
False False

In practice — parsing unstructured log lines. Pulling a timestamp, log level, or error code out of a line like 2026-08-24 14:03:12 ERROR payment failed for order 482 is one of the most common real uses of re in backend and DevOps work — log-analysis tools, alerting pipelines, and one-off "find every error in this file" scripts are almost always built on exactly this kind of pattern matching.

15.2 The `re` Module

Python's regex support lives entirely in the built-in re module — there's no special regex syntax in the language itself. A pattern is always just an ordinary str, passed as an argument to a function like re.search(pattern, text).

Common mistake — forgetting raw strings. Both Python string literals and regex patterns use backslash escapes, and they don't agree on what every escape means. A plain "\d" isn't a real Python escape sequence, so Python keeps it as the two characters \ and d — but warns about it, since that's rarely intentional. A raw string (r"...", 5.2.1) turns off Python's own escape processing entirely, so the backslash reaches re exactly as typed. The convention is to always write regex patterns as raw strings:

In [2]5 lines

plain_pattern = "\d{3}"   # SyntaxWarning: invalid escape sequence '\d' - works here, but by accident
raw_pattern = r"\d{3}"      # the correct, warning-free way to write any regex pattern

print(plain_pattern == raw_pattern)                 # True -> same two characters, in this particular case
print(bool(re.fullmatch(raw_pattern, "123")))   # True

<unknown>:1: SyntaxWarning: invalid escape sequence '\d'
True
True

re.compile(pattern) turns a pattern string into a reusable Pattern object. Calling a module-level function like re.search(pattern, text) directly still has to resolve pattern to a compiled form on every call — but re keeps an internal cache of the most recently compiled patterns (a few hundred entries), so repeated calls with the exact same pattern string are usually cache hits rather than a fresh compile every time. Even so, compiling once up front with a pattern that's used repeatedly (in a loop, or across many calls) is still clearer, guarantees no repeated compilation regardless of what else is filling the cache, and reads cleanly since the pattern only has to be written once. The resulting object exposes the same matching methods (.match(), .search(), .fullmatch(), ...) without needing the pattern passed in again:

In [3]5 lines

digit_pattern = re.compile(r"\d+")   # compiled once - reusable

print(type(digit_pattern))                       # <class 're.Pattern'>
print(digit_pattern.match("42 apples"))     # <re.Match object; span=(0, 2), match='42'>
print(digit_pattern.search("apples: 42"))   # <re.Match object; span=(8, 10), match='42'>

<class 're.Pattern'>
<re.Match object; span=(0, 2), match='42'>
<re.Match object; span=(8, 10), match='42'>

.finditer(text) returns every non-overlapping match as a lazy iterator (14.1, 14.2) of Match objects, instead of collecting them all into a list up front — findall() vs. finditer() is the same eager-vs-lazy trade-off as a list comprehension vs. a generator expression (14.5). Each Match object carries .group() (the matched text), .start()/.end() (its position), and .span() (both, as a tuple):

In [4]10 lines

text = "3 cats, 12 dogs, 7 birds"
matches = digit_pattern.finditer(text)   # a lazy iterator, nothing computed yet

print(type(matches))   # <class 'callable_iterator'>

for m in matches:
    print(m.group(), m.span())
# 3 (0, 1)
# 12 (8, 10)
# 7 (17, 18)

<class 'callable_iterator'>
3 (0, 1)
12 (8, 10)
7 (17, 18)

15.3 Character Classes

A character class — [...] — matches exactly one character, chosen from whatever is listed inside the brackets. A range like a-z or 0-9 can stand in for listing every character individually, and ranges can be combined in one class:

In [5]5 lines

text = "The Quick Brown Fox"
print(re.findall(r"[aeiouAEIOU]", text))   # every vowel, one character per match

color = "#1a2B3c"
print(bool(re.fullmatch(r"#[0-9a-fA-F]{6}", color)))   # True -> a valid 6-digit hex color

['e', 'u', 'i', 'o', 'o']
True

Putting ^ first inside the brackets negates the class — it then matches any character not in the set:

In [6]3 lines

phone = "call (555) 123-4567 now"
digits_only = re.sub(r"[^0-9]", "", phone)   # replace every character that's NOT a digit with ""
print(digits_only)   # 5551234567

5551234567

15.4 Predefined Character Classes

Some character classes come up so often that re provides a one-character shorthand for each — no brackets needed:

Shorthand	Equivalent to	Matches
`\d`	`[0-9]`	a digit
`\D`	`[^0-9]`	a non-digit
`\w`	`[a-zA-Z0-9_]`	a "word" character (letter, digit, or underscore)
`\W`	`[^a-zA-Z0-9_]`	a non-word character
`\s`	`[ \t\n\r\f\v]`	a whitespace character
`\S`	`[^ \t\n\r\f\v]`	a non-whitespace character
`.`	—	any character except a newline

Each is a drop-in replacement anywhere a character class could go, and combines with quantifiers (15.5) exactly the same way:

In [7]6 lines

text = "Order #482: 3 items, ship by 2026-08-24"

print(re.findall(r"\d+", text))   # ['482', '3', '2026', '08', '24'] -> runs of digits
print(re.findall(r"\w+", text))   # ['Order', '482', '3', 'items', 'ship', 'by', '2026', '08', '24']
print(re.split(r"\s+", text))       # split on any run of whitespace
print(re.findall(r"\D+", text))   # runs of NON-digit characters, punctuation and spaces included

['482', '3', '2026', '08', '24']
['Order', '482', '3', 'items', 'ship', 'by', '2026', '08', '24']
['Order', '#482:', '3', 'items,', 'ship', 'by', '2026-08-24']
['Order #', ': ', ' items, ship by ', '-', '-']

Going deeper — these shorthands are Unicode-aware by default. In Python 3, \w, \d, and \s don't just mean "ASCII letters/digits/whitespace" — by default they match the full Unicode categories (any Unicode digit, not just 0-9; any Unicode word character, not just a-z/A-Z/0-9/_). That's usually what you want for general text, but it matters when validating something like a username or product ID against unexpected Unicode look-alikes: pass re.ASCII to restrict \w/\d/\s/\b back to plain ASCII:

In [8]4 lines

text = "user_٣٤٥"   # trailing characters are Arabic-Indic digits (Unicode), not ASCII 0-9

print(re.findall(r"\d+", text))                    # ['٣٤٥'] -> Unicode-aware by default, this counts as digits
print(re.findall(r"\d+", text, flags=re.ASCII))   # [] -> re.ASCII restricts \d (and \w, \s, \b) to plain ASCII

['٣٤٥']
[]

Common mistake — treating \b as a character class. \b (word boundary) looks like it belongs on this list, but it's different in kind: it matches a position (between a \w character and a non-\w character), not an actual character, so it never consumes anything itself. Without it, a plain search for "cat" matches inside unrelated words too:

In [9]3 lines

text = "cat catalog concatenate"
print(re.findall(r"cat", text))          # ['cat', 'cat', 'cat'] -> matches inside other words too
print(re.findall(r"\bcat\b", text))   # ['cat'] -> only the whole word

['cat', 'cat', 'cat']
['cat']

In practice — a content filter that shouldn't flag substrings. A moderation or log-scrubbing filter that needs to catch the standalone word "ass" or a banned username shouldn't also flag it inside "class" or "assassin" — \b (or, more robustly, \w+-based tokenizing) is exactly what a real filter uses to avoid this class of false positive.

15.5 Quantifiers

A quantifier attaches to whatever comes right before it (a literal character, a character class, or a group) and says how many times to repeat it:

Quantifier	Means
`*`	0 or more
`+`	1 or more
`?`	0 or 1 (optional)
`{n}`	exactly `n`
`{n,}`	`n` or more
`{n,m}`	between `n` and `m` (inclusive)

In [10]5 lines

print(re.findall(r"ab*", "a ab abb abbb"))       # * -> 0+ 'b': ['a', 'ab', 'abb', 'abbb']
print(re.findall(r"ab+", "a ab abb abbb"))       # + -> 1+ 'b': ['ab', 'abb', 'abbb'] ("a" alone drops out)
print(re.findall(r"colou?r", "color colour"))   # ? -> optional 'u': ['color', 'colour']
print(re.findall(r"\d{4}", "2026-08-24"))         # {4} -> exactly 4 digits: ['2026']
print(re.findall(r"\d{2,4}", "12 123 1234"))     # {2,4} -> between 2 and 4 digits

['a', 'ab', 'abb', 'abbb']
['ab', 'abb', 'abbb']
['color', 'colour']
['2026']
['12', '123', '1234']

In practice — cleaning messy spreadsheet/CSV data before loading it. Real-world exports routinely have inconsistent whitespace or delimiters — "John Smith", "a,b ,, c" — and a data-cleaning step in an ETL pipeline normalizes them with exactly this kind of quantifier-based pattern (re.sub(r"\s+", " ", text)) before the data is loaded into a database or handed to downstream analysis.

Greedy vs. lazy. Every quantifier above is greedy by default — it matches as much text as it possibly can. Appending ? to a quantifier (*?, +?, ??) makes it lazy instead — it matches as little as it can get away with. The difference is easy to miss until it silently swallows far more than intended:

In [11]4 lines

html = "<b>bold</b> and <i>italic</i>"

print(re.findall(r"<.*>", html))     # greedy - .* grabs as much as possible, spans BOTH tags
print(re.findall(r"<.*?>", html))   # lazy - the trailing ? stops at the FIRST '>' instead

['<b>bold</b> and <i>italic</i>']
['<b>', '</b>', '<i>', '</i>']

Common mistake — catastrophic backtracking (ReDoS). A pattern with nested quantifiers — like (a+)+ — can force the regex engine to try an exponential number of ways to split the same run of characters before it can conclude a match ultimately fails. On a short mismatching input this is invisible; on a moderately longer one, matching time explodes and the program appears to hang. This is exactly the risk 15.1 flagged under "validation of untrusted input": a user-supplied string, not a pattern you wrote, ends up deciding how long the engine grinds:

In [12]12 lines

import time

pattern = r"(a+)+b"   # nested quantifier - the classic ReDoS shape
for n in [10, 15, 20, 22, 24]:
    s = "a" * n + "c"   # deliberately does NOT end in 'b' - forces exhaustive backtracking
    start = time.perf_counter()
    re.match(pattern, s)
    elapsed = time.perf_counter() - start
    print(n, f"{elapsed:.4f}s")
# time roughly triples to quadruples with each 2 extra characters here; scaled up to
# realistic input lengths (dozens of characters), the same pattern can hang for minutes
# or effectively forever - never run an unbounded nested quantifier over untrusted input

15.6 The Full `re` Module Function Catalog

Every top-level function re provides, in one place — compile() and finditer() were already covered in depth in 15.2:

Function	Does
`re.match(pattern, s)`	matches only at the start of `s`
`re.fullmatch(pattern, s)`	the entire string must match
`re.search(pattern, s)`	finds the first match anywhere in `s`
`re.findall(pattern, s)`	returns every match as a `list` of strings
`re.finditer(pattern, s)`	returns every match as a lazy iterator of `Match` objects (15.2)
`re.sub(pattern, repl, s)`	returns `s` with every match replaced by `repl`
`re.subn(pattern, repl, s)`	same as `sub()`, plus the number of replacements made
`re.split(pattern, s)`	splits `s` wherever `pattern` matches
`re.compile(pattern)`	precompiles into a reusable `Pattern` object (15.2)
`re.escape(s)`	escapes every regex-special character in `s`, for safe use as a literal pattern

match/fullmatch/search/findall side by side, on the same input, showing exactly how each one differs:

In [13]9 lines

print(re.match(r"\d+", "42 apples"))        # <re.Match object; span=(0, 2), match='42'>
print(re.match(r"\d+", "apples 42"))        # None -> doesn't start with a digit

print(re.fullmatch(r"\d+", "42"))              # <re.Match object; span=(0, 2), match='42'>
print(re.fullmatch(r"\d+", "42 apples"))    # None -> trailing text breaks a full match

print(re.search(r"\d+", "apples 42"))        # <re.Match object; span=(7, 9), match='42'>

print(re.findall(r"\d+", "3 cats, 12 dogs"))   # ['3', '12']

<re.Match object; span=(0, 2), match='42'>
None
<re.Match object; span=(0, 2), match='42'>
None
<re.Match object; span=(7, 9), match='42'>
['3', '12']

sub/subn/split/escape, each with a minimal example:

In [14]8 lines

print(re.sub(r"\d+", "#", "3 cats, 12 dogs"))     # # cats, # dogs -> every match replaced
print(re.subn(r"\d+", "#", "3 cats, 12 dogs"))     # ('# cats, # dogs', 2) -> plus a replacement count

print(re.split(r"\s*,\s*", "red, green,blue ,  yellow"))   # ['red', 'green', 'blue', 'yellow']

user_input = "3.14 (approx.)"
print(re.escape(user_input))                                          # every special char escaped
print(bool(re.fullmatch(re.escape(user_input), user_input)))   # True -> now safe as a literal pattern

# cats, # dogs
('# cats, # dogs', 2)
['red', 'green', 'blue', 'yellow']
3\.14\ \(approx\.\)
True

15.7 Capturing Groups

Every example so far has treated a match as one solid block of text — useful for validation, but real extraction usually needs a piece of what matched, not the whole thing. Wrapping part of a pattern in parentheses (...) turns it into a capturing group: it doesn't change what the pattern matches, but it tells re to remember exactly which substring matched that part, retrievable afterward from the Match object. Groups are numbered left-to-right by their opening parenthesis, starting at 1 — group 0 (via .group(0), the same as plain .group()) is always the entire match:

In [15]7 lines

m = re.search(r"(\d{4})-(\d{2})-(\d{2})", "Shipped on 2026-08-24 today")

print(m.group(0))   # 2026-08-24 -> the whole match, same as m.group()
print(m.group(1))   # 2026 -> year
print(m.group(2))   # 08 -> month
print(m.group(3))   # 24 -> day
print(m.groups())     # ('2026', '08', '24') -> every captured group, as a tuple

2026-08-24
2026
08
24
('2026', '08', '24')

Numbering groups by position works, but it's fragile the moment the pattern changes — insert one more group earlier in the pattern and every later .group(n) call now refers to the wrong thing. (?P<name>...) names a group instead of numbering it, so the intent stays readable in the pattern itself, and the code keeps working even if the pattern is edited later. .group("name") retrieves one named group by name; .groupdict() returns all named groups at once as a dict:

In [16]4 lines

m = re.search(r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})", "Shipped on 2026-08-24 today")

print(m.group("year"))   # 2026 -> retrieve by name instead of position
print(m.groupdict())        # {'year': '2026', 'month': '08', 'day': '24'}

2026
{'year': '2026', 'month': '08', 'day': '24'}

In practice — extracting structured pieces from a web form field. A signup form that accepts a free-text address needs to pull out a ZIP/postal code, or a support ticket parser needs to extract an order number embedded in a subject line — named capturing groups like year/month/day above are exactly how a real validation layer both checks the shape and pulls out the specific pieces it actually needs.

Sometimes parentheses are only needed to group a quantifier or an alternation (below) over several characters — not to capture anything for later. (?:...) is a non-capturing group: it groups without adding an entry to .groups()/.group(n), which keeps group numbering simple in patterns that need grouping purely for structure:

Group type	Syntax	How to retrieve it
Indexed (capturing)	`(...)`	`.group(1)`, `.group(2)`, ... or `.groups()`
Named (capturing)	`(?P<name>...)`	`.group("name")` or `.groupdict()`
Non-capturing	`(?:...)`	not retrievable at all — doesn't consume a group number

In [17]3 lines

m = re.fullmatch(r"(?:https?|ftp)://\S+", "https://example.com")
print(bool(m))     # True -> matches fine
print(m.groups())   # () -> the (?:...) grouped the alternation but captured nothing

True
()

Common mistake — assuming .group() always returns a string. An optional capturing group can legitimately match zero times — in that case .group(n) returns None for it, not an error, because the group is a real part of the pattern that simply didn't participate in this particular match. Asking for a group number that doesn't exist in the pattern at all is a different situation, and raises IndexError instead:

In [18]9 lines

m = re.match(r"(\d+)(-(\d+))?", "42")
print(m.group(1))   # 42
print(m.group(2))   # None -> the optional "-<digits>" part never matched, but the group is real
print(m.group(3))   # None -> same story for the group nested inside it

try:
    m.group(4)   # there IS no group 4 in this pattern at all
except IndexError as e:
    print("IndexError:", e)   # no such group

42
None
None
IndexError: no such group

A few more building blocks round out the toolkit:

Alternation (|) — matches either side, like a regex "or": cat|dog matches "cat" or "dog".
Anchors (^, $) — match a position, not a character: ^ anchors to the start of the string (or of each line, with re.MULTILINE below), $ to the end. This is a different meaning from the ^ seen in 15.3 — inside [...] it negates a character class; outside of one, it's a position anchor.
re.IGNORECASE — matches regardless of letter case.
re.MULTILINE — makes ^/$ match at the start/end of each line within the string, not just the string as a whole.

One line each:

In [19]8 lines

print(re.findall(r"cat|dog", "I have a cat and a dog"))   # ['cat', 'dog']

print(re.findall(r"^\d+", "42 apples, 7 oranges"))   # ['42'] -> only a match right at the start

print(bool(re.search(r"python", "I love Python", flags=re.IGNORECASE)))   # True -> case ignored

text_ml = "line1\nline2\nline3"
print(re.findall(r"^line\d", text_ml, flags=re.MULTILINE))   # ['line1', 'line2', 'line3'] -> per line, not just string-start

['cat', 'dog']
['42']
True
['line1', 'line2', 'line3']

15.1 Regex Fundamentals

15.2 The re Module

15.3 Character Classes

15.4 Predefined Character Classes

15.5 Quantifiers

15.6 The Full re Module Function Catalog

15.7 Capturing Groups

15.2 The `re` Module

15.6 The Full `re` Module Function Catalog