Regular Expression Fundamentals

Always use a raw string

import re

pattern = r"\d+\s\w+"        # raw: the backslashes reach the regex engine
bad = "\d+\s\w+"             # non-raw: Python tries to interpret \d, \s, \w first

print(re.findall(r"\bcat\b", "the cat sat"))     # ['cat']
print(len("\n"), len(r"\n"))                     # 1 2

Backslashes are meaningful to both Python and the regex engine. A raw string stops Python from touching them, so the pattern you wrote is the pattern that runs. Write r"..." for every regex, without exception.

Literal characters

import re

print(re.findall(r"cat", "the cat sat on the cat mat"))     # ['cat', 'cat']
print(re.findall(r"CAT", "the cat"))                        # [] - case sensitive
print(re.findall(r"CAT", "the cat", re.IGNORECASE))         # ['cat']

Character classes

import re

text = "Order A-42 shipped on 2026-08-22 for Rs 1,250."

print(re.findall(r"\d", text))            # every single digit
print(re.findall(r"\d+", text))           # runs of digits
print(re.findall(r"[A-Z]", text))         # capitals
print(re.findall(r"[aeiou]", text)[:5])   # vowels
print(re.findall(r"[^\w\s]", text))       # punctuation

Inside square brackets most special characters lose their meaning. [.+*] matches a literal dot, plus or asterisk. Only ^ (at the start), - (between characters) and ] need care.

Quantifiers

import re

print(re.findall(r"ab*c", "ac abc abbc abbbc"))       # all four
print(re.findall(r"ab+c", "ac abc abbc"))             # not 'ac'
print(re.findall(r"colou?r", "color colour"))         # both spellings
print(re.findall(r"\d{4}", "2026 and 42 and 12345"))  # ['2026', '1234']
print(re.findall(r"\b\d{4}\b", "2026 and 12345"))     # ['2026'] only

Greedy and lazy

import re

html = "<b>bold</b> and <i>italic</i>"

print(re.findall(r"<.+>", html))      # greedy: one huge match
print(re.findall(r"<.+?>", html))     # lazy: each tag separately
print(re.findall(r"<[^>]+>", html))   # better still: never cross a >

greedy  <.+>    -> ['<b>bold</b> and <i>italic</i>']
lazy    <.+?>   -> ['<b>', '</b>', '<i>', '</i>']

Quantifiers are greedy by default: they take as much as possible and then give characters back until the rest of the pattern fits. Adding ? makes them lazy: take as little as possible. A negated character class is usually clearer and faster than either.

Anchors and boundaries

import re

print(re.findall(r"^the", "the cat sat"))               # ['the']
print(re.findall(r"cat$", "the cat"))                   # ['cat']

text = "cat category concat"
print(re.findall(r"cat", text))                         # 3 matches
print(re.findall(r"\bcat\b", text))                     # 1 - the whole word only
print(re.findall(r"\bcat", text))                       # 2 - at word starts

lines = "first line\nsecond line\nthird line"
print(re.findall(r"^\w+", lines))                       # ['first']
print(re.findall(r"^\w+", lines, re.MULTILINE))         # all three

\b is the single most useful thing in this note. It is the difference between finding cat inside category and not.

Groups

import re

match = re.search(r"(\d{4})-(\d{2})-(\d{2})", "issued on 2026-08-22")

print(match.group(0))       # 2026-08-22 - the whole match
print(match.group(1))       # 2026
print(match.groups())       # ('2026', '08', '22')
print(match.start(), match.end())

Named groups

import re

pattern = r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})"
match = re.search(pattern, "issued on 2026-08-22")

print(match.group("year"))
print(match.groupdict())     # {'year': '2026', 'month': '08', 'day': '22'}

Named groups turn a match into a readable record. Prefer them to numbered groups for anything with more than two parts.

Non-capturing groups

import re

print(re.findall(r"(https?)://(\S+)", "see http://a.com and https://b.com"))
# [('http', 'a.com'), ('https', 'b.com')]

print(re.findall(r"(?:https?)://(\S+)", "see http://a.com and https://b.com"))
# ['a.com', 'b.com'] - the protocol is grouped but not captured

Use (?:...) when you need brackets for grouping or alternation but do not want the text back.

Alternation

import re

print(re.findall(r"cat|dog", "a cat and a dog"))
print(re.findall(r"\b(?:cat|dog|bird)s?\b", "cats dog birds catalogue"))
print(re.findall(r"^(?:GET|POST|PUT)\b", "POST /api", re.MULTILINE))

Lookahead and lookbehind

import re

# A number only when it is followed by "kg"
print(re.findall(r"\d+(?=\s*kg)", "5 kg, 10 lb, 20kg"))     # ['5', '20']

# A number NOT followed by a percent sign
print(re.findall(r"\d+(?!\d*%)", "50% and 30 items"))

# Text after a label, without capturing the label
print(re.findall(r"(?<=Total: )\d+", "Total: 250"))          # ['250']

# A word not preceded by "not "
print(re.findall(r"(?<!not )\bgood\b", "good and not good"))

import re

password = "Passw0rd!"

checks = {
    "at least 8 characters": r".{8,}",
    "a lowercase letter": r"[a-z]",
    "an uppercase letter": r"[A-Z]",
    "a digit": r"\d",
    "a symbol": r"[^\w\s]",
}

for label, pattern in checks.items():
    found = re.search(pattern, password) is not None
    print(f"{('ok' if found else 'MISSING'):<8} {label}")

Escaping

import re

print(re.findall(r"3.14", "3.14 and 3x14"))        # both - the dot is a wildcard
print(re.findall(r"3\.14", "3.14 and 3x14"))       # ['3.14'] only

user_input = "cost (in Rs)"
print(re.escape(user_input))                        # every special character escaped
print(re.findall(re.escape(user_input), "the cost (in Rs) is 5"))

The characters needing escaping outside brackets are . ^ $ * + ? { } [ ] \ | ( ). When a pattern comes from user input, always pass it through re.escape.

Flags

import re

text = "Line One\nLine Two"

print(re.findall(r"line", text, re.IGNORECASE))
print(re.findall(r"^line", text, re.IGNORECASE | re.MULTILINE))
print(re.findall(r"one.line", text, re.IGNORECASE | re.DOTALL))   # . crosses \n

import re

pattern = re.compile(r"""
    (?P<area>\d{3})      # the area code
    [-.\s]?              # an optional separator
    (?P<number>\d{7})    # the number
""", re.VERBOSE)

print(pattern.search("call 080-1234567").groupdict())

re.VERBOSE lets a pattern span lines with comments and ignored whitespace. Use it for anything longer than about forty characters - an unreadable regex is a maintenance problem, not a clever one.

Catastrophic backtracking

import re
import time

# Dangerous: nested quantifiers over overlapping alternatives
pattern = r"^(a+)+$"
text = "a" * 25 + "b"

start = time.perf_counter()
re.match(pattern, text)
print(f"{time.perf_counter() - start:.2f}s")      # seconds, or much worse

# Safe rewrite
print(re.match(r"^a+$", text))

A pattern such as (a+)+ can try an exponential number of ways to match before failing. On attacker supplied input this is a denial of service vulnerability. Avoid nesting quantifiers, and prefer a negated character class to .*.

When not to use a regex

import re

text = "name: Meera"

# A regex is overkill here
print(re.search(r"^name: (.+)$", text).group(1))

# String methods are clearer and faster
print(text.split(": ", 1)[1])
print(text.removeprefix("name: "))

print("Meera" in text)                # instead of re.search(r"Meera", text)
print(text.startswith("name"))        # instead of re.match(r"name", text)

• Fixed text: use in, startswith, endswith.
• A single separator: use split or partition.
• HTML, XML, JSON or CSV: use a real parser. Nested structures cannot be matched by a regular expression.
• A regex is right when the shape varies but follows a rule.

Common mistakes

• Not using a raw string.
• Forgetting \b and matching inside longer words.
• Using greedy .* where lazy or a negated class was needed.
• Forgetting to escape a literal dot.
• Using . and expecting it to match a newline without re.DOTALL.
• Building a pattern from user input without re.escape.
• Writing a fifty character pattern with no re.VERBOSE comments.

Best practices

• Always use raw strings.
• Use \b for whole words and anchors for whole strings.
• Use named groups, and (?:...) when you do not need the text.
• Use re.VERBOSE and comments for anything non-trivial.
• Prefer [^x]* to .*? when you know what must not be crossed.
• Test against strings that should not match, not only ones that should.

Practice

1. Write a pattern matching a date in the form DD/MM/YYYY with named groups.
2. Extract every word of exactly five letters from a paragraph.
3. Match a number only when it is preceded by a currency symbol.
4. Explain the difference between <.+>, <.+?> and <[^>]+> on a line of markup.
5. Write a pattern that finds the word test but not testing or latest.

Conclusion

Character classes describe what, quantifiers describe how many, anchors describe where, and groups decide what you keep. Write patterns as raw strings, use \b far more than you expect to, and reach for string methods when the text is fixed.