intermediate30 minby DevFox

Regular Expressions in Moderation

Enough regex to be useful — groups, anchors, greediness, compiled patterns — plus the cases where a parser, a split, or three lines of plain code is the correct answer.

Python

Someone needs to pull the date out of filenames like 2026-07-29_sunrise.jpg. You write a regular expression. It works. Two weeks later a filename arrives with a different shape, the expression matches something it should not, and you are staring at forty characters of punctuation trying to remember what you meant by it.

Regular expressions are genuinely useful and routinely reached for when three lines of ordinary code would be clearer, safer and faster. By the end of this lesson you will write the patterns worth writing, know the two that can hang your server, and — the more valuable half — recognise the jobs that are not regex jobs.

The pieces

A regular expression is a pattern describing a set of strings. Python's are in the re module:

python

import re

match = re.search(r"\d{4}-\d{2}-\d{2}", "2026-07-29_sunrise.jpg")
print(match.group())        # 2026-07-29

The r prefix is not optional in practice. A raw string stops Python interpreting backslashes before the regex engine sees them, and without it "\d" and "\b" mean something else entirely. Always write r"..." for a pattern.

The notation you will actually use:

text

.        any character except a newline
\d \w \s a digit, a word character, whitespace
\D \W \S the opposite of each
[abc]    any one of these        [^abc]  any one NOT of these
[a-z]    a range

*        zero or more            +       one or more
?        zero or one             {3}     exactly three
{2,4}    two to four             {2,}    two or more

^        start of the string     $       end of the string
\b       a word boundary
|        either                  ( )     a group

The five functions

python

re.search(pattern, text)      # first match anywhere, or None
re.match(pattern, text)       # only at the START, or None
re.fullmatch(pattern, text)   # the WHOLE string, or None
re.findall(pattern, text)     # every match, as a list
re.finditer(pattern, text)    # every match, as match objects
re.sub(pattern, repl, text)   # replace
re.split(pattern, text)       # split on a pattern

match versus search catches people constantly. re.match anchors at the beginning, so it is not "does this match" — it is "does this start with". For validation you almost always want fullmatch, which requires the entire string:

python

re.match(r"\d{4}", "2026x")        # matches! there are four digits
re.fullmatch(r"\d{4}", "2026x")    # None, which is what you meant

Groups, and naming them

Parentheses capture part of a match:

python

m = re.search(r"(\d{4})-(\d{2})-(\d{2})", filename)
year, month, day = m.groups()

Positional groups become unreadable the moment there are more than two, and they renumber when someone adds a group in the middle. Name them:

python

pattern = r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})"

m = re.search(pattern, filename)
print(m["year"], m["month"])

(?P<name>...) names a group and m["name"] reads it. The pattern is longer and the code using it is self-explanatory, which is the right trade for anything that survives the day you wrote it.

Always check for None before using a match:

python

m = re.search(pattern, filename)
if m is None:
    raise InvalidFilenameError(f"no date in {filename!r}")

AttributeError: 'NoneType' object has no attribute 'group' is the single most common regex error, and it means the pattern did not match.

Compile what you reuse

python

DATE_PATTERN = re.compile(
    r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})"
)

for filename in filenames:
    m = DATE_PATTERN.search(filename)

Compiling once at module level is faster in a loop, and more importantly it gives the pattern a name and a place to live. A compiled pattern with a name near the top of a module is findable; the same pattern inline in a loop body is a thing people scroll past.

For anything non-trivial, re.VERBOSE lets you write it with whitespace and comments:

That is the same pattern as a forty-character line of punctuation, and it can be reviewed.

Greedy by default

Quantifiers take as much as they can:

The ? after a quantifier makes it lazy. Three ways to write the same intention, and they are not equally good:

(.*)

Takes everything it can.

. matches the closing bracket too, so the match runs to the last </caption> in the string and swallows both captions.

(.*?)

Takes as little as it can.

Correct here, but the pattern still says "any characters" — it works because of where the engine stops, not because of what you described.

([^<]*)

Takes what you meant.

"Characters that are not an opening bracket" cannot cross a tag boundary at all, so it is right for a reason a reader can see.

Same text, three patterns, two of which happen to work on this input

The pattern that hangs your server

Bad — nested quantifiers over overlapping character sets.

Good — no nesting, and a check that is honest about what it verifies.

The first pattern has a + inside a group that is itself repeated with +. On input that nearly matches, the engine tries every way of dividing the characters between the two quantifiers, and the number of ways grows exponentially with length — so thirty characters can take minutes and forty can take longer than the heat death of anything you care about. That is catastrophic backtracking, and when the input comes from a user it is a denial-of-service vulnerability with no exception and no log line: the process just stops responding.

The tell is a quantifier applied to a group containing another quantifier, where both can match the same characters. If you see (a+)+, (a*)* or (a|a)*, rewrite it.

When not to use one

The most valuable regex skill is recognising that the job is not a regex job.

Fixed text
"ERROR" in line is faster and clearer than a pattern. str.startswith, endswith, replace, split and partition cover an enormous share of what people reach for regex to do.
Structured formats
HTML, XML, JSON, CSV and URLs have parsers. A regex over HTML works on your examples and fails on an attribute containing a bracket, a comment, or a tag split across lines. Use json, csv, urllib.parse, or a real HTML parser.
Paths and filenames
pathlib answers suffix, stem, parent and match directly.
Anything that counts or nests
Regular expressions cannot match balanced brackets. If your pattern is growing to handle nesting, you need a parser.

Four jobs that look like regex jobs and are not

A useful test: if you cannot read the pattern back in a month, it belongs in code. Three lines of str methods with a comment beat one line of punctuation nobody dares modify.

Substitution and flags

re.sub replaces, and the replacement can reference groups or be a function:

The function form is the escape hatch for anything the pattern cannot express on its own.

Flags worth knowing:

A cheat sheet to keep

Where to go next

You can write patterns that are readable in six months, capture what you need by name, and avoid the construction that turns a validation check into an outage. The judgement to keep is the last section: most of the time the answer is str methods or a real parser, and reaching for those is not a failure to use the clever tool.

Next is Files, Paths, and the Filesystem, which goes past the basics from the foundations course into the parts that bite in production — atomic writes, permissions, temporary files and the reasons a path that works on your machine fails on someone else's.

Before you move on, take a regex you have written and rewrite it with re.VERBOSE, named groups and comments. Then look at whether it should be a regex at all. In a fair proportion of cases the honest answer is split and strip, and finding that out is the point of the exercise.