Comprehensions

Three of the loops in the loops lesson had exactly the same shape: make an empty list, go through something, add some of it to the list. Four lines of scaffolding around one line of actual intent.

Python has a shorter way to write that shape, and it is everywhere in real code — you cannot read other people's Python without it. By the end of this lesson you will write comprehensions comfortably, and, just as importantly, know the point at which one has become worse than the loop it replaced.

The same loop, in one line

Here is the transforming loop from earlier:

names = []
for photo in photos:
    names.append(photo.name.lower())

And here it is as a list comprehension:

names = [photo.name.lower() for photo in photos]

It is the same three parts as the loop, in a different order.

sizes_in_mb = [photo.size / 1024 for photo in photos]
titles = [make_caption(f) for f in filenames]
squares = [n * n for n in range(10)]

Keeping only some of them

Add an if at the end to filter:

large = []
for photo in photos:
    if photo.size > 1000:
        large.append(photo)

becomes:

large = [photo for photo in photos if photo.size > 1000]

The order in the comprehension matches the order of the loop: go through, test, collect. Filtering and transforming together is the common case:

big_names = [p.name.upper() for p in photos if p.size > 1000]

Read it as: take each photo, keep the ones over 1000, and collect their names in capitals.

There is a second place an if can go, and it means something completely different.

Dictionaries and sets too

The same notation with curly braces builds a dictionary:

sizes = {photo.name: photo.size for photo in photos}

The key: value on the left is what makes it a dictionary rather than a set. Without the colon, you get a set:

unique_tags = {photo.tag for photo in photos}

A dictionary comprehension is the tidiest way to turn a list of records into a lookup table, which is a thing you will do constantly:

by_id = {p.id: p for p in photos}
print(by_id[4471].name)         # instant lookup, no scanning

Inverting a dictionary is a one-liner:

name_to_count = {"ana": 400, "bruno": 128}
count_to_name = {count: name for name, count in name_to_count.items()}

The one that is not a list

Round brackets look like they should make a tuple. They make something else — a generator, which produces values one at a time instead of building the whole collection first:

total = sum(photo.size for photo in photos)

Nothing is stored. sum asks for the next value, gets it, adds it, and asks again. For four hundred photos that saves nothing. For four million it is the difference between working and running out of memory.

Use a generator whenever the collection is being consumed immediately rather than kept:

sum(p.size for p in photos)
max(p.size for p in photos)
any(p.is_corrupt for p in photos)
all(p.is_processed for p in photos)

You have already seen this form in the loops lesson without a name for it. any(...) and all(...) stop as soon as the answer is decided, so any over a million photos where the second is corrupt examines two.

Generators are covered properly in the intermediate course. What matters now is recognising the round brackets and knowing they mean "one at a time, not all at once".

More than one for

A comprehension can contain a second for, and it works like a nested loop:

all_photos = [photo
              for photographer in photographers
              for photo in photographer.photos]

The order is the same as the nested loops it replaces — outer first, inner second. Flattening a list of lists is the case where this genuinely reads well.

Beyond that, be careful. Two fors and an if in one expression is at the edge of what anyone can read at a glance, and past it a loop is the better answer.

When a loop is better

Comprehensions are not a scoring system. They are good at one specific thing — building a collection from another collection — and reaching for them elsewhere makes code worse.

Bad — everything crammed into one expression.

results = [transform(p.name.strip().lower())
           for photographer in photographers
           for p in photographer.photos
           if p.size > 1000 and not p.is_corrupt
           and p.name.endswith((".jpg", ".png"))]

Good — a named test, and a loop where the logic deserves room.

def is_worth_processing(photo):
    if photo.is_corrupt or photo.size <= 1000:
        return False
    return photo.name.endswith((".jpg", ".png"))

results = []
for photographer in photographers:
    for photo in photographer.photos:
        if is_worth_processing(photo):
            results.append(transform(photo.name.strip().lower()))

The first version is one expression, so when it produces the wrong count there is nowhere to put a print and no line to set a breakpoint on — you take it apart to debug it, then put it back. The second names the condition, which both explains it and lets you test it on its own. Length is not the measure; the second is longer and you can answer "why was this photo skipped?" without rewriting anything.

Three practical signs a comprehension has gone too far: it does not fit on one line, it needs a comment to explain it, or it has a side effect.

That last one is worth stating plainly:

[send_email(p.owner) for p in photos]     # don't

That builds a list of Nones and throws it away. The comprehension is there to make a collection — if you do not want the collection, write a loop, which says what you mean.

A cheat sheet to keep

LIST
  [expr for item in things]
  [expr for item in things if test]
  [a if test else b for item in things]     choose, not filter

  filter -> if at the END
  choose -> if/else at the START, else required

DICTIONARY
  {k: v for item in things}
  {p.id: p for p in photos}           records -> lookup table
  {v: k for k, v in d.items()}        invert

SET
  {expr for item in things}           no colon

GENERATOR - round brackets
  (expr for item in things)           one at a time, nothing stored
  sum(p.size for p in photos)
  any(...)  all(...)                  stop as soon as decided

NESTED
  [x for outer in a for x in outer]   outer first, inner second
                                      good for flattening, little else

WRITE A LOOP INSTEAD WHEN
  it does not fit on one line
  it needs a comment
  you are doing it for a side effect
  you would have to take it apart to debug it

Where to go next

You can now write the filter-and-transform shape in the form most Python code uses, build dictionaries and sets the same way, and use a generator when the collection is consumed rather than kept. You also have three concrete tests for when to stop, which matter more than the syntax — comprehensions are easy to learn and easy to overuse, and overusing them is how readable code becomes clever code.

Next is When Things Go Wrong: Exceptions, which is about what happens when your program meets something it cannot handle. Every lesson so far has assumed the file exists and the user typed a number. That one drops the assumption.

Before you move on, take the three accumulation loops from the loops lesson and rewrite each as a comprehension. Then take the deliberately awful one from this lesson and expand it back into a loop. Going both directions is what builds the judgement about which to use, and the judgement is the part that matters.