advanced35 minby DevFox

Asyncio from the Event Loop Up

Coroutines, tasks and the loop that drives them, what await actually suspends, and the single blocking call that stalls an entire async application.

Python
Async

The async service handles four thousand connections beautifully in testing. In production it stalls for two seconds at a time, every few seconds, and every single connection stalls together. The cause is one line added last week: a call to a library that reads a configuration file, synchronously, inside a coroutine.

Asyncio gives you enormous concurrency for waiting work and asks one thing in return — that you never block. By the end of this lesson you will know what a coroutine actually is, what await suspends, how the loop schedules everything, and why that one line stopped four thousand connections rather than one.

One thread, many waits

Threads let the operating system interrupt you anywhere. Asyncio runs one thread and switches only at points you can see:

python

import asyncio

async def fetch(client, url):
    response = await client.get(url)      # a switch can happen here
    return response.json()                # and nowhere in this line

async def main():
    async with httpx.AsyncClient() as client:
        results = await asyncio.gather(*(fetch(client, u) for u in urls))

async def makes a coroutine function. Calling it returns a coroutine object and runs nothing:

python

coro = fetch(client, url)      # nothing has happened
result = await coro            # now it runs

That is the same laziness as a generator, and it is the same machinery underneath — a coroutine is a resumable function that suspends at await and keeps its state.

await means: start this, and if it cannot finish immediately, suspend me and let the loop run something else until it can.

The event loop is the scheduler. It keeps a queue of ready tasks, runs one until it suspends, then runs the next. There is no preemption — a coroutine keeps the thread until it awaits something.

Concurrency comes from tasks

Awaiting one thing after another is sequential, whatever the keywords look like:

python

async def main():
    a = await fetch(client, url_a)     # waits for a...
    b = await fetch(client, url_b)     # ...then starts b

Two seconds of waiting takes two seconds. To overlap, the work must be scheduled as tasks:

python

async def main():
    async with asyncio.TaskGroup() as group:
        task_a = group.create_task(fetch(client, url_a))
        task_b = group.create_task(fetch(client, url_b))

    return task_a.result(), task_b.result()

create_task hands the coroutine to the loop, which starts it at the next opportunity. Now both waits overlap and the pair takes as long as the slower one.

gather is the older, more compact form:

python

results = await asyncio.gather(*(fetch(client, u) for u in urls))

Both are correct. TaskGroup has better failure behaviour, which the next lesson covers; gather is what most existing code uses.

The rule that catches everyone: a bare await in a loop is sequential.

python

for url in urls:
    results.append(await fetch(client, url))     # one at a time

That is four hundred requests, in order, and it is exactly the program asyncio was supposed to replace.

The blocking call

Bad — a synchronous call inside a coroutine.

python

async def handle(request):
    config = json.loads(Path("config.json").read_text())   # blocks
    rows = database.query(request.id)                      # blocks
    return render(rows, config)

Good — awaiting async equivalents, and offloading what has none.

python

async def handle(request):
    config = await load_config()                  # an async read
    rows = await database.fetch(request.id)       # an async driver
    return render(rows, config)

python

    text = await asyncio.to_thread(path.read_text)   # no async version

The first version does not error and works correctly for one request. But there is only one thread: while it waits for the disk and the database, nothing else in the process runs — not the other four thousand connections, not the health check, not the timeout that was supposed to fire. Every client experiences every other client's slowest synchronous call.

That is why the stall in the opening affected everything at once.

A threaded server

One slow request.

The thread handling it blocks. Every other thread carries on, because the operating system schedules them independently.

An async server

A 200ms pause for the whole process.

There is one thread. While it waits, nothing else runs — not the other connections, not the health check, not the timeout that was supposed to fire.

One request makes a 200ms synchronous database call. What does that cost the other four thousand?

asyncio.to_thread is the escape hatch: it runs the blocking call in a worker thread and awaits the result, so the loop stays free. Use it for libraries with no async version, and for the CPU-bound work that would otherwise hold the loop.

Debugging a stalled loop

Because a blocked loop looks like a slow program rather than an error, asyncio ships with a detector:

python

asyncio.run(main(), debug=True)

text

Executing <Task ...> took 2.041 seconds

That names the coroutine that held the thread. Turning debug mode on the first time an async service behaves oddly is usually a two-minute diagnosis, and it also warns about coroutines that were never awaited:

python

async def save(photo): ...

save(photo)          # RuntimeWarning: coroutine was never awaited

Calling a coroutine function and forgetting await does nothing at all — no error, no work done. The warning is the only symptom, which is a good reason to keep it visible.

Entering and leaving async

python

asyncio.run(main())        # creates a loop, runs, closes it

One asyncio.run at the top of your program. Nested calls raise, and creating loops by hand invites having two.

The boundary is one-directional in an important way: async code can call synchronous code freely — that is just a function call — but synchronous code cannot await anything. This is the "function colouring" problem, and it means introducing asyncio partway into an existing codebase tends to propagate outward until it reaches the entry point.

To call async from sync at a boundary you do not control:

python

result = asyncio.run(fetch_everything())      # only from sync code

And to run a blocking function from async, to_thread again. Being deliberate about where the boundary sits — usually the outermost layer — saves a great deal of retrofitting.

The async protocols

The dunder methods from the data-model lesson have async counterparts:

python

async with lock:                       # __aenter__ / __aexit__
    ...

async for chunk in response.aiter_bytes():     # __aiter__ / __anext__
    ...

python

async def read_pages(client, url):
    while url:
        response = await client.get(url)
        payload = response.json()
        for item in payload["items"]:
            yield item                  # an async generator
        url = payload.get("next")

python

async for photo in read_pages(client, "/photos"):
    await handle(photo)

That is the lazy pagination generator from the practice course, now awaiting between pages. Nothing about the shape changed — async variants of the protocols exist so the same patterns work when the steps involve waiting.

Asyncio's own synchronisation primitives mirror the threading ones and are not interchangeable with them:

python

asyncio.Lock()  asyncio.Semaphore(10)  asyncio.Queue()  asyncio.Event()

Using a threading.Lock in async code blocks the loop while it waits. A Semaphore is the common one — it bounds how many requests you have in flight, which matters because "start four thousand at once" is rarely what a remote service wants.

A cheat sheet to keep

text

THE MODEL
  one thread; switches happen ONLY at await
  no preemption - a coroutine holds the thread until it awaits
  async def -> a coroutine FUNCTION
  calling it -> a coroutine OBJECT, which runs nothing

CONCURRENCY COMES FROM TASKS
  await a; await b                    sequential
  async with asyncio.TaskGroup() as g:
      g.create_task(...)              concurrent
  asyncio.gather(*coros)              the older form

  a bare await in a for loop is SEQUENTIAL

NEVER BLOCK
  requests, open(), time.sleep(), a sync DB driver, a big parse
  one blocking call stops EVERY connection in the process
  use the async library, or:
      await asyncio.to_thread(blocking_call, arg)
      a ProcessPoolExecutor for pure-Python CPU work

DEBUGGING
  asyncio.run(main(), debug=True)
    -> "Executing <Task> took 2.041 seconds"
    -> warns about coroutines never awaited
  a forgotten await does NOTHING, with no error

BOUNDARIES
  one asyncio.run() at the top; never nest
  async can call sync freely; sync cannot await
  colouring spreads outward - decide where the boundary is

PROTOCOLS
  async with   __aenter__ / __aexit__
  async for    __aiter__ / __anext__
  async def + yield  -> an async generator

  asyncio.Lock / Semaphore / Queue / Event
  NOT threading's - those block the loop
  Semaphore to bound requests in flight

Where to go next

You can now write concurrent code where every switch is visible in the source, and you know the failure that has no equivalent in threaded code: one synchronous call stalls the entire process. The habit that prevents it is narrow — when adding a library to async code, check whether the call you are making awaits.

Next is Structured Concurrency and Cancellation, which addresses what this lesson left open. Tasks that outlive their creator, exceptions that vanish because nobody collected a result, and timeouts that need to stop work already in flight — TaskGroup and CancelledError are how those are handled, and cancellation has rules that are easy to get subtly wrong.

Before you move on, write two versions of a script that fetches twenty URLs: one awaiting in a loop, one with a TaskGroup. Time both. Then add a time.sleep(1) inside one coroutine and watch it delay every other task, not just its own.