Skip to content
Tyrone Software
Python Coding

TaskGroup versus gather: what happens when one task fails

Abstract illustration for the article TaskGroup versus gather: what happens when one task fails

Two tasks start together and one of them raises. What happens to the other one depends on how you started them, and the difference causes leaks that are hard to trace.

What we measured

The test starts a task that fails after 10 ms and a task that would sleep for a second. With asyncio.TaskGroup on Python 3.14:

  • the failing task's exception is raised as an ExceptionGroup when the block exits,
  • the slow task received a CancelledError and cleaned up.

With asyncio.gather(failing(), slow_task) and the default settings:

  • gather raised the ValueError at once,
  • the slow task was still running afterwards and nobody was waiting for its result.
Comparison table of gather and TaskGroup behaviour when a task fails
Behaviour confirmed by running both variants on Python 3.14.

Why this matters in real code

A task that keeps running after its caller has moved on holds connections, file handles and memory, and may write results nowhere. In a request handler, that is a background task per failed request. gather(..., return_exceptions=True) avoids the leftover task, but then every exception comes back as a value in the results list and you must inspect each of them, so errors are easy to swallow.

Using TaskGroup

async def load_all(ids):
    results = {}
    async with asyncio.TaskGroup() as tg:
        for i in ids:
            tg.create_task(fetch_one(i, results))
    return results

Leaving the async with block waits for all tasks. If any of them failed, the remaining ones are cancelled and the errors arrive together, which you handle with except*:

try:
    await load_all(ids)
except* TimeoutError:
    ...   # handle timeouts
except* ValueError as group:
    ...   # group.exceptions lists every ValueError

Limits to keep in mind

Your tasks must handle cancellation properly. A task that catches CancelledError and continues, or that runs a long blocking call, will stall the group. Also, TaskGroup does not give you partial results the way return_exceptions=True does: if you want the successes even when some tasks fail, catch exceptions inside each task and return a result object.

gather still has a place for tiny fan-outs where you actually want independent tasks to continue. But when the tasks belong to one operation, TaskGroup states that intent and enforces it.

gather forwards cancellation too, just from the other direction

The comparison above covers what happens when a child task fails. There is a second, less obvious case: what happens when the code awaiting gather() itself gets cancelled, for example by an outer timeout. We tested it: cancelling the object returned by gather() propagates the cancellation to every task it was gathering, and each of them received CancelledError individually.

gathered = asyncio.gather(task1, task2)
gathered.cancel()          # cancels task1 and task2 as well

So gather is not entirely hands-off about sibling tasks; it just does not do it automatically when a sibling fails on its own, only when the gather itself is cancelled from outside. TaskGroup behaves consistently in both directions, which is part of why it produces fewer surprises: a failure inside the group and a cancellation from outside both result in every task being cancelled and accounted for.

Keep reading

Technologies

  • Python
  • PHP
  • Laravel
  • Django
  • PostgreSQL
  • MySQL
  • HTML5
  • CSS3
  • Bootstrap