When one request depends on several services

An order status page may need inventory, payment and delivery information. Sequential requests accumulate waiting time, while unlimited concurrency can exhaust connections and overload dependencies. Before optimizing latency, establish which request owns each operation, when it must stop and how failure reaches the caller. TaskGroup provides a useful scope for related asynchronous work.

Leaving a TaskGroup waits for its members. A non-cancellation failure cancels the remaining tasks, and failures may be reported as an ExceptionGroup. That behavior suits operations where partial success is unacceptable. A dashboard with independent panels needs a different contract: expected dependency failures can become explicit unavailable results inside each task. Catching every exception and returning an empty value would conceal programming errors.

Give concurrent work a scope and deadline

Write down the end-to-end deadline before choosing retries. A two-second interaction cannot accommodate several independent two-second retries. Connection establishment, response reading, retry delays and cleanup all consume time. In the example, asyncio.timeout surrounds the group and a Semaphore limits entry to the simulated dependency. Artificial inputs make the exercise reproducible without network access or credentials.

Cancellation is a request to stop, not a guarantee that an external side effect never happened. Release resources in finally blocks and propagate CancelledError after cleanup. Avoid blocking calls and CPU-heavy work directly on the event loop. A thread or process may be appropriate, but choose after profiling the real operation. The semaphore bounds active work in its scope; creating millions of waiting tasks would still consume memory.

Evaluate three cases separately: success, a slow dependency and a failing dependency. Record elapsed time, open connections and unfinished work. Add a controlled delay in a test environment to verify that the deadline actually governs the call. Keep request identifiers in diagnostics rather than logging private dependency payloads. Decide whether unavailable data should block the whole page or affect only one panel.

Code example and verification

The notes state the prerequisites and expected outcome. Run the example in a test environment, then adapt it to your project’s data and constraints.

1. Python 3.11+; standard library only

import asyncio

async def fetch(name, gate):
    async with gate:
        await asyncio.sleep(0.05)
        return name.upper()

async def main():
    gate = asyncio.Semaphore(2)
    async with asyncio.timeout(1):
        async with asyncio.TaskGroup() as group:
            jobs = [group.create_task(fetch(name, gate))
                    for name in ("stock", "payment", "delivery")]
    print([job.result() for job in jobs])

asyncio.run(main())

Run with python; expect STOCK, PAYMENT and DELIVERY in the printed list. Change the sleep to 2 to observe TimeoutError. The example deliberately propagates deadline failure; translate it at the application request boundary.

2. Failure propagation and group cleanup

Run this standalone file as group_failure.py. A dependency failure must reach the caller while the slower task gets a cleanup opportunity. The finally block demonstrates an observable lifecycle requirement.

import asyncio

async def slow():
    try:
        await asyncio.sleep(30)
    finally:
        print("slow task cleaned up")

async def broken():
    await asyncio.sleep(0.01)
    raise ValueError("dependency failed")

async def main():
    try:
        async with asyncio.TaskGroup() as group:
            group.create_task(slow())
            group.create_task(broken())
    except* ValueError as errors:
        print("handled errors:", len(errors.exceptions))

asyncio.run(main())

Expect cleanup and handled errors: 1. except* handles ValueError while leaving unrelated errors visible. Replace the cleanup message with real connection or file cleanup in an application.

3. Translate a timeout at the request boundary

Here the request boundary owns its deadline. It returns an explicit unavailable result rather than an empty value that an interface could mistake for genuine data.

import asyncio

async def load_summary():
    await asyncio.sleep(0.2)
    return {"status": "ready"}

async def handle_request():
    try:
        async with asyncio.timeout(0.05):
            return await load_summary()
    except TimeoutError:
        return {"status": "unavailable", "reason": "deadline"}

result = asyncio.run(handle_request())
assert result == {"status": "unavailable", "reason": "deadline"}
print(result)

Expect reason: deadline. Change the loading delay to 0.01 to exercise success and update the assertion for that scenario. Do not insert unbounded retries inside the deadline.

4. Bound input with a queue and a fixed worker pool

A semaphore bounds active work, not the number of waiting task objects. This standalone pool uses two workers and a four-item queue so producers must wait when the queue is full.

import asyncio

async def worker(queue, results):
    while True:
        item = await queue.get()
        try:
            if item is None:
                return
            await asyncio.sleep(0.001)
            results.append(item * 2)
        finally:
            queue.task_done()

async def main():
    queue = asyncio.Queue(maxsize=4)
    results = []
    async with asyncio.TaskGroup() as group:
        for _ in range(2):
            group.create_task(worker(queue, results))
        for item in range(10):
            await queue.put(item)
        for _ in range(2):
            await queue.put(None)
        await queue.join()
    assert sorted(results) == list(range(0, 20, 2))
    print("processed:", len(results))

asyncio.run(main())

Expect processed: 10. Completion order is not the result contract. For unbounded input, stream results instead of accumulating this list. TaskGroup propagates worker failures; side-effecting work needs its own retry and outcome policy.

Turn a demonstration into an operational contract

Start with a read-only path and a small set of dependencies. Define acceptable partial results and latency before tuning concurrency. Compare limits under representative load and review the destination’s capacity. For students, the useful lesson extends beyond async syntax: every concurrent operation needs ownership, cleanup and a measurable completion condition.

Implementation checklist

  • Verify Python 3.11 or newer.
  • Set an overall deadline separately from concurrency.
  • Exercise cancellation and failure paths.
  • Check for blocking calls and leaked resources.

References

Editorial guidance: Liyan Knowledge.