TL;DR

For a big batch of I/O jobs, start a fixed number of workers that pull from an asyncio.Queue(maxsize=...) inside a TaskGroup. Don’t wrap every job in its own task. On 100,000 simulated network calls, gather() created 100,000 tasks and the process grew by 163 MB. The same batch through 100 queue workers kept 100 tasks alive, grew by 0.2 MB, and finished in 1.7 s instead of 3.2 s. The pool has three traps: a task_done() that an exception skips (so join() hangs), total += await ... (which loses updates), and shutdown. On Python 3.13 and later, Queue.shutdown() replaces the sentinel values for shutdown.

FactValueSourceVerified
Queue.shutdown() and QueueShutDown addedPython 3.13What’s New in 3.13Nov 19, 2026
asyncio.Queue thread safetyNot thread-safeasyncio queue docsNov 19, 2026
TaskGroup addedPython 3.11asyncio task docsNov 19, 2026
Interpreter for every Python run belowCPython 3.12.3, 2 vCPUsthis article’s sandboxNov 19, 2026
Go version for the channel examplego1.27.4Go PlaygroundNov 19, 2026

Why a queue instead of one task per job

My TaskGroup vs gather comparison ended with a semaphore example and one open question. A semaphore caps how many jobs run at once. The number of task objects still grows to one per job. Every coroutine you pass to gather() becomes a task immediately. With 20 jobs, the cost is too small to measure. With a million URLs from a database cursor, each task still holds memory while it waits for a semaphore slot.

A worker pool reverses the setup. You start N long-lived tasks, and each one loops: take an item from the queue, process it, mark it done. The queue sits between the producer and the workers. With maxsize set, the producer blocks once the buffer is full, so the input never gets ahead of the work. That property is called backpressure, and the plain gather() approach doesn’t have it.

The asyncio queue docs include a worker example, but it creates workers with create_task(), joins the queue, and then cancels the workers by hand. That was the normal pattern before Python 3.11. This tutorial uses TaskGroup, catches per-item errors, and adds a 3.13 shutdown. Each code block below ran in the sandbox, and I copied the output under each block from that run.

100,000 jobs three ways: the numbers

Each job is await asyncio.sleep(0.001), a stand-in for a 1 ms network call. The script runs the same 100,000 jobs three ways. Each mode runs in a fresh process, so the memory numbers don’t mix:

CONCURRENCY = 100

async def job(i: int) -> int:
    await asyncio.sleep(0.001)  # stand-in for one network call
    return i * 2

async def run_gather() -> int:
    results = await asyncio.gather(*(job(i) for i in range(N)))
    return sum(results)

async def run_semaphore() -> int:
    sem = asyncio.Semaphore(CONCURRENCY)

    async def limited(i):
        async with sem:
            return await job(i)
    results = await asyncio.gather(*(limited(i) for i in range(N)))
    return sum(results)

async def run_pool() -> int:
    queue: asyncio.Queue[int | None] = asyncio.Queue(maxsize=CONCURRENCY * 2)
    total = 0

    async def worker():
        nonlocal total
        while True:
            i = await queue.get()
            try:
                if i is None:
                    return
                result = await job(i)
                total += result
            finally:
                queue.task_done()

    async with asyncio.TaskGroup() as tg:
        for _ in range(CONCURRENCY):
            tg.create_task(worker())
        for i in range(N):
            await queue.put(i)
        for _ in range(CONCURRENCY):
            await queue.put(None)
    return total

The harness reads ru_maxrss before and after each run and reports the growth. The peak task count comes from a separate run with a sampler that calls asyncio.all_tasks() every 5 ms. I kept that sampler out of the timed runs on purpose. My first version left it in, and the semaphore mode took 26 s. A cProfile run showed that all_tasks() was the cause, because that call walks every live task and 100,000 tasks stayed alive for seconds. Timings at 100,000 jobs are the median of three runs. The 1,000,000-job rows ran once each, without the sampler:

Strategy (100,000 jobs unless noted)Peak live tasksRSS growthWall time
gather(), no limit100,000163.0 MB3.24 s
gather() + Semaphore(100)100,000145.3 MB3.93 s
Queue + 100 workers1000.2 MB1.68 s
gather(), no limit, 1,000,000 jobs1,000,000 (by construction)1,627.2 MB36.88 s
Queue + 100 workers, 1,000,000 jobs1000.2 MB17.12 s
~1.6 KB
RSS per pending task in the gather run
0.2 MB
RSS growth for 1M jobs through the pool
1.9x
Pool vs plain gather wall time

The semaphore barely helped memory, and it was the slowest of the three. It limits concurrency, but all 100,000 tasks and their coroutine frames still exist from the first second. At a million jobs, gather() used 1,627 MB, about 1.6 KB per pending task. The pool’s memory stayed flat when the job count went up ten times, and its wall time grew linearly, from 1.68 s to 17.12 s. The theoretical floor is 1.0 s per 100,000 jobs (100,000 × 1 ms ÷ 100 workers). The extra 0.68 s is about 7 µs per item for queue operations, task switches and timer overshoot combined.

Unbounded gather() was slower than the pool here, even though it ran all 100,000 sleeps at once. Creating and scheduling 100,000 tasks and timers costs CPU on a single event loop. With real HTTP calls, unbounded fan-out also runs into limits: the server or your connection pool starts to reject requests. The httpx vs aiohttp benchmark covers how each client caps its own connection count.

Build the worker pool step by step

The pool below fetches 12 fake URLs with 3 workers. One URL fails. The goal is a pool that logs the failure and keeps going.

  1. Create the queue with a maxsize of twice the worker count.
  2. Start the workers inside async with asyncio.TaskGroup().
  3. Put each item with await queue.put().
  4. After the last item, put one None sentinel per worker.
  5. Leave the async with block.

The block doesn’t exit until every worker has returned.

import asyncio
import random
import time

T0 = time.perf_counter()


def log(msg: str) -> None:
    print(f"[{time.perf_counter() - T0:5.2f}s] {msg}")


async def fetch(url: str) -> int:
    """Pretend network call: 50-150 ms, fails for one URL."""
    await asyncio.sleep(random.uniform(0.05, 0.15))
    if url.endswith("/7"):
        raise ConnectionError(f"reset by peer: {url}")
    return len(url)


async def worker(name: str, queue: asyncio.Queue, results: dict, errors: dict):
    while True:
        url = await queue.get()
        try:
            if url is None:          # sentinel: no more work
                log(f"{name} exiting")
                return
            try:
                results[url] = await fetch(url)
            except ConnectionError as exc:
                errors[url] = repr(exc)  # record, keep the worker alive
        finally:
            queue.task_done()


async def main(urls: list[str], n_workers: int = 3) -> None:
    queue: asyncio.Queue[str | None] = asyncio.Queue(maxsize=n_workers * 2)
    results: dict[str, int] = {}
    errors: dict[str, str] = {}

    async with asyncio.TaskGroup() as tg:
        for n in range(n_workers):
            tg.create_task(worker(f"w{n}", queue, results, errors))
        for url in urls:
            await queue.put(url)     # blocks while the queue is full
        log(f"producer done, {queue.qsize()} items still queued")
        for _ in range(n_workers):
            await queue.put(None)    # one sentinel per worker

    log(f"ok={len(results)} failed={len(errors)}")
    for url, err in errors.items():
        log(f"  {url}: {err}")


random.seed(42)
asyncio.run(main([f"https://api.example.com/item/{i}" for i in range(12)]))

Output:

[ 0.12s] producer done, 6 items still queued
[ 0.33s] w0 exiting
[ 0.34s] w1 exiting
[ 0.42s] w2 exiting
[ 0.42s] ok=11 failed=1
[ 0.42s]   https://api.example.com/item/7: ConnectionError('reset by peer: https://api.example.com/item/7')

The first line shows backpressure at work. The producer finished at 0.12 s, but 6 items were still in the queue, which is exactly maxsize. It couldn’t run further ahead than that.

Two choices in this code are deliberate. The inner try catches only ConnectionError, the failure I expect and want to record. Anything else (a KeyError from a bug, say) escapes the worker, and the TaskGroup stops the whole pool. That’s the behavior I want for bugs, and a later section shows it. The outer try/finally holds task_done(), so the queue’s counter stays correct on every path.

If a failed item deserves another attempt, retry inside the worker rather than putting the item back on the queue. A re-queued item can loop forever. The tenacity tutorial has an AsyncRetrying setup with backoff that fits inside fetch().

How task_done() and join() count

asyncio.Queue keeps a private counter of unfinished items. The 3.13 source shows the whole mechanism in a few lines. put_nowait() adds 1. task_done() subtracts 1, and it raises ValueError if the counter is already zero. join() waits on an internal Event that task_done() sets when the counter reaches zero.

get() doesn’t touch the counter. As a result, an item that a worker took but never marked done keeps join() waiting forever. The usual way to cause that is a task_done() call that an exception jumps over:

import asyncio


async def process(item: int) -> None:
    await asyncio.sleep(0.01)
    if item == 3:
        raise ValueError(f"bad item {item}")


async def leaky_worker(queue: asyncio.Queue) -> None:
    while True:
        item = await queue.get()
        try:
            await process(item)
            queue.task_done()        # skipped when process() raises
        except ValueError as exc:
            print(f"  logged: {exc}")


async def safe_worker(queue: asyncio.Queue) -> None:
    while True:
        item = await queue.get()
        try:
            await process(item)
        except ValueError as exc:
            print(f"  logged: {exc}")
        finally:
            queue.task_done()        # runs on every path


async def run(worker_fn) -> None:
    print(f"{worker_fn.__name__}:")
    queue: asyncio.Queue[int] = asyncio.Queue()
    for i in range(6):
        queue.put_nowait(i)
    task = asyncio.create_task(worker_fn(queue))
    try:
        await asyncio.wait_for(queue.join(), timeout=1.0)
        print("  join() returned")
    except TimeoutError:
        print(f"  join() still blocked after 1s, unfinished={queue._unfinished_tasks}")
    task.cancel()


async def main() -> None:
    await run(leaky_worker)
    await run(safe_worker)


asyncio.run(main())
leaky_worker:
  logged: bad item 3
  join() still blocked after 1s, unfinished=1
safe_worker:
  logged: bad item 3
  join() returned

The _unfinished_tasks attribute is private, and I read it here only for the demo. In real code, the symptom is a program that processes everything and then never exits, with no error. The wait_for() timeout is the only reason this demo ends at all.

My rule: call task_done() only in a finally block that starts right after get() returns.

Backpressure: what maxsize does

With maxsize=0 (the default), the queue has no limit. A fast producer then loads the whole input into memory, which brings back the gather() problem in a new form. With a limit, await queue.put() suspends the producer until a worker frees a slot. The demo below pairs a producer with one consumer that handles 10 items per second, through a queue of size 3:

import asyncio
import time

T0 = time.perf_counter()


async def producer(queue: asyncio.Queue, n: int) -> None:
    for i in range(n):
        t = time.perf_counter()
        await queue.put(i)
        waited = time.perf_counter() - t
        if waited > 0.01:
            print(f"[{time.perf_counter() - T0:4.2f}s] put({i}) waited "
                  f"{waited * 1000:.0f} ms, qsize={queue.qsize()}")
    await queue.put(None)


async def consumer(queue: asyncio.Queue) -> None:
    while (item := await queue.get()) is not None:
        await asyncio.sleep(0.1)  # slow downstream: 10 items/s
        queue.task_done()
    queue.task_done()


async def main() -> None:
    queue: asyncio.Queue[int | None] = asyncio.Queue(maxsize=3)
    async with asyncio.TaskGroup() as tg:
        tg.create_task(consumer(queue))
        tg.create_task(producer(queue, 8))
    print(f"[{time.perf_counter() - T0:4.2f}s] done")


asyncio.run(main())
[0.10s] put(4) waited 101 ms, qsize=3
[0.20s] put(5) waited 101 ms, qsize=3
[0.30s] put(6) waited 101 ms, qsize=3
[0.40s] put(7) waited 101 ms, qsize=3
[0.81s] done

Items 0 to 3 go in without waiting: the consumer takes item 0 at once, and items 1, 2 and 3 fill the buffer. From item 4 on, every put() waits about 100 ms, which is one consumer cycle. The producer now runs at the consumer’s speed.

For maxsize, I start at 1x to 2x the worker count. A bigger buffer only helps when producing an item is slow or bursty, for example a paginated API that returns 500 rows per page. In that case, set maxsize to about one page so a worker never waits on an empty queue while the next page loads.

The lost-update trap: total += await

This one caught my own benchmark script. My first run_pool() kept a running sum with total += await job(i), and the sanity check failed: the total came out at about 1% of the correct value. Here’s the bug in isolation, with 10 workers and 1,000 items that each add 1:

import asyncio


async def job(i: int) -> int:
    await asyncio.sleep(0.001)
    return 1


async def pool(n_items: int, safe: bool) -> int:
    queue: asyncio.Queue[int | None] = asyncio.Queue()
    total = 0

    async def worker():
        nonlocal total
        while (i := await queue.get()) is not None:
            if safe:
                result = await job(i)
                total += result
            else:
                total += await job(i)   # reads total, suspends, writes stale sum
            queue.task_done()
        queue.task_done()

    for i in range(n_items):
        queue.put_nowait(i)
    async with asyncio.TaskGroup() as tg:
        for _ in range(10):
            tg.create_task(worker())
        await queue.join()
        for _ in range(10):
            queue.put_nowait(None)
    return total


async def main():
    print("total += await job(i)      ->", await pool(1000, safe=False))
    print("result = await job(i); +=  ->", await pool(1000, safe=True))


asyncio.run(main())
total += await job(i)      -> 100
result = await job(i); +=  -> 1000

Python evaluates total += await job(i) as “load total, await the job, add, store”. The load happens before the await. While one worker sleeps, the other nine store their own sums, and the first worker then overwrites them with its stale value plus 1. The 10 workers all sleep for the same 1 ms, so they read total at nearly the same moment, and 9 of every 10 updates disappeared. asyncio runs on one thread, so people assume shared state is safe, but that holds only between await points. Move the await onto its own line. Then the read-modify-write has no suspension in the middle.

When a worker raises an unexpected exception

The pool above catches ConnectionError and lets everything else escape. Inside a TaskGroup, an escaped exception stops the whole pool. This demo raises a KeyError on item 5:

import asyncio


async def worker(name: str, queue: asyncio.Queue) -> None:
    try:
        while True:
            item = await queue.get()
            try:
                await asyncio.sleep(0.01)
                if item == 5:
                    raise KeyError(f"no handler for item {item}")
            finally:
                queue.task_done()
    except asyncio.CancelledError:
        print(f"  {name} cancelled")
        raise


async def main() -> None:
    queue: asyncio.Queue[int] = asyncio.Queue(maxsize=2)
    sent = 0
    try:
        async with asyncio.TaskGroup() as tg:
            for n in range(3):
                tg.create_task(worker(f"w{n}", queue))
            for i in range(100):
                await queue.put(i)      # the producer is cancelled here too
                sent += 1
    except* KeyError as eg:
        print(f"TaskGroup raised: {eg.exceptions!r}")
    print(f"producer got {sent} of 100 items in before the group stopped")


asyncio.run(main())
  w0 cancelled
  w1 cancelled
TaskGroup raised: (KeyError('no handler for item 5'),)
producer got 10 of 100 items in before the group stopped

The failing worker was w2, so the TaskGroup cancelled the other two workers. It also cancelled the producer, which was the body of the async with and was waiting in put(). The error reached the caller as an ExceptionGroup, and nothing kept running in the background. With gather() and default arguments, the other workers would have kept going with nobody watching them. The TaskGroup vs gather comparison linked at the top runs that failure case side by side.

Clean shutdown on Python 3.13+: Queue.shutdown()

Sentinels work, but they’re fragile. You need exactly one per worker. If a worker crashes, its sentinel stays in the queue. If two producers share the queue, someone has to count them. Python 3.13 added Queue.shutdown() and the QueueShutDown exception. After a shutdown, put() raises at once, and get() raises after the queue is empty. Each worker simply exits when get() raises.

The sandbox for this article runs Python 3.12, which doesn’t have the method. To get real output anyway, I downloaded the 3.13 queues.py and ran it on 3.12’s event loop. I changed only its two relative imports to from asyncio import locks, mixins. The file is pure Python, and the 3.14 version of that file is byte-for-byte the same. On the main branch, the only code change drops a redundant empty() check. The output below comes from the 3.13 Queue logic running on a 3.12 interpreter.

async def worker(name: str, queue, done: list) -> None:
    while True:
        try:
            item = await queue.get()
        except QueueShutDown:
            print(f"  {name}: QueueShutDown, exiting")
            return
        try:
            await asyncio.sleep(0.05)
            done.append(item)
        finally:
            queue.task_done()


async def graceful() -> None:
    print("graceful shutdown():")
    queue, done = Queue(maxsize=4), []
    async with asyncio.TaskGroup() as tg:
        for n in range(2):
            tg.create_task(worker(f"w{n}", queue, done))
        for i in range(6):
            await queue.put(i)
        queue.shutdown()             # no sentinels needed
        try:
            await queue.put(99)
        except QueueShutDown:
            print("  put() after shutdown -> QueueShutDown")
    print(f"  processed {len(done)} of 6 items: {sorted(done)}")


async def immediate() -> None:
    print("shutdown(immediate=True):")
    queue, done = Queue(), []
    for i in range(10):
        queue.put_nowait(i)
    async with asyncio.TaskGroup() as tg:
        for n in range(2):
            tg.create_task(worker(f"w{n}", queue, done))
        await asyncio.sleep(0.07)    # let the workers finish a few items
        queue.shutdown(immediate=True)
        await queue.join()
        print(f"  join() returned with {len(done)} processed, "
              f"{10 - len(done)} dropped (in-flight ones finish first)")
    print(f"  final processed: {sorted(done)}")
Python 3.12.3, asyncio.Queue has shutdown: False, vendored 3.13 Queue: True
graceful shutdown():
  put() after shutdown -> QueueShutDown
  w0: QueueShutDown, exiting
  w1: QueueShutDown, exiting
  processed 6 of 6 items: [0, 1, 2, 3, 4, 5]
shutdown(immediate=True):
  w0: QueueShutDown, exiting
  w1: QueueShutDown, exiting
  join() returned with 4 processed, 6 dropped (in-flight ones finish first)
  final processed: [0, 1, 2, 3]

On Python 3.13 or later, use asyncio.Queue and asyncio.QueueShutDown directly. The vendored names in this script only stand in for them.

The default shutdown() is a graceful close: the workers drain all 6 items, then exit. shutdown(immediate=True) removes everything still in the queue and subtracts the removed items from the unfinished counter. The two items already in progress (2 and 3) still finished, because join() waited for their task_done() calls. The shutdown discarded the other 6 items without an error.

The docs warn about this case: an immediate shutdown unblocks join() “even when no work has been done on the tasks”. If you use join() to mean “everything got processed”, an immediate shutdown breaks that meaning. I’d keep the immediate form for Ctrl+C handling, where the goal is to stop the program.

Feeding the queue from a thread

asyncio.Queue isn’t thread-safe, and the docs say so in the first paragraph of the page. If items come from a blocking library that runs in its own thread, don’t call put_nowait() from that thread. Schedule the call on the event loop’s thread with loop.call_soon_threadsafe():

import asyncio
import threading
import time


def blocking_reader(loop: asyncio.AbstractEventLoop, queue: asyncio.Queue) -> None:
    """Runs in a plain thread, e.g. a blocking SDK or a serial port."""
    for i in range(5):
        time.sleep(0.02)
        loop.call_soon_threadsafe(queue.put_nowait, f"line {i}")
    loop.call_soon_threadsafe(queue.put_nowait, None)


async def main() -> None:
    queue: asyncio.Queue[str | None] = asyncio.Queue()
    loop = asyncio.get_running_loop()
    threading.Thread(target=blocking_reader, args=(loop, queue), daemon=True).start()
    while (item := await queue.get()) is not None:
        print("got", item, "on thread", threading.current_thread().name)


asyncio.run(main())
got line 0 on thread MainThread
got line 1 on thread MainThread
got line 2 on thread MainThread
got line 3 on thread MainThread
got line 4 on thread MainThread

put_nowait() and an unbounded queue are the right pair here. A bounded queue would raise QueueFull in the scheduled callback, because call_soon_threadsafe() can’t await put(). If the thread must slow down when the queue is full, use asyncio.run_coroutine_threadsafe(queue.put(item), loop).result(). That call blocks the thread until a slot opens. Free-threaded Python doesn’t change any of this. The Python 3.14 free-threading post covers what the GIL removal does and doesn’t make safe.

The same pool in Go

If you also write Go, the asyncio pool maps almost one-to-one onto a buffered channel. I ran this program on the Go Playground (go1.27.4):

package main

import (
	"fmt"
	"sync"
	"time"
)

func main() {
	jobs := make(chan int, 6) // bounded buffer, like Queue(maxsize=6)
	var mu sync.Mutex
	var wg sync.WaitGroup
	done := 0

	for w := 0; w < 3; w++ {
		wg.Add(1)
		go func(id int) {
			defer wg.Done()
			for range jobs { // loop ends when jobs is closed and drained
				time.Sleep(10 * time.Millisecond)
				mu.Lock()
				done++
				mu.Unlock()
			}
			fmt.Printf("worker %d exiting\n", id)
		}(w)
	}

	for i := 0; i < 12; i++ {
		jobs <- i // blocks while the buffer is full
	}
	close(jobs) // Go's version of Queue.shutdown()
	wg.Wait()
	fmt.Println("processed", done, "jobs")
}
worker 0 exiting
worker 2 exiting
worker 1 exiting
processed 12 jobs
asyncioGo
asyncio.Queue(maxsize=6)make(chan int, 6)
await queue.put(x) blocks when fulljobs <- x blocks when full
queue.shutdown() (3.13+)close(jobs)
except QueueShutDown: returnfor range jobs ends
async with TaskGroup() waits for workerswg.Wait()
Sum is safe between await points onlyNeeds sync.Mutex or atomics

The last row shows the biggest difference. Goroutines run in parallel, so Go needs the mutex on every update. asyncio needs no lock, but only if no await sits between the read and the write. Go has had close() since Go 1.0 in 2012. Python got the equivalent only in 3.13 (October 2024), so most Python code still uses sentinels. I’d drop them in any codebase that already requires 3.13.

FAQ

Is asyncio.Queue thread safe?

No. The docs state that asyncio queues aren’t thread-safe. Use them only from coroutines on one event loop. To add items from another thread, call loop.call_soon_threadsafe(queue.put_nowait, item) or asyncio.run_coroutine_threadsafe(queue.put(item), loop). For queues shared between threads with no event loop, use the standard queue.Queue.

What does asyncio queue task_done do?

task_done() marks one item from get() as fully processed and subtracts 1 from the queue’s unfinished counter. When that counter reaches zero, join() returns. If you call it more times than items were put, it raises ValueError. Call it in a finally block so an exception can’t skip it.

Why does asyncio queue join() hang?

join() waits until every item that was put has a matching task_done() call. It hangs when a worker takes an item and an exception skips task_done(), as in the leaky_worker demo above. It also hangs when the workers crash or exit early while items are still queued.

How do you stop asyncio queue workers?

On Python 3.13+, call queue.shutdown() after the last item. Each worker exits when get() raises QueueShutDown. On older versions, put one sentinel (such as None) per worker, or cancel the workers after await queue.join(). Inside a TaskGroup, an unhandled exception in any worker cancels all of them.

Can an asyncio queue have multiple consumers?

Yes. Any number of coroutines can await get() on the same queue. The queue gives each item to exactly one of them, in the order the consumers started waiting. That’s the worker-pool pattern from this article. For priority order, use asyncio.PriorityQueue. For last-in-first-out order, use asyncio.LifoQueue. Both have the same API.

Sources

Bottom Line

I’d use a queue worker pool for any batch over a few thousand items, or any batch whose size comes from outside the code. The pool costs about 15 more lines than gather(). In return, you get flat memory, a producer that slows to the workers’ speed, and a fixed number of connections to whatever you’re calling. Put task_done() in a finally block. Before you touch shared state, put the await on its own line. Run the pool inside a TaskGroup, so a bug stops everything. If your minimum version is 3.13, replace the sentinel loop with a single shutdown() call.