TL;DR
For a big batch of I/O jobs, start a fixed number of workers that pull from an asyncio.Queue(maxsize=...) inside a TaskGroup. Don’t wrap every job in its own task. On 100,000 simulated network calls, gather() created 100,000 tasks and the process grew by 163 MB. The same batch through 100 queue workers kept 100 tasks alive, grew by 0.2 MB, and finished in 1.7 s instead of 3.2 s. The pool has three traps: a task_done() that an exception skips (so join() hangs), total += await ... (which loses updates), and shutdown. On Python 3.13 and later, Queue.shutdown() replaces the sentinel values for shutdown.
| Fact | Value | Source | Verified |
|---|---|---|---|
Queue.shutdown() and QueueShutDown added | Python 3.13 | What’s New in 3.13 | Nov 19, 2026 |
asyncio.Queue thread safety | Not thread-safe | asyncio queue docs | Nov 19, 2026 |
TaskGroup added | Python 3.11 | asyncio task docs | Nov 19, 2026 |
| Interpreter for every Python run below | CPython 3.12.3, 2 vCPUs | this article’s sandbox | Nov 19, 2026 |
| Go version for the channel example | go1.27.4 | Go Playground | Nov 19, 2026 |
Why a queue instead of one task per job
My TaskGroup vs gather comparison ended with a semaphore example and one open question. A semaphore caps how many jobs run at once. The number of task objects still grows to one per job. Every coroutine you pass to gather() becomes a task immediately. With 20 jobs, the cost is too small to measure. With a million URLs from a database cursor, each task still holds memory while it waits for a semaphore slot.
A worker pool reverses the setup. You start N long-lived tasks, and each one loops: take an item from the queue, process it, mark it done. The queue sits between the producer and the workers. With maxsize set, the producer blocks once the buffer is full, so the input never gets ahead of the work. That property is called backpressure, and the plain gather() approach doesn’t have it.
The asyncio queue docs include a worker example, but it creates workers with create_task(), joins the queue, and then cancels the workers by hand. That was the normal pattern before Python 3.11. This tutorial uses TaskGroup, catches per-item errors, and adds a 3.13 shutdown. Each code block below ran in the sandbox, and I copied the output under each block from that run.
100,000 jobs three ways: the numbers
Each job is await asyncio.sleep(0.001), a stand-in for a 1 ms network call. The script runs the same 100,000 jobs three ways. Each mode runs in a fresh process, so the memory numbers don’t mix:
CONCURRENCY = 100
async def job(i: int) -> int:
await asyncio.sleep(0.001) # stand-in for one network call
return i * 2
async def run_gather() -> int:
results = await asyncio.gather(*(job(i) for i in range(N)))
return sum(results)
async def run_semaphore() -> int:
sem = asyncio.Semaphore(CONCURRENCY)
async def limited(i):
async with sem:
return await job(i)
results = await asyncio.gather(*(limited(i) for i in range(N)))
return sum(results)
async def run_pool() -> int:
queue: asyncio.Queue[int | None] = asyncio.Queue(maxsize=CONCURRENCY * 2)
total = 0
async def worker():
nonlocal total
while True:
i = await queue.get()
try:
if i is None:
return
result = await job(i)
total += result
finally:
queue.task_done()
async with asyncio.TaskGroup() as tg:
for _ in range(CONCURRENCY):
tg.create_task(worker())
for i in range(N):
await queue.put(i)
for _ in range(CONCURRENCY):
await queue.put(None)
return total
The harness reads ru_maxrss before and after each run and reports the growth. The peak task count comes from a separate run with a sampler that calls asyncio.all_tasks() every 5 ms. I kept that sampler out of the timed runs on purpose. My first version left it in, and the semaphore mode took 26 s. A cProfile run showed that all_tasks() was the cause, because that call walks every live task and 100,000 tasks stayed alive for seconds. Timings at 100,000 jobs are the median of three runs. The 1,000,000-job rows ran once each, without the sampler:
| Strategy (100,000 jobs unless noted) | Peak live tasks | RSS growth | Wall time |
|---|---|---|---|
gather(), no limit | 100,000 | 163.0 MB | 3.24 s |
gather() + Semaphore(100) | 100,000 | 145.3 MB | 3.93 s |
| Queue + 100 workers | 100 | 0.2 MB | 1.68 s |
gather(), no limit, 1,000,000 jobs | 1,000,000 (by construction) | 1,627.2 MB | 36.88 s |
| Queue + 100 workers, 1,000,000 jobs | 100 | 0.2 MB | 17.12 s |
The semaphore barely helped memory, and it was the slowest of the three. It limits concurrency, but all 100,000 tasks and their coroutine frames still exist from the first second. At a million jobs, gather() used 1,627 MB, about 1.6 KB per pending task. The pool’s memory stayed flat when the job count went up ten times, and its wall time grew linearly, from 1.68 s to 17.12 s. The theoretical floor is 1.0 s per 100,000 jobs (100,000 × 1 ms ÷ 100 workers). The extra 0.68 s is about 7 µs per item for queue operations, task switches and timer overshoot combined.
Unbounded gather() was slower than the pool here, even though it ran all 100,000 sleeps at once. Creating and scheduling 100,000 tasks and timers costs CPU on a single event loop. With real HTTP calls, unbounded fan-out also runs into limits: the server or your connection pool starts to reject requests. The httpx vs aiohttp benchmark covers how each client caps its own connection count.
Build the worker pool step by step
The pool below fetches 12 fake URLs with 3 workers. One URL fails. The goal is a pool that logs the failure and keeps going.
- Create the queue with a
maxsizeof twice the worker count. - Start the workers inside
async with asyncio.TaskGroup(). - Put each item with
await queue.put(). - After the last item, put one
Nonesentinel per worker. - Leave the
async withblock.
The block doesn’t exit until every worker has returned.
import asyncio
import random
import time
T0 = time.perf_counter()
def log(msg: str) -> None:
print(f"[{time.perf_counter() - T0:5.2f}s] {msg}")
async def fetch(url: str) -> int:
"""Pretend network call: 50-150 ms, fails for one URL."""
await asyncio.sleep(random.uniform(0.05, 0.15))
if url.endswith("/7"):
raise ConnectionError(f"reset by peer: {url}")
return len(url)
async def worker(name: str, queue: asyncio.Queue, results: dict, errors: dict):
while True:
url = await queue.get()
try:
if url is None: # sentinel: no more work
log(f"{name} exiting")
return
try:
results[url] = await fetch(url)
except ConnectionError as exc:
errors[url] = repr(exc) # record, keep the worker alive
finally:
queue.task_done()
async def main(urls: list[str], n_workers: int = 3) -> None:
queue: asyncio.Queue[str | None] = asyncio.Queue(maxsize=n_workers * 2)
results: dict[str, int] = {}
errors: dict[str, str] = {}
async with asyncio.TaskGroup() as tg:
for n in range(n_workers):
tg.create_task(worker(f"w{n}", queue, results, errors))
for url in urls:
await queue.put(url) # blocks while the queue is full
log(f"producer done, {queue.qsize()} items still queued")
for _ in range(n_workers):
await queue.put(None) # one sentinel per worker
log(f"ok={len(results)} failed={len(errors)}")
for url, err in errors.items():
log(f" {url}: {err}")
random.seed(42)
asyncio.run(main([f"https://api.example.com/item/{i}" for i in range(12)]))
Output:
[ 0.12s] producer done, 6 items still queued
[ 0.33s] w0 exiting
[ 0.34s] w1 exiting
[ 0.42s] w2 exiting
[ 0.42s] ok=11 failed=1
[ 0.42s] https://api.example.com/item/7: ConnectionError('reset by peer: https://api.example.com/item/7')
The first line shows backpressure at work. The producer finished at 0.12 s, but 6 items were still in the queue, which is exactly maxsize. It couldn’t run further ahead than that.
Two choices in this code are deliberate. The inner try catches only ConnectionError, the failure I expect and want to record. Anything else (a KeyError from a bug, say) escapes the worker, and the TaskGroup stops the whole pool. That’s the behavior I want for bugs, and a later section shows it. The outer try/finally holds task_done(), so the queue’s counter stays correct on every path.
If a failed item deserves another attempt, retry inside the worker rather than putting the item back on the queue. A re-queued item can loop forever. The tenacity tutorial has an AsyncRetrying setup with backoff that fits inside fetch().
How task_done() and join() count
asyncio.Queue keeps a private counter of unfinished items. The 3.13 source shows the whole mechanism in a few lines. put_nowait() adds 1. task_done() subtracts 1, and it raises ValueError if the counter is already zero. join() waits on an internal Event that task_done() sets when the counter reaches zero.
get() doesn’t touch the counter. As a result, an item that a worker took but never marked done keeps join() waiting forever. The usual way to cause that is a task_done() call that an exception jumps over:
import asyncio
async def process(item: int) -> None:
await asyncio.sleep(0.01)
if item == 3:
raise ValueError(f"bad item {item}")
async def leaky_worker(queue: asyncio.Queue) -> None:
while True:
item = await queue.get()
try:
await process(item)
queue.task_done() # skipped when process() raises
except ValueError as exc:
print(f" logged: {exc}")
async def safe_worker(queue: asyncio.Queue) -> None:
while True:
item = await queue.get()
try:
await process(item)
except ValueError as exc:
print(f" logged: {exc}")
finally:
queue.task_done() # runs on every path
async def run(worker_fn) -> None:
print(f"{worker_fn.__name__}:")
queue: asyncio.Queue[int] = asyncio.Queue()
for i in range(6):
queue.put_nowait(i)
task = asyncio.create_task(worker_fn(queue))
try:
await asyncio.wait_for(queue.join(), timeout=1.0)
print(" join() returned")
except TimeoutError:
print(f" join() still blocked after 1s, unfinished={queue._unfinished_tasks}")
task.cancel()
async def main() -> None:
await run(leaky_worker)
await run(safe_worker)
asyncio.run(main())
leaky_worker:
logged: bad item 3
join() still blocked after 1s, unfinished=1
safe_worker:
logged: bad item 3
join() returned
The _unfinished_tasks attribute is private, and I read it here only for the demo. In real code, the symptom is a program that processes everything and then never exits, with no error. The wait_for() timeout is the only reason this demo ends at all.
My rule: call task_done() only in a finally block that starts right after get() returns.
Backpressure: what maxsize does
With maxsize=0 (the default), the queue has no limit. A fast producer then loads the whole input into memory, which brings back the gather() problem in a new form. With a limit, await queue.put() suspends the producer until a worker frees a slot. The demo below pairs a producer with one consumer that handles 10 items per second, through a queue of size 3:
import asyncio
import time
T0 = time.perf_counter()
async def producer(queue: asyncio.Queue, n: int) -> None:
for i in range(n):
t = time.perf_counter()
await queue.put(i)
waited = time.perf_counter() - t
if waited > 0.01:
print(f"[{time.perf_counter() - T0:4.2f}s] put({i}) waited "
f"{waited * 1000:.0f} ms, qsize={queue.qsize()}")
await queue.put(None)
async def consumer(queue: asyncio.Queue) -> None:
while (item := await queue.get()) is not None:
await asyncio.sleep(0.1) # slow downstream: 10 items/s
queue.task_done()
queue.task_done()
async def main() -> None:
queue: asyncio.Queue[int | None] = asyncio.Queue(maxsize=3)
async with asyncio.TaskGroup() as tg:
tg.create_task(consumer(queue))
tg.create_task(producer(queue, 8))
print(f"[{time.perf_counter() - T0:4.2f}s] done")
asyncio.run(main())
[0.10s] put(4) waited 101 ms, qsize=3
[0.20s] put(5) waited 101 ms, qsize=3
[0.30s] put(6) waited 101 ms, qsize=3
[0.40s] put(7) waited 101 ms, qsize=3
[0.81s] done
Items 0 to 3 go in without waiting: the consumer takes item 0 at once, and items 1, 2 and 3 fill the buffer. From item 4 on, every put() waits about 100 ms, which is one consumer cycle. The producer now runs at the consumer’s speed.
For maxsize, I start at 1x to 2x the worker count. A bigger buffer only helps when producing an item is slow or bursty, for example a paginated API that returns 500 rows per page. In that case, set maxsize to about one page so a worker never waits on an empty queue while the next page loads.
The lost-update trap: total += await
This one caught my own benchmark script. My first run_pool() kept a running sum with total += await job(i), and the sanity check failed: the total came out at about 1% of the correct value. Here’s the bug in isolation, with 10 workers and 1,000 items that each add 1:
import asyncio
async def job(i: int) -> int:
await asyncio.sleep(0.001)
return 1
async def pool(n_items: int, safe: bool) -> int:
queue: asyncio.Queue[int | None] = asyncio.Queue()
total = 0
async def worker():
nonlocal total
while (i := await queue.get()) is not None:
if safe:
result = await job(i)
total += result
else:
total += await job(i) # reads total, suspends, writes stale sum
queue.task_done()
queue.task_done()
for i in range(n_items):
queue.put_nowait(i)
async with asyncio.TaskGroup() as tg:
for _ in range(10):
tg.create_task(worker())
await queue.join()
for _ in range(10):
queue.put_nowait(None)
return total
async def main():
print("total += await job(i) ->", await pool(1000, safe=False))
print("result = await job(i); += ->", await pool(1000, safe=True))
asyncio.run(main())
total += await job(i) -> 100
result = await job(i); += -> 1000
Python evaluates total += await job(i) as “load total, await the job, add, store”. The load happens before the await. While one worker sleeps, the other nine store their own sums, and the first worker then overwrites them with its stale value plus 1. The 10 workers all sleep for the same 1 ms, so they read total at nearly the same moment, and 9 of every 10 updates disappeared. asyncio runs on one thread, so people assume shared state is safe, but that holds only between await points. Move the await onto its own line. Then the read-modify-write has no suspension in the middle.
When a worker raises an unexpected exception
The pool above catches ConnectionError and lets everything else escape. Inside a TaskGroup, an escaped exception stops the whole pool. This demo raises a KeyError on item 5:
import asyncio
async def worker(name: str, queue: asyncio.Queue) -> None:
try:
while True:
item = await queue.get()
try:
await asyncio.sleep(0.01)
if item == 5:
raise KeyError(f"no handler for item {item}")
finally:
queue.task_done()
except asyncio.CancelledError:
print(f" {name} cancelled")
raise
async def main() -> None:
queue: asyncio.Queue[int] = asyncio.Queue(maxsize=2)
sent = 0
try:
async with asyncio.TaskGroup() as tg:
for n in range(3):
tg.create_task(worker(f"w{n}", queue))
for i in range(100):
await queue.put(i) # the producer is cancelled here too
sent += 1
except* KeyError as eg:
print(f"TaskGroup raised: {eg.exceptions!r}")
print(f"producer got {sent} of 100 items in before the group stopped")
asyncio.run(main())
w0 cancelled
w1 cancelled
TaskGroup raised: (KeyError('no handler for item 5'),)
producer got 10 of 100 items in before the group stopped
The failing worker was w2, so the TaskGroup cancelled the other two workers. It also cancelled the producer, which was the body of the async with and was waiting in put(). The error reached the caller as an ExceptionGroup, and nothing kept running in the background. With gather() and default arguments, the other workers would have kept going with nobody watching them. The TaskGroup vs gather comparison linked at the top runs that failure case side by side.
Clean shutdown on Python 3.13+: Queue.shutdown()
Sentinels work, but they’re fragile. You need exactly one per worker. If a worker crashes, its sentinel stays in the queue. If two producers share the queue, someone has to count them. Python 3.13 added Queue.shutdown() and the QueueShutDown exception. After a shutdown, put() raises at once, and get() raises after the queue is empty. Each worker simply exits when get() raises.
The sandbox for this article runs Python 3.12, which doesn’t have the method. To get real output anyway, I downloaded the 3.13 queues.py and ran it on 3.12’s event loop. I changed only its two relative imports to from asyncio import locks, mixins. The file is pure Python, and the 3.14 version of that file is byte-for-byte the same. On the main branch, the only code change drops a redundant empty() check. The output below comes from the 3.13 Queue logic running on a 3.12 interpreter.
async def worker(name: str, queue, done: list) -> None:
while True:
try:
item = await queue.get()
except QueueShutDown:
print(f" {name}: QueueShutDown, exiting")
return
try:
await asyncio.sleep(0.05)
done.append(item)
finally:
queue.task_done()
async def graceful() -> None:
print("graceful shutdown():")
queue, done = Queue(maxsize=4), []
async with asyncio.TaskGroup() as tg:
for n in range(2):
tg.create_task(worker(f"w{n}", queue, done))
for i in range(6):
await queue.put(i)
queue.shutdown() # no sentinels needed
try:
await queue.put(99)
except QueueShutDown:
print(" put() after shutdown -> QueueShutDown")
print(f" processed {len(done)} of 6 items: {sorted(done)}")
async def immediate() -> None:
print("shutdown(immediate=True):")
queue, done = Queue(), []
for i in range(10):
queue.put_nowait(i)
async with asyncio.TaskGroup() as tg:
for n in range(2):
tg.create_task(worker(f"w{n}", queue, done))
await asyncio.sleep(0.07) # let the workers finish a few items
queue.shutdown(immediate=True)
await queue.join()
print(f" join() returned with {len(done)} processed, "
f"{10 - len(done)} dropped (in-flight ones finish first)")
print(f" final processed: {sorted(done)}")
Python 3.12.3, asyncio.Queue has shutdown: False, vendored 3.13 Queue: True
graceful shutdown():
put() after shutdown -> QueueShutDown
w0: QueueShutDown, exiting
w1: QueueShutDown, exiting
processed 6 of 6 items: [0, 1, 2, 3, 4, 5]
shutdown(immediate=True):
w0: QueueShutDown, exiting
w1: QueueShutDown, exiting
join() returned with 4 processed, 6 dropped (in-flight ones finish first)
final processed: [0, 1, 2, 3]
On Python 3.13 or later, use asyncio.Queue and asyncio.QueueShutDown directly. The vendored names in this script only stand in for them.
The default shutdown() is a graceful close: the workers drain all 6 items, then exit. shutdown(immediate=True) removes everything still in the queue and subtracts the removed items from the unfinished counter. The two items already in progress (2 and 3) still finished, because join() waited for their task_done() calls. The shutdown discarded the other 6 items without an error.
The docs warn about this case: an immediate shutdown unblocks join() “even when no work has been done on the tasks”. If you use join() to mean “everything got processed”, an immediate shutdown breaks that meaning. I’d keep the immediate form for Ctrl+C handling, where the goal is to stop the program.
Feeding the queue from a thread
asyncio.Queue isn’t thread-safe, and the docs say so in the first paragraph of the page. If items come from a blocking library that runs in its own thread, don’t call put_nowait() from that thread. Schedule the call on the event loop’s thread with loop.call_soon_threadsafe():
import asyncio
import threading
import time
def blocking_reader(loop: asyncio.AbstractEventLoop, queue: asyncio.Queue) -> None:
"""Runs in a plain thread, e.g. a blocking SDK or a serial port."""
for i in range(5):
time.sleep(0.02)
loop.call_soon_threadsafe(queue.put_nowait, f"line {i}")
loop.call_soon_threadsafe(queue.put_nowait, None)
async def main() -> None:
queue: asyncio.Queue[str | None] = asyncio.Queue()
loop = asyncio.get_running_loop()
threading.Thread(target=blocking_reader, args=(loop, queue), daemon=True).start()
while (item := await queue.get()) is not None:
print("got", item, "on thread", threading.current_thread().name)
asyncio.run(main())
got line 0 on thread MainThread
got line 1 on thread MainThread
got line 2 on thread MainThread
got line 3 on thread MainThread
got line 4 on thread MainThread
put_nowait() and an unbounded queue are the right pair here. A bounded queue would raise QueueFull in the scheduled callback, because call_soon_threadsafe() can’t await put(). If the thread must slow down when the queue is full, use asyncio.run_coroutine_threadsafe(queue.put(item), loop).result(). That call blocks the thread until a slot opens. Free-threaded Python doesn’t change any of this. The Python 3.14 free-threading post covers what the GIL removal does and doesn’t make safe.
The same pool in Go
If you also write Go, the asyncio pool maps almost one-to-one onto a buffered channel. I ran this program on the Go Playground (go1.27.4):
package main
import (
"fmt"
"sync"
"time"
)
func main() {
jobs := make(chan int, 6) // bounded buffer, like Queue(maxsize=6)
var mu sync.Mutex
var wg sync.WaitGroup
done := 0
for w := 0; w < 3; w++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
for range jobs { // loop ends when jobs is closed and drained
time.Sleep(10 * time.Millisecond)
mu.Lock()
done++
mu.Unlock()
}
fmt.Printf("worker %d exiting\n", id)
}(w)
}
for i := 0; i < 12; i++ {
jobs <- i // blocks while the buffer is full
}
close(jobs) // Go's version of Queue.shutdown()
wg.Wait()
fmt.Println("processed", done, "jobs")
}
worker 0 exiting
worker 2 exiting
worker 1 exiting
processed 12 jobs
| asyncio | Go |
|---|---|
asyncio.Queue(maxsize=6) | make(chan int, 6) |
await queue.put(x) blocks when full | jobs <- x blocks when full |
queue.shutdown() (3.13+) | close(jobs) |
except QueueShutDown: return | for range jobs ends |
async with TaskGroup() waits for workers | wg.Wait() |
Sum is safe between await points only | Needs sync.Mutex or atomics |
The last row shows the biggest difference. Goroutines run in parallel, so Go needs the mutex on every update. asyncio needs no lock, but only if no await sits between the read and the write. Go has had close() since Go 1.0 in 2012. Python got the equivalent only in 3.13 (October 2024), so most Python code still uses sentinels. I’d drop them in any codebase that already requires 3.13.
FAQ
Is asyncio.Queue thread safe?
No. The docs state that asyncio queues aren’t thread-safe. Use them only from coroutines on one event loop. To add items from another thread, call loop.call_soon_threadsafe(queue.put_nowait, item) or asyncio.run_coroutine_threadsafe(queue.put(item), loop). For queues shared between threads with no event loop, use the standard queue.Queue.
What does asyncio queue task_done do?
task_done() marks one item from get() as fully processed and subtracts 1 from the queue’s unfinished counter. When that counter reaches zero, join() returns. If you call it more times than items were put, it raises ValueError. Call it in a finally block so an exception can’t skip it.
Why does asyncio queue join() hang?
join() waits until every item that was put has a matching task_done() call. It hangs when a worker takes an item and an exception skips task_done(), as in the leaky_worker demo above. It also hangs when the workers crash or exit early while items are still queued.
How do you stop asyncio queue workers?
On Python 3.13+, call queue.shutdown() after the last item. Each worker exits when get() raises QueueShutDown. On older versions, put one sentinel (such as None) per worker, or cancel the workers after await queue.join(). Inside a TaskGroup, an unhandled exception in any worker cancels all of them.
Can an asyncio queue have multiple consumers?
Yes. Any number of coroutines can await get() on the same queue. The queue gives each item to exactly one of them, in the order the consumers started waiting. That’s the worker-pool pattern from this article. For priority order, use asyncio.PriorityQueue. For last-in-first-out order, use asyncio.LifoQueue. Both have the same API.
Sources
- asyncio Queues, Python docs:
Queue,task_done(),join(),shutdown(), the thread-safety note, and the official worker example - CPython
Lib/asyncio/queues.py, 3.13 branch: the source that the shutdown demo ran - What’s New in Python 3.13:
Queue.shutdown()andQueueShutDown(gh-104228) - asyncio Task Groups, Python docs: cancellation behavior when a child task fails
- Developing with asyncio: concurrency and multithreading:
call_soon_threadsafe()andrun_coroutine_threadsafe() - Go Tour: buffered channels and range and close: the Go side of the comparison
Bottom Line
I’d use a queue worker pool for any batch over a few thousand items, or any batch whose size comes from outside the code. The pool costs about 15 more lines than gather(). In return, you get flat memory, a producer that slows to the workers’ speed, and a fixed number of connections to whatever you’re calling. Put task_done() in a finally block. Before you touch shared state, put the await on its own line. Run the pool inside a TaskGroup, so a bug stops everything. If your minimum version is 3.13, replace the sentinel loop with a single shutdown() call.