Skip to content

Fix docs for Queue.shutdown #137028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jul 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 24 additions & 11 deletions Doc/library/asyncio-queue.rst
Original file line number Diff line number Diff line change
Expand Up @@ -102,17 +102,33 @@ Queue

.. method:: shutdown(immediate=False)

Shut down the queue, making :meth:`~Queue.get` and :meth:`~Queue.put`
Put a :class:`Queue` instance into a shutdown mode.

The queue can no longer grow.
Future calls to :meth:`~Queue.put` raise :exc:`QueueShutDown`.
Currently blocked callers of :meth:`~Queue.put` will be unblocked
and will raise :exc:`QueueShutDown` in the formerly blocked thread.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
and will raise :exc:`QueueShutDown` in the formerly blocked thread.
and will raise :exc:`QueueShutDown` in the formerly blocked tasks.

asyncio queues are not thread safe, they can only be used with tasks.


If *immediate* is false (the default), the queue can be wound
down normally with :meth:`~Queue.get` calls to extract tasks
that have already been loaded.

And if :meth:`~Queue.task_done` is called for each remaining task, a
pending :meth:`~Queue.join` will be unblocked normally.

Once the queue is empty, future calls to :meth:`~Queue.get` will
raise :exc:`QueueShutDown`.

By default, :meth:`~Queue.get` on a shut down queue will only
raise once the queue is empty. Set *immediate* to true to make
:meth:`~Queue.get` raise immediately instead.
If *immediate* is true, the queue is terminated immediately.
The queue is drained to be completely empty. All callers of
:meth:`~Queue.join` are unblocked regardless of the number
of unfinished tasks. Blocked callers of :meth:`~Queue.get`
are unblocked and will raise :exc:`QueueShutDown` because the
queue is empty.

All blocked callers of :meth:`~Queue.put` and :meth:`~Queue.get`
will be unblocked. If *immediate* is true, a task will be marked
as done for each remaining item in the queue, which may unblock
callers of :meth:`~Queue.join`.
Use caution when using :meth:`~Queue.join` with *immediate* set
to true. This unblocks the join even when no work has been done
on the tasks, violating the usual invariant for joining a queue.

.. versionadded:: 3.13

Expand All @@ -129,9 +145,6 @@ Queue
call was received for every item that had been :meth:`~Queue.put`
into the queue).

``shutdown(immediate=True)`` calls :meth:`task_done` for each
remaining item in the queue.

Raises :exc:`ValueError` if called more times than there were
items placed in the queue.

Expand Down
44 changes: 30 additions & 14 deletions Doc/library/queue.rst
Original file line number Diff line number Diff line change
Expand Up @@ -187,9 +187,6 @@ fully processed by daemon consumer threads.
processed (meaning that a :meth:`task_done` call was received for every item
that had been :meth:`put` into the queue).

``shutdown(immediate=True)`` calls :meth:`task_done` for each remaining item
in the queue.

Raises a :exc:`ValueError` if called more times than there were items placed in
the queue.

Expand All @@ -204,6 +201,9 @@ fully processed by daemon consumer threads.
count of unfinished tasks drops to zero, :meth:`join` unblocks.


Waiting for task completion
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Example of how to wait for enqueued tasks to be completed::

import threading
Expand Down Expand Up @@ -233,22 +233,38 @@ Example of how to wait for enqueued tasks to be completed::
Terminating queues
^^^^^^^^^^^^^^^^^^

:class:`Queue` objects can be made to prevent further interaction by shutting
them down.
When no longer needed, :class:`Queue` objects can be wound down
until empty or terminated immediately with a hard shutdown.

.. method:: Queue.shutdown(immediate=False)

Shut down the queue, making :meth:`~Queue.get` and :meth:`~Queue.put` raise
:exc:`ShutDown`.
Put a :class:`Queue` instance into a shutdown mode.

The queue can no longer grow.
Future calls to :meth:`~Queue.put` raise :exc:`ShutDown`.
Currently blocked callers of :meth:`~Queue.put` will be unblocked
and will raise :exc:`ShutDown` in the formerly blocked thread.

If *immediate* is false (the default), the queue can be wound
down normally with :meth:`~Queue.get` calls to extract tasks
that have already been loaded.

And if :meth:`~Queue.task_done` is called for each remaining task, a
pending :meth:`~Queue.join` will be unblocked normally.

Once the queue is empty, future calls to :meth:`~Queue.get` will
raise :exc:`ShutDown`.

By default, :meth:`~Queue.get` on a shut down queue will only raise once the
queue is empty. Set *immediate* to true to make :meth:`~Queue.get` raise
immediately instead.
If *immediate* is true, the queue is terminated immediately.
The queue is drained to be completely empty. All callers of
:meth:`~Queue.join` are unblocked regardless of the number
of unfinished tasks. Blocked callers of :meth:`~Queue.get`
are unblocked and will raise :exc:`ShutDown` because the
queue is empty.

All blocked callers of :meth:`~Queue.put` and :meth:`~Queue.get` will be
unblocked. If *immediate* is true, a task will be marked as done for each
remaining item in the queue, which may unblock callers of
:meth:`~Queue.join`.
Use caution when using :meth:`~Queue.join` with *immediate* set
to true. This unblocks the join even when no work has been done
on the tasks, violating the usual invariant for joining a queue.

.. versionadded:: 3.13

Expand Down
7 changes: 2 additions & 5 deletions Lib/asyncio/queues.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,9 +227,6 @@ def task_done(self):
been processed (meaning that a task_done() call was received for every
item that had been put() into the queue).

shutdown(immediate=True) calls task_done() for each remaining item in
the queue.

Raises ValueError if called more times than there were items placed in
the queue.
"""
Expand Down Expand Up @@ -257,8 +254,8 @@ def shutdown(self, immediate=False):
'immediate' to True to make gets raise immediately instead.

All blocked callers of put() and get() will be unblocked. If
'immediate', a task is marked as done for each item remaining in
the queue, which may unblock callers of join().
'immediate', unblock callers of join() regardless of the
number of unfinished tasks.
"""
self._is_shutdown = True
if immediate:
Expand Down
7 changes: 2 additions & 5 deletions Lib/queue.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,6 @@ def task_done(self):
have been processed (meaning that a task_done() call was received
for every item that had been put() into the queue).

shutdown(immediate=True) calls task_done() for each remaining item in
the queue.

Raises a ValueError if called more times than there were items
placed in the queue.
'''
Expand Down Expand Up @@ -240,8 +237,8 @@ def shutdown(self, immediate=False):
'immediate' to True to make gets raise immediately instead.

All blocked callers of put() and get() will be unblocked. If
'immediate', a task is marked as done for each item remaining in
the queue, which may unblock callers of join().
'immediate', callers of join() are unblocked regardless of
the number of unfinished tasks.
'''
with self.mutex:
self.is_shutdown = True
Expand Down
Loading