Skip to content

[3.13] gh-132898: Add a note in multiprocessing.Process docs about creating a process in a REPL (GH-137118) #137155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Doc/library/concurrent.futures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,11 @@ that :class:`ProcessPoolExecutor` will not work in the interactive interpreter.
Calling :class:`Executor` or :class:`Future` methods from a callable submitted
to a :class:`ProcessPoolExecutor` will result in deadlock.

Note that the restrictions on functions and arguments needing to picklable as
per :class:`multiprocessing.Process` apply when using :meth:`~Executor.submit`
and :meth:`~Executor.map` on a :class:`ProcessPoolExecutor`. A function defined
in a REPL or a lambda should not be expected to work.

.. class:: ProcessPoolExecutor(max_workers=None, mp_context=None, initializer=None, initargs=(), max_tasks_per_child=None)

An :class:`Executor` subclass that executes calls asynchronously using a pool
Expand Down
56 changes: 48 additions & 8 deletions Doc/library/multiprocessing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,10 @@
For an explanation of why the ``if __name__ == '__main__'`` part is
necessary, see :ref:`multiprocessing-programming`.

The arguments to :class:`Process` usually need to be unpickleable from within
the child process. If you tried typing the above example directly into a REPL it
could lead to an :exc:`AttributeError` in the child process trying to locate the
*f* function in the ``__main__`` module.


.. _multiprocessing-start-methods:
Expand Down Expand Up @@ -218,9 +222,12 @@
the *fork* context cannot be passed to processes started using the
*spawn* or *forkserver* start methods.

A library which wants to use a particular start method should probably
use :func:`get_context` to avoid interfering with the choice of the
library user.
Libraries using :mod:`multiprocessing` or
:class:`~concurrent.futures.ProcessPoolExecutor` should be designed to allow
their users to provide their own multiprocessing context. Using a specific
context of your own within a library can lead to incompatibilities with the
rest of the library user's application. Always document if your library
requires a specific start method.

.. warning::

Expand Down Expand Up @@ -518,9 +525,42 @@
to pass to *target*.

If a subclass overrides the constructor, it must make sure it invokes the
base class constructor (:meth:`Process.__init__`) before doing anything else
base class constructor (``super().__init__()``) before doing anything else
to the process.

.. note::

In general, all arguments to :class:`Process` must be picklable. This is
frequently observed when trying to create a :class:`Process` or use a
:class:`concurrent.futures.ProcessPoolExecutor` from a REPL with a
locally defined *target* function.

Passing a callable object defined in the current REPL session causes the
child process to die via an uncaught :exc:`AttributeError` exception when
starting as *target* must have been defined within an importable module
in order to be loaded during unpickling.

Example of this uncatchable error from the child::

>>> import multiprocessing as mp
>>> def knigit():
... print("Ni!")
...
>>> process = mp.Process(target=knigit)
>>> process.start()
>>> Traceback (most recent call last):
File ".../multiprocessing/spawn.py", line ..., in spawn_main
File ".../multiprocessing/spawn.py", line ..., in _main
AttributeError: module '__main__' has no attribute 'knigit'
>>> process
<SpawnProcess name='SpawnProcess-1' pid=379473 parent=378707 stopped exitcode=1>

See :ref:`multiprocessing-programming-spawn`. While this restriction is

Check warning on line 558 in Doc/library/multiprocessing.rst

View workflow job for this annotation

GitHub Actions / Docs / Docs

undefined label: 'multiprocessing-programming-spawn' [ref.ref]
not true if using the ``"fork"`` start method, as of Python ``3.14`` that
is no longer the default on any platform. See
:ref:`multiprocessing-start-methods`.
See also :gh:`132898`.

.. versionchanged:: 3.3
Added the *daemon* parameter.

Expand Down Expand Up @@ -2993,10 +3033,10 @@

More picklability

Ensure that all arguments to :meth:`Process.__init__` are picklable.
Also, if you subclass :class:`~multiprocessing.Process` then make sure that
instances will be picklable when the :meth:`Process.start
<multiprocessing.Process.start>` method is called.
Ensure that all arguments to :class:`~multiprocessing.Process` are
picklable. Also, if you subclass ``Process.__init__``, you must make sure
that instances will be picklable when the
:meth:`Process.start <multiprocessing.Process.start>` method is called.

Global variables

Expand Down
Loading