Skip to content

GH-128469: warn when libpython was loaded from outside the build directory #128645

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jan 29, 2025
Merged

GH-128469: warn when libpython was loaded from outside the build directory #128645

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
867cda8
GH-128469: warn when libpython was loaded from outside the build dire…
FFY00 Jan 8, 2025
29db6f0
Update Modules/getpath.py
FFY00 Jan 11, 2025
bf40c17
Fix check
FFY00 Jan 11, 2025
7bb0e9f
Add test
FFY00 Jan 11, 2025
fe5e882
Fix test availability
FFY00 Jan 11, 2025
004f193
Make test availability a bit stricter
FFY00 Jan 11, 2025
59d2263
Merge branch 'main' into gh-128469-warning
FFY00 Jan 29, 2025
a8c9fa7
Fix test without --enable-dynamic
FFY00 Jan 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions Lib/test/test_getpath.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
import copy
import ntpath
import os
import pathlib
import posixpath
import shutil
import subprocess
import sys
import sysconfig
import tempfile
import unittest

from test.support import verbose
Expand Down Expand Up @@ -864,6 +870,37 @@ def test_PYTHONHOME_in_venv(self):
actual = getpath(ns, expected)
self.assertEqual(expected, actual)


class RealGetPathTests(unittest.TestCase):
@unittest.skipUnless(
sysconfig.is_python_build(),
'Test only available when running from the buildir',
)
@unittest.skipUnless(
any(sys.platform.startswith(p) for p in ('linux', 'freebsd', 'centos')),
'Test only support on Linux-like OS-es (support LD_LIBRARY_PATH)',
)
@unittest.skipUnless(
sysconfig.get_config_var('LDLIBRARY') != sysconfig.get_config_var('LIBRARY'),
'Test only available when using a dynamic libpython',
)
def test_builddir_wrong_library_warning(self):
library_name = sysconfig.get_config_var('INSTSONAME')
with tempfile.TemporaryDirectory() as tmpdir:
shutil.copy2(
os.path.join(sysconfig.get_config_var('srcdir'), library_name),
os.path.join(tmpdir, library_name)
)
env = os.environ.copy()
env['LD_LIBRARY_PATH'] = tmpdir
process = subprocess.run(
[sys.executable, '-c', ''],
env=env, check=True, capture_output=True, text=True,
)
error_msg = 'The runtime library has been loaded from outside the build directory'
self.assertTrue(process.stderr.startswith(error_msg), process.stderr)


# ******************************************************************************

DEFAULT_NAMESPACE = dict(
Expand Down
13 changes: 13 additions & 0 deletions Modules/getpath.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -785,6 +785,19 @@ def search_up(prefix, *landmarks, test=isfile):
base_exec_prefix = config.get('base_exec_prefix') or EXEC_PREFIX or base_prefix


# ******************************************************************************
# MISC. RUNTIME WARNINGS
# ******************************************************************************

# When running Python from the build directory, if libpython is dynamically
# linked, the wrong library might be loaded.
if build_prefix and library and not dirname(abspath(library)).startswith(build_prefix):
msg = f'The runtime library has been loaded from outside the build directory ({library})!'
if os_name == 'posix':
msg += ' Consider setting LD_LIBRARY_PATH=. to force it to be loaded from the build directory.'
warn(msg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to preemptively error out here and force the user to set up LD_LIBRARY_PATH or a similar mechanism?

The other approach (rpath) was rejected for requiring multiple link jobs but would have enforced the correct library. Making it an error to use the wrong library feels a bit closer in spirit to that. And I'm not sure there's a valid use case for people building a worktree python but loading the wrong library, except when messing up -- and the consequences could be strange forms of crashing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like a warning is enough for now. I think erroring out may possibly be a good next step, but I want to see how this impacts the development workflow for other folks before turning it into a hard error.



# ******************************************************************************
# SET pythonpath FROM _PTH FILE
# ******************************************************************************
Expand Down
Loading