local_runtime_repo fails to discover and link against python3.8 shared libraries on Ubuntu 20.04 #3099
Description
🐞 bug report
Affected Rule
local_runtime_repo()
Description
The @rules_python//python/cc:current_py_cc_libs rule fails to properly link against libpython3.8.so.
When debugging, I found that local_runtime_repo() failed to configure a :libpython with any srcs. This may be a python3.8-specific bug.
local_runtime_repo(
name = "local_python38",
interpreter_path = "python3.8",
on_failure = "fail",
)
Debugging
On python3.8 (on Ubuntu 20.04):
import sysconfig
sysconfig.get_config_vars("LIBDIR", "MULTIARCH") -> ["/usr/lib", "x86_64-linux-gnu"]
With python3.12 (on Ubuntu 24.04):
import sysconfig
sysconfig.get_config_vars("LIBDIR", "MULTIARCH") -> ["/usr/lib/x86_64-linux-gnu", "x86_64-linux-gnu"]
The issue goes away when I locally apply the following patch to rules_python@v1.5.0:
diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py
index 19db3a29..c5f62bb1 100644
--- a/python/private/get_local_runtime_info.py
+++ b/python/private/get_local_runtime_info.py
@@ -45,6 +45,8 @@ config_vars = [
# The platform-specific filename suffix for library files.
# Includes the dot, e.g. `.so`
"SHLIB_SUFFIX",
+ # Architecture-specific tuple. (Used for discovering library files.)
+ "MULTIARCH",
]
data.update(zip(config_vars, sysconfig.get_config_vars(*config_vars)))
print(json.dumps(data))
diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl
index ec0643e4..81a410a8 100644
--- a/python/private/local_runtime_repo.bzl
+++ b/python/private/local_runtime_repo.bzl
@@ -119,6 +119,8 @@ def _local_runtime_repo_impl(rctx):
shared_lib_names = {v: None for v in shared_lib_names}.keys()
shared_lib_dir = info["LIBDIR"]
+ multiarch = info["MULTIARCH"]
+
# The specific files are symlinked instead of the whole directory
# because it can point to a directory that has more than just
# the Python runtime shared libraries, e.g. /usr/lib, or a Python
@@ -126,12 +128,16 @@ def _local_runtime_repo_impl(rctx):
rctx.report_progress("Symlinking external Python shared libraries")
for name in shared_lib_names:
origin = rctx.path("{}/{}".format(shared_lib_dir, name))
+ origin_multiarch = rctx.path("{}/{}/{}".format(shared_lib_dir, multiarch, name))
# The reported names don't always exist; it depends on the particulars
# of the runtime installation.
if origin.exists:
repo_utils.watch(rctx, origin)
rctx.symlink(origin, "lib/" + name)
+ if origin_multiarch.exists:
+ repo_utils.watch(rctx, origin_multiarch)
+ rctx.symlink(origin_multiarch, "lib/" + name)
rctx.file("WORKSPACE", "")
rctx.file("MODULE.bazel", "")
🌍 Your Environment
Operating System:
Ubuntu 20.04
Rules_python version:
1.5.0
Anything else relevant?