Skip to content

Incorrect platform-specific dependency selected from universal requirements.txt using pip.parseΒ #2690

@ewianda

Description

@ewianda

🐞 bug report

Affected Rule

pip.parse

Is this a regression?

Unknown β€” this is observed with the current behavior in the latest release.

Description

I'm using a universal requirements.txt generated with uv

I want to use platform-specific dependencies like this:

requirements.in:

optimum[onnxruntime]; sys_platform == 'darwin' and sys_platform != 'linux'
optimum[onnxruntime-gpu]; sys_platform == 'linux'

requirements.txt (generated):

optimum[onnxruntime]==1.17.1 ; python_full_version < '3.12' and platform_python_implementation == 'CPython' and sys_platform == 'darwin'
optimum[onnxruntime-gpu]==1.17.1 ; python_full_version < '3.12' and platform_python_implementation == 'CPython' and sys_platform == 'linux'

MODULE.bazel:

pip.parse(
    requirements_by_platform = {
        "//:requirements.txt": "linux_*,osx_*",
    },
)

However, when building on macOS, Bazel attempts to use optimum[onnxruntime-gpu] instead of the appropriate optimum[onnxruntime].

It seems that the logic in the following file prioritizes the wrong requirement:

πŸ”—

requirements_dict = {
(normalize_name(entry[0]), _extract_version(entry[1])): entry
for entry in sorted(
parse_result.requirements,
# Get the longest match and fallback to original WORKSPACE sorting,
# which should get us the entry with most extras.
#
# FIXME @aignas 2024-05-13: The correct behaviour might be to get an
# entry with all aggregated extras, but it is unclear if we
# should do this now.
key = lambda x: (len(x[1].partition("==")[0]), x),
)
}.values()

As a result, the build fails with:

πŸ”₯ Exception or Error


rules_python~~pip~py_deps/onnxruntime_gpu/BUILD.bazel:6:12: configurable attribute "actual" in @@rules_python~~pip~py_deps//onnxruntime_gpu:_no_matching_repository doesn't match this configuration: No matching wheel for current configuration's Python version.

The current build configuration's Python version doesn't match any of the Python
wheels available for this distribution. This distribution supports the following Python
configuration settings:
    //_config:is_cp311_cp311_manylinux_2_28_x86_64
    //_config:is_cp311_cp311_manylinux_2_31_x86_64
    //_config:is_cp311_cp311_manylinux_x86_64

To determine the current config, I ran:

bazel config <config_id>

πŸ”¬ Minimal Reproduction

Repro steps:

  1. Use a universal requirements.in as shown above.
  2. Generate requirements.txt with uv pip compile requirements.in.
  3. Use pip.parse(requirements_by_platform=...) in MODULE.bazel.
  4. Build on macOS.

🌍 Your Environment

Operating System:


macOS 14.x

Output of bazel version:


Bazel 7.x

Rules_python version:


1.2

Anything else relevant?

A workaround mentioned by @aignas in Slack is to split requirements into separate files for different platforms. That works, but it would be ideal if this could be handled correctly from a single universal requirements file.

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: pippip/pypi integration

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions