Skip to content

Add CI test for cross-builds #127432

@FFY00

Description

@FFY00

Feature or enhancement

Proposal:

Cross-builds depend on a fair bit of sysconfig internals, it would be good have a test in the CI to ensure they work.

This would have prevented #127178 (comment).

I don't think there's a need to run the full test suite, we should be fine running just the sysconfig, site, and test_embed tests. It would also make sense to run the tests in the build directory, and after installation.

The test should consist of:

  • Building a host Python, specifying a custom prefix (eg. workdir/host-python)
  • Installing the host Python
  • Configuring a new build using --with-build-python=workdir/host-python/bin/python, and also using a custom prefix (eg. workdir/cross-python)
  • Running the tests in the build directory (./python -m test test_sysconfig test_site test_embed)
  • Installing the cross-build
  • Running the tests again, but with the installed Python (workdir/cross-python/bin/python -m test test_sysconfig test_site test_embed)

I think we should be fine with Linux-only, but Windows would also be welcome. Additionally, possibly, we could also use a different architecture in the cross-build, like jit.yml does for "Emulated Linux".

- name: Emulated Linux
if: runner.os == 'Linux' && matrix.architecture != 'x86_64'
# The --ignorefile on ./python -m test is used to exclude tests known to fail when running on an emulated Linux.
run: |
sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
./configure --prefix="$(pwd)/../build"
make install --jobs 4
make clean --jobs 4
export HOST=${{ matrix.architecture }}-linux-gnu
sudo apt install --yes "gcc-$HOST" qemu-user
export QEMU_LD_PREFIX="/usr/$HOST"
CC="$HOST-gcc" \
CPP="$HOST-gcc --preprocess" \
HOSTRUNNER=qemu-${{ matrix.architecture }} \
./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' || '' }} --build=x86_64-linux-gnu --host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes
make all --jobs 4
./python -m test --ignorefile=Tools/jit/ignore-tests-emulated-linux.txt --multiprocess 0 --timeout 4500 --verbose2 --verbose3

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildThe build process and cross-buildeasyinfraCI, GitHub Actions, buildbots, Dependabot, etc.type-featureA feature request or enhancement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions