Skip to content

CI: cache pip dependencies #22574

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

tacaswell
Copy link
Member

This appears to be a new feature now that we are on v3 of the action.

Follow on to #22573 .

https://github.com/actions/setup-python#caching-packages-dependencies

@tacaswell tacaswell added this to the v3.6.0 milestone Feb 28, 2022
This appears to be a new feature now that we are on v3 of the action.

Follow on to matplotlib#22573 .
@tacaswell
Copy link
Member Author

The requirements file format allows to specify dependency versions using logical operators (for example chardet>=3.0.4) or specify dependencies without any versions. In this case the pip install -r requirements.txt command will always try to install the latest available package version. To be sure that the cache will be used, please stick to a specific dependency version and update it manually if necessary.

Although, it looks like both this and the caching we are currently doing end up being counter productive because we do not pin tightly enough in our requirements file (other than the min version run).

@greglucas
Copy link
Contributor

Is there an easy way to tell if this is hitting the cache before merging?
From the action examples, it looks like you might need to add the dependency wildcard path.

cache-dependency-path: 'requirements/*/*.txt'

@tacaswell tacaswell marked this pull request as draft March 6, 2022 04:15
@tacaswell
Copy link
Member Author

I'm not sure this is doing the right thing at all (and have concerns about our current caching as well!)

I spent some time yesterday and today trying to get tests to run on old versions of Matplotlib which....was harder than I wished it would be. We may want to have one very pinned requirements file (possible managed by one of the auto-bump services 😞 or tools) that we rename to fully-pinned-v3.6.x.txt on branching. We can then use that (exclusively) for tests on the backport branch which should hopefully reduce the amount of dep-only backports we have to do (and provide a useful reference for anyone who wants to get old versions testing again).

@tacaswell
Copy link
Member Author

This is bit rotted and I do not think this is the right apporach by itself. Replaced by #23548

@tacaswell tacaswell closed this Aug 3, 2022
@tacaswell tacaswell deleted the ci_gha_pipcache branch August 3, 2022 22:02
@tacaswell tacaswell modified the milestones: v3.7.0, unassigned Aug 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants