Feature or enhancement

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Proposal:

I propose to add in libregrtest the support of selecting tests not only by names, but also by labels. Some labels are set when the test is decorated with corresponding decorator, others can be set by user manually.

For example, "requires_cpu" will mark tests decorated with @requires_resouce('cpu'), "impl_detail_cpython" will mark tests decorated with @cpython_only, "bigmemtest" will mark tests decorated with @bigmemtest(...). Tests related to pickling can be manually marked with label "pickletest".

Added two options --label=NAME for including only tests with the specified label, --no-label=NAME for excluding tests with the specified label.

You can request all test cases with the specified label using option --list-cases. You can run only tests with or without the specified label.

Added function mark() in the test.support module for manual marking tests.

@test.support.mark('pickletest')
def test_pickling(self):

Added also functions skipIf() and skipUnless() with additional argument specifying the label. It is for easier creation of custom decorators, e.g.

requires_foo = test.support.skipUnless(has_foo, 'requires Foo', label='requires_foo')

If you simply decorate the test method or class, you can just combine decorators:

@unittest.skipUnless(has_foo, 'requires Foo')
@test.support.mark('requires_foo')
def test_with_foo(self):

Adding the label to the test class is equivalent to adding the label to every method of that class.

Currently it only works with decorated test classes and test methods. If the whole module is skipped by using requires('gui'), classes and tests in that module cannot be found by the "requires_gui" label. If the test requires, for example, the "network" resource, but calls @requires('network') instead of been decorated with @requires_resource('network'), it cannot be found by the "requires_network" label.

It is an experimental feature, and can be changed in future (for example support labels with values, support glob patterns for labels). I think that in future some version of this feature will be added in unittest.

Linked PRs

• gh-108828: Support selecting tests by labels #108829