-
Notifications
You must be signed in to change notification settings - Fork 252
Description
Currently, allure-pytest
has a bunch of filters to run tests selectively:
--allure-epics
--allure-features
--allure-severities
--allure-stories
--allure-ids
We need to replace them with one general filtering mechanism based on labels.
Once this is done, the old filtering parameters could be deprecated in favor of the new one.
Motivation
Label-based test selection is more flexible in general. Since epics, features, stories, severities and ids are all labels, those filtering scenarios will be automatically supported.
In addition, other scenarios, like #633 will be supported automatically.
Additionally, custom labels (like ones, created from system env vars, see #721) would also be usable for test selection.
The syntax will become a little bit bulkier but it's always possible to wrap the invocation with a particular set of filters into a command runner like poetry
or hatch
.
Other information
New parameter may look like this:
--allure-label LABEL=VALUE
or
--allure-label-filter LABEL=VALUE
(longer, but more expressive)
Conjunction of filters
It should be possible to specify the parameter multiple times to replace the following usage
pytest --alluredir=allure-results --allure-epics=my-epic --allure-severities=blocker
with the new syntax:
pytest --alluredir=allure-results --allure-label epic=my-epic --allure-label severity=blocker
This usage only selects tests matching both filters at the same time.
Disjunction of filters
It should be also possible to provide an alternative to calls like this:
pytest --alluredir=allure-results --allure-severities=blocker,critical
where tests with either severity=blocker
or severity=critical
labels are selected.
Since all the current filters use the comma symbol to separate allowed values, we can maintain this behavior. Additionally, we should provide for possibility of a value itself containing a comma by implementing an escaping mechanism (currently this is not supported; but labels like feature, or story may potentially contain commas).
Example 1: selecting tests with severity either blocker
or critical
:
pytest --alluredir=allure-results --allure-label severity=blocker,critical
Example 2: selecting tests for the features Red, green
or Plus\minus
:
pytest --alluredir=allure-results --allure-label 'feature=Red\, green,Plus\\minus'