Skip to content

Fix historyId to be dependent on dynamic allure parameters #743

@delatrie

Description

@delatrie

I'm submitting a ...

  • bug report
  • feature request
  • support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Parameters added via the allure.dynamic.parameter inside a test body don't affect allure history of the test.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Run the following example multiple times to produce multiple *-result.json files:

import allure
import time

def test_issue743_reproduction():
    allure.dynamic.parameter("time", time.perf_counter())

These results all have the same historyId. In such a case allure reporter shows us only one test case with several retries:

image

What is the expected behavior?

There should exists one test case per run, each with no retries in a way, similar to native pytest parameters.

  1. Take the nodeid of a test.
  2. Take all dynamic parameters of the test with excluded set to False.
  3. Sort the parameters alphabetically by their names (historyId should not depend on parameters order).
  4. Append a string representation of the values to the nodeid (use some separator to prevent collisions with other tests).
  5. Calculate hash of the resulting string and use it as historyId.

Related code:

test_result.historyId = md5(item.nodeid)

Please tell us about your environment:

  • Allure version: 2.20.1
  • Test framework: pytest@7.3.1
  • Allure adaptor: allure-pytest@2.13.1

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions