Test Using Cucumber

Cucumber

Cucumber is a tool for running automated tests. It combines business specification and test documentation into one, resulting in an executable specification. This collaboration encourages teams to keep their organization’s business goals at the center.

Automated testing through Cucumber helps teams avoid heavy regressions and resultant costs.

Here is how you can specify the Test Cases and Test Steps that will be created in QMetry as a part of the result files -

Test Case Tag

Test Step Tag

elements > name

elements > steps > name

Supported Cucumber Version: 1.2.5

Supported file types: JSON, XML

The following feature file shows the syntax for story association and test case reusability.

Cucumber Feature File

Sample Test Result File

Click Cucumber.json to download the Sample Test Result File.

Story Association

While importing the test result file, if you want us to link the test case to any of the existing stories then it is possible. An annotation is added to the file to associate the test case with a particular story in JIRA.

Keyword

Download Sample File

Description

@storykey

Cucumber_multipleKeys.json

  • The created/linked test case will be linked to Story Key.

  • Multiple story keys (i.e. storykey) can be defined to link with the test case.

    For example, the story keys can be mentioned in tags as follows.

    Cucumber Storykey Example

  • If multiple test case keys are mentioned, then the multiple requirements will be linked individually with the test cases.

  • Users can also link cross-project stories to the test case.

    • Users should have access to the cross project.

Test Case Reusability

While importing the test result file, if you want us to reuse the Test Case then it is possible. An annotation is added to the file to reuse test cases in Jira.

Keyword

Download Sample File

Description

@testcasekey

Cucumber_multipleKeys.json

  • Multiple test case keys (i.e. testcasekey) can be defined for a single scenario, which will update/create multiple test cases for that single scenario and link these test cases with an automated test cycle.

For example, the test case keys can be mentioned in tags as follows.

Cucumber Test Case Key Example

  • When an invalid/archived test case entity is passed through the result JSON file:

    • If the executed test steps match the steps and summary of any version of the existing test case, then the existing test case version will be reused.

      • If the executed test steps do not match with the steps and summary of any version of the existing test case, then a new test case will be created using that element.

  • The mapping of test steps depends on the matchTestSteps parameter.

If the matchTestSteps parameter is set as “True” (Default): Create/Reuse a test case with a summary and test steps that exactly match the automated test case uploaded through the result file. The execution results and other execution details of the test case and steps will be imported from the automation result file.

  • If Test Case Summary and Test Step Summary (for all steps) match with the automated Test Case Name ( elements > name) and steps (elements > steps > name), Test Case Key and version will be reused.

    • If Test Case Summary matches with the automated Test Case ( elements > name) but Test Step Summary does not match (for any of the steps), Test Case Key will be reused but a new version will be created.

    • If Test Case Summary does not match, a new test case will be created.

If the matchTestSteps parameter is set as “False”:

When the Test Cycle is not been Reused

Create/Reuse a test case with a summary or test case key that exactly matches the automated test case uploaded through the result file, and exclude matching of test steps. The execution results of the test case will be imported or calculated based on the test case/step results from the automation result file. The execution result of the test case will be propagated to the test steps in the case of test case reuse/creation. Individual test case steps will not be matched and their execution results/details will not be picked from the result file.

When the Test Cycle is been Reused

  • When the Test Case Key is mapped in the result file and the Test Case Key is found linked to the Test Cycle.

    The existing linked test case version, which is part of the Test Cycle will be used. If multiple versions of the same test case key are linked to the test cycle, the one which traced first will be used. The test steps will not be matched to create a new version or link a different version.

  • When the Test Case Key is mapped in the result file and the Test Case Key is not linked to the Test Cycle and the Test Case Key exists in the Test Case Library.

    The existing latest version of the test case that matches based on the test case summary will be linked to the existing Test Cycle. If there are multiple test cases with the same summary exist, the one which is traced first will be linked to the existing Test Cycle. The test steps will not be matched to create a new version or link a different version.

  • When the Test Case Key is mapped in the result file and the Test Case Key is not found linked to the Test Cycle and the Test Case Key is not found in the Test Case Library, and the Test Case Summary matches with any existing test cases linked to the Test Cycle.

    The existing linked Test Case version which is part of the Test Cycle will be used. The test steps will not be matched to create a new version.

  • The Test Case Key is not mentioned in the result file and the Test Case Summary matches any existing test case that is already linked to the Test Cycle.

    The existing linked Test Case version which is part of the Test Cycle will be used based on the summary. The test steps will not be matched to create a new version.

  • When the Test Case Key is mapped in the result file and the Test Case Key does not exist in the Test Cycle OR is Not found in Library and the Test Case Summary also does not match any existing Test case in the Test Cycle and Test Case with the same summary is found in the Test Case Library.

    The existing latest version of the test case that matches based on the test case summary will be linked to the existing Test Cycle. If there are multiple test cases with the same summary exist, the one which is traced first will be linked to the existing Test Cycle. The test steps will not be matched to create a new version or link a different version.

  • When the Test Case Key is mapped in the result file and the Test Case Key does not exist in the Test Cycle OR is Not found in Library and the Test Case Summary also does not match any existing Test case in the Test Cycle OR is not found in the Test Case Library.

    A new test case without steps will be created and will be linked to the Test Cycle being reused.

    The execution results of the test case will be imported or calculated based on the test case/step results from the automation result file. The execution result of the test case will be propagated to the test steps in the case of test case reuse/creation. Individual test case steps will not be matched and their execution results/details will not be picked from the result file.

    In a project where propagation is off, the status of the step will not be mapped/changed.

How Test Case Reusability works if test case key is not defined in BDD file

While importing test result file, if you want us to reuse Test Case then it is possible.

  • If Test case summary and Test step summary (for all steps) match with the automated Test case name ( elements > name) and steps (elements > steps > name), Test case key and version will be reused.

  • If Test case summary matches with the automated Test case ( elements > name) but Test step summary does not match (for any of the steps), Test case key will be reused but a new version will be created.

  • If Test case summary does not match, a new Test case will be created.

Execution Attachment

To add attachments in test run execution, you need to capture the log file and screenshots during the execution of automation. The attachment file needs to be base64 encoded with mime_type in the embedding tag of the report file.

Configure Cucumber Project

Configure Cucumber Project

Result File Output with Attachment

Result File Output with Attachment

List of file types (mime types) supported in the cucumber framework.

Types of Attachment

Supported File Types

Image

image/png, image/bmp, image/gif, image/jpeg, image/x-icon

Text Document

text/plain, text/richtext, text/scriptlet, text/html, application/xhtml+xml, text/css, application/x-javascript, application/typescript, application/x-sh, application/pdf, application/rtf, application/xml

Excel, Word, PPT

application/msword, application/vnd.openxmlformats-officedocument.wordprocessingml.document, application/vnd.ms-excel, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, application/vnd.ms-powerpoint, application/vnd.openxmlformats-officedocument.presentationml.presentation

Audio File

audio/mpeg

Compressed File

application/zip, application/x-rar-compressed, application/x-7z-compressed, application/x-tar

Using Background Keyword

The Background in an element is applicable to all the scenarios in the element of the test result file.

Automation import in QTM4J supports uploading Cucumber execution result files that contain Background and test steps with tabular and other input values added to the feature file.

The following feature file shows the syntax for the Background keyword and test steps in tabular format.

Cucumber Background Keyword

Sample Test Result File

Click Cucumber File.json to download a Sample Test Result File.

In this section:
© 2025
Publication date: