Leapwork Test Automation is often used in conjunction with Test Management products. This document describes the integration between Leapwork Test Automation and Jira Xray Test Management.
Leapwork’s no-code Test Automation technology makes it quicker and easier for regular business users and IT teams to adopt, so that they can scale automation faster - ultimately empowering quality assurance teams, leading to shorter test cycles and immediate return on investment. This means that enterprises can continue to grow and optimize their performance because they can rely on their business software to work as it should, with test automation as an integral part of their software development lifecycle.
The role of software testing is becoming increasingly important in the software development lifecycle to ensure that software is released with the highest possible quality. With an increased amount of testing, more advanced test plans and strategies, it is evident that a proper test management tool is needed. Xray for Jira is a Test Management tool that provides the structure to organize, plan, and report with accuracy on the progress of testing as well as the readiness to deploy.
CI-CD Pipeline implementation of the Continuous Integration/Continuous Deployment software is the backbone of the modern DevOps environment. Most of the organization using DevOps practices and using Jenkins. Jenkins is an open source CI tool which can be used to execute the Leapwork tests. Jenkins is a Continuous Integration tool which does this verification periodically or whenever the developer pushes the code to the repository.
Both Leapwork and Jira Xray integrate natively with Jenkins, and so witch each other.
Picture 1: Example of Leapwork reporting in XRay
Both the Jira Xray and Leapwork plugins are installed from within the Jenkins “Plugin” store.
Open Manage Jenkins -> Manage Plugins
Picture 2: Plugin configuration in Jenkins
Leapwork plugin for Jenkins: Leapwork Integration
Jira Xray integration: Xray - Test Management for Jira Plugin
Picture 3: Leapwork Plugin in Jenkins
This section will describe the configuration of the different components in Leapwork, Xray and Jenkins
Picture 4: Architecture of Xray integration and Jenkins.
Picture 5: Entity relationships between Xray and Leapwork
Jira Xray configuration
Before configuring the Jenkins pipe, two things need to be configured in Jira Xray:
- One or more test plans which correspond with a run lists in Leapwork.
The Jenkins pipeline will map a Test Plan with a Run List in Leapwork. In Jira Xray create an issue of type “test plan”. A logical name could be to use the corresponding name of the Run list in Leapwork.
- The authentication for Jenkins.
Jenkins will create the tests and update the test plan on behalf of the Leapwork server. It therefore makes sense to create a new user in Xray called “Leapwork Server” or similar. After creating the user, goto the “XRAY API Keys” (Jira settings -> Apps -> Xray API Keys), and create a new key for Jenkins. Note both the “Client ID and Client secret.
NOTE: The Xray API keys are not the same as Jira API Keys!
Picture 6: API Keys in XRay
The Leapwork configuration is limited to:
- Creating an API key. (Settings-> API access Keys). Make a note of the key
- A Runlist. Create a runlist with required test (flows).
- A Schedule Create an “on Demand” schedule and make a note of the name.
Picture 7: Run lists in Leapwork
First step in Jenkins is to configure the Jira Xray server instance. In Jenkins goto:
Manage Jenkins -> Configure System
When the Xray plugin is configured correctly you will find an “Xray Configuration section”. Configure the server details and provide an alias. For credentials press “add” and use the Xray Client ID and Xray API key created previously. Use the “test connection” button to test the configuration.
Picture 8: Xray application configuration in Jenkins
Then press “New Item” button in Jenkins. A logical name could be similar to the Runlist \ Test plan names. Next to optional other configuration, press the “Add build step” button and select “Leapwork integration”. The Leapwork plugin will appear. Configure the following steps:
Picture 9: Leapwork plugin configuration in Jenkins
Optionally create access token:
In order to launch the Jenkins task from within Jira, an access token is required. In Jenkins goto:
Your username (top right corner) -> Configure -> API token
Make a note of the token. If the Jenkins server and Jira server are not within the same domain than you will need to troubleshoot the CSRF token authentication.
After configuring the Leapwork Plugin you may configure the Xray plugin. Press the button “Add post-build action” and select “Xray: Results Import Task”. Configure the following steps:
Picture 10: XRay plugin configuration in Jenkins
Optional JIRA configuration: Execute the pipeline from within Jira
You may want to start the Automated Tests manualy from within the Jira Test Plan. In order to do this, in Jira browse to Settings -> System -> Automation Rules
- Press the “Create Rule” button”.
- Select “Manual trigger”
- Specify user group
- Click “new action”
- Select “Send web request”
- Configure the web request as specified below.
Picture 11: Optionally configure Jira automation rule.
Leapwork Jira Xray integration in action
After configuring the plugins, one or more release pipelines can be configured in Jenkins in order to launch the automated tests, and report the results back to Xray.
Step 1: Trigger CI/CD Pipeline
In order to launch (start) the pipeline you may configure a manual start from within Jira Xray. This is configured within the Jira “automation rules”. Another option could be to trigger the Jenkins pipeline by other means.
Picture 12: Optionally start pipe with Jira automation rule.
Step 2: Execution of Jenkins pipeline
The progress of the Jenkins pipeline can be monitored by means of the console output: Pipeline\Instance\Console output
The Leapwork Automated tests could run for quite some time (depending on the test requirements). During this period the Jenkins job will keep polling the Leapwork server for results. The polling interval is based on the Leapwork plugin configuration. (
During each server request the following text will be displayed:
The schedule status is already 'Executing' - wait a minute...
The default polling interval is 5 seconds.
Picture 13. Sample console output Leapwork automated tests.
Step 3: Creation or update of Jira Xray “Tests”.
When the Jenkins pipeline is finished, the Xray plugin will create or update a “test” for each flow which was part of the runlist.
Picture 14: Test case in XRay
The “Test Runs” section will contain an instance for every time the flow (test) was executed:
Picture 15: Xray Test runs
For each test run, the flow (test) details can be retrieved:
Picture 16: Xray Test details page with Leapwork keyframes.
Step 4: Update o associated Jira Xray “Test Plan”
The associated Jira Xray “Test Plan” will display the latest test results per flow (tests):
Picture 17: Xray Test Plan
Step 5: For each run a new “Execution results” ticket will be created
Picture 18: Execution result in Xray.
For any clarification, please contact our Priority Support.