Articles in this section

Leapwork Azure DevOps PowerShell based Integration

Avatar
Praveen Kumar
  • Updated
Follow

Automatic creation of bugs in Azure DevOps Server

An important part of running automated test cases is to monitor, inspect, and react to the results. This is best accomplished in the test management and bug tracking system that we use for our daily work.

In this article, we will review three different scenarios on how to trigger and create/update bugs in the Azure DevOps Server, formally known as Visual Studio Team Services (VSTS) or Team Foundation Server (TFS), based on the results of test cases running in Leapwork.

 

Use a plugin for Azure DevOps Servers

Use the Leapwork plugin to easily integrate with the Azure DevOps Servers. This extension can be installed from the Visual Studio Marketplace as follows:

This Leapwork plugin is a community-developed app and can be found on GitHub.

 

Create bugs from schedules

To run a collection of test cases in one or more environments (machines, devices, etc.) on a scheduled basis or by triggering test cases through the Leapwork REST API, we must define a schedule. In a schedule, we can select one or more test cases from a Leapwork project and pair them with one or more environments. Furthermore, we can set the duration and frequency of scheduled test case runs.

In a schedule, we can select Add action and define one or more actions that should be performed during a test case. One example of an action is to send an e-mail when a schedule has ended. Another is to call a specified web service if more than X number of test cases in the schedule has failed.

In this case, a PowerShell action is triggered when the schedule is finished. This action is used to call a local PowerShell script and pass the result of the schedule to the PowerShell script. An example of how to specify this action is shown in the following figure:

tfs

The command tells Leapwork to run a PowerShell script, c:\scripts\createTFSbugs.ps1. 

The following parameters have been added: 

  • -result: This parameter is read by the PowerShell script and is the result of the entire schedule. To add the result data, right-click in the command field and select Insert token -> RESULTS-JSON.

    Warning: The inserted token must be in between

    @”

    [


    and

    ]

    “@

    and must include the line breaks. This structure ensures that PowerShell transfers the JSON result format correctly.
  • -rootPath: This parameter is read by the PowerShell script and is the file path to the folder containing the videos and screenshots recorded during the execution of the test cases.

The PowerShell script, CreateTFSbugs.ps1, will connect to the Azure DevOps Server REST API and create or update bugs based on the failed test cases in the schedule.

To use the PowerShell script, we need to specify a few settings about our Azure DevOps Server installation:

tfs 2

Domain: If we are using Visual Studio Online (VSO), insert our visualstudio.com domain name, for example, https://LEAPWORK.visualstudio.com

tfsPath: The path to our project.

pat: Personal Access Token (PAT). We can find more information about setting up a PAT here.

logfile: The path to the integration log file. All log entries are appended to this file.

The script has been tested against VSO, but it’s the same API used for the on-premise installations of Azure DevOps Server. There may be differences in various REST API versions. Refer to Azure DevOps documentation for further information.

 

Trigger schedules from API and create bugs in Azure DevOps Server

Integrating Leapwork with any third-party system through the REST API can be done in a few simple steps. This enables us to easily set up the automatic creation or updating of bugs in Azure DevOps Server when test automation cases fail in Leapwork.

 

As part of the integration, we have a Windows Powershell script that runs a pre-defined LEAPWORK run list, polls for the results until they are available, and then loops through all failed cases and creates bugs in JIRA as appropriate using API's.

 

# LEAPWORK REST API example: Run a schedule, iterate through the results and create bugs in TFS.# 
#
# Author: Claus Topholt.


# Function that finds a schedule in LEAPWORK based on a title, runs it and polls for the results.
function RunScheduleAndGetResults($schedule)
{
    Write-Host "Getting id for schedule '$schedule'."

    # Get the id of the schedule.
    $runScheduleId = "";
    $headers = @{}
    $headers.Add("AccessKey","bTyGAd0UGL70JFQg")
    $runSchedules = Invoke-WebRequest -ContentType "application/json" -Headers $headers "http://localhost:9001/api/v3/schedules" | ConvertFrom-Json
    foreach($runScheduleItem in $runSchedules)
    {
        if ($runScheduleItem.title -eq $schedule) { $runScheduleId = $runScheduleItem.id }
    }
    if ($runScheduleId -eq "") { throw "Could not find schedule '$schedule'." }

    Write-Host "Running the schedule."

    # Run the schedule now.
    $timestamp = [DateTime]::UtcNow.ToString("ddMMyyyy HHmmss")
    Start-Sleep 1
    $runNow = Invoke-WebRequest -Method PUT -ContentType "application/json" -Headers $headers "http://localhost:9001/api/v3/schedules/$runScheduleId/runNow"
    if ($runNow.StatusCode -ne 200) { throw "Could not run schedule." }
    $runNowResponse=$runNow.Content | ConvertFrom-Json
    $runId=$runNowResponse.RunId
    # Get the result, keep polling every 5 seconds until a new result is returned.
    do
    {
        Start-Sleep -Seconds 5

        Write-Host "Polling for run results."

        $runResult = Invoke-WebRequest -ContentType "application/json" -Headers $headers "http://localhost:9001/api/v3/run/$runId" | ConvertFrom-Json         

    } 
    while ($runResult.Status -ne 'Finished')

    Write-Host "Results received."

    return $runResult
}


# Function that creates a bug with a specific title in TFS or updates if it already exists.
function CreateOrUpdateBug($title, $description)
{
    # Create an authorization header using a Personal Access Token.
    $pat = "YOUR PERSONAL ACCESS TOKEN HERE"
    $patEncoded = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($pat)"));
    $header = @{"Authorization" = "Basic $patEncoded"}

    # See if a bug with the same title already exists.
    $queryAlreadyExists = '{ "query": "SELECT [System.Id], [System.Title], [System.State] FROM WorkItems ' +
                          'WHERE [System.WorkItemType] = ''Bug'' AND [System.Title] = ''' + $bugTitle + ''' AND [System.State] <> ''Removed'' ' + 
                          'ORDER BY [System.Id] DESC" }'
    $urlAlreadyExists = "https://YOURINSTANCE.visualstudio.com/DefaultCollection/YOURPROJECT/_apis/wit/wiql?api-version=1.0"
    $header['Content-Type'] = 'application/json'
    $resultAlreadyExists = Invoke-RestMethod -Headers $header -Method Post -Body $queryAlreadyExists $urlAlreadyExists

    # If the bug already exists, update it.
    if ($resultAlreadyExists.workItems.Count -gt 0)
    {
        Write-Host "Updating existing bug $($resultAlreadyExists.workItems[0].id)."

        # Add a note that this bug was updated or re-opened.
        $timestamp = [DateTime]::UtcNow.ToString("dd-MM-yyyy HH:mm:ss")
        $bugDescription = "Updated or re-opened by LEAPWORK on $timestamp.

" + $bugDescription

        # Update bug.
        $queryUpdateBug = '[ { "op" : "replace", "path" : "/fields/System.Title", "value" : "' + $bugTitle + '" }, ' +
        '{ "op" : "add", "path" : "/fields/Microsoft.VSTS.TCM.ReproSteps", "value" : "' + $bugDescription + '" }, ' + 
        '{ "op" : "replace", "path" : "/fields/System.State", "value" : "New" } ]'
        $urlUpdateBug = "https://YOURINSTANCE.visualstudio.com/DefaultCollection/_apis/wit/workitems/$($resultAlreadyExists.workItems[0].id)?api-version=1.0"
        $header['Content-Type'] = 'application/json-patch+json'
        $resultUpdateBug = Invoke-RestMethod -Headers $header -Method Patch -Body $queryUpdateBug $urlUpdateBug
    }
    else
    {
        Write-Host "Creating new bug."

        # Create new bug.
        $queryCreateNewBug = '[ { "op" : "add", "path" : "/fields/System.Title", "value" : "' + $bugTitle + '" }, ' +
        '{ "op" : "add", "path" : "/fields/Microsoft.VSTS.TCM.ReproSteps", "value" : "' + $bugDescription + '" }, ' + 
        '{ "op" : "add", "path" : "/fields/System.State", "value" : "New" } ]'
        $urlCreateNewBug = 'https://YOURINSTANCE.visualstudio.com/DefaultCollection/YOURPROJECT/_apis/wit/workitems/$Bug?api-version=1.0'
        $header['Content-Type'] = 'application/json-patch+json'
        $resultCreateNewBug = Invoke-RestMethod -Headers $header -Method Patch -Body $queryCreateNewBug $urlCreateNewBug
    }
}


# Run the LEAPWORK schedule "My Test Schedule" and get the results.
$runResult = RunScheduleAndGetResults("My Test Schedule")

# If there are any failed cases in the results, iterate through them.
if ($runResult.Failed -gt 0)
{
    Write-Host "Found $($runResult.Failed) failed case(s)."
    $headers = @{}
    $headers.Add("AccessKey","bTyGAd0UGL70JFQg")
    $runId=$runResult.RunId
    $runItemIds = Invoke-WebRequest -ContentType "application/json" -Headers $headers http://localhost:9001/api/v3/run/$runId/runItemIds | ConvertFrom-Json 

    $rootPath =$runResult.RunFolderPath
    foreach ($runItemId in $runItemIds.RunItemIds)
    {   
   
    $runItems = Invoke-WebRequest -ContentType "application/json" -Headers $headers http://localhost:9001/api/v3/runItems/$runItemId | ConvertFrom-Json 

        if($runItems.FlowInfo.Status -eq 'Failed')
        {

         # Create a title for the bug.
            $bugTitle = "LEAPWORK: " + $runItems.FlowInfo.FlowTitle

            # Create a description that contains the log messages.
            $newline = "\r\n";
           
            $keyFrames = Invoke-WebRequest -ContentType "application/json" -Headers $headers http://localhost:9001/api/v3/runItems/$runItemId/keyframes/1 | ConvertFrom-Json 

            $bugDescription = "Log from LEAPWORK:$newline $newline"
            foreach ($keyframe in $keyFrames)
            {
                if ($keyframe.Level -ge 1)
                {
                    $keyframeTimestamp = get-date($keyframe.Timestamp.LocalDateTime) -Format "dd-MM-yyyy HH:mm:ss"
                    $bugDescription += "$keyframeTimestamp - $($keyframe.LogMessage) $newline"
                }
            }

            # Add path to video and screenshots.
            $mediaPath = Join-Path -Path $rootPath  "$($runItems.RunItemId)"
            $videoPath = Join-Path -Path $mediaPath  "$($runItems.RunItemId).avi"
            $videoPath = $videoPath.Replace('\', '\\')
            $screenshotsPath = Join-Path -Path $mediaPath "Screenshots"
            $screenshotsPath = $screenshotsPath.Replace('\', '\\')
            $bugDescription += "$newline Video: $videoPath $newline"
            $bugDescription += "$newline Screenshots (if any): $screenshotsPath $newline"

           
            # Create or update bug in TFS.
            CreateOrUpdateBug($bugTitle, $bugDescription)
        }
    }

}
else
{
    Write-Host "No failed cases found."
}

 

The following changes are required above script:-

  • Generate an AccessKey from Leapwork Studio->Settings->Add Key.
  • Replace the generated key with Accesskey value

          $headers.Add("AccessKey","bTyGAd0alksslkals")

# Create an authorization header using a Personal Access Token.
$pat = "YOUR PERSONAL ACCESS TOKEN HERE"
  • Enter the Azure DevOps Instance URL
 $urlAlreadyExists = "https://YOURINSTANCE.visualstudio.com/DefaultCollection/YOURPROJECT/_apis/wit/wiql?api-version=1.0" 

After running the PowerShell script, the schedule will be executed and for failed flows, a bug will be logged in Azure DevOps.

Note: The script contains no error handling or logging mechanisms. It is meant to only demonstrate the core functionality of integrating LEAPWORK with Azure DevOps Server.

After running, cases are created in Azure DevOps Server and can be managed, for instance, through Visual Studio:

tfs 3

The previous script uses PATs to authorize access to Azure DevOps Server. For more information about how to setup PATs and how to perform Azure DevOps REST calls, see the following links:

https://www.visualstudio.com/en-us/docs/setup-admin/team-services/use-personal-access-tokens-to-authenticate

https://www.visualstudio.com/en-us/docs/integrate/get-started/rest/basics

If we have the Controller installed on our computer, explore the previously mentioned endpoints by going to the following URL: http://localhost:9001/help/index.

Read the full documentation on the Leapwork REST API.

If you have any questions, contact priority support on prioritysupport@leapwork.com.

 

 

Related articles

Comments

0 comments

Please sign in to leave a comment.