I get inconsistent results with image recognition

On a very fundamental level, image recognition in Leapwork will always find — pixel for pixel — something that was captured if it appears on the screen.

However, there are many ways in which the image shown on the screen can differ (sometimes ever so slightly) from what was captured. The following are a few common examples to be aware of.

When capturing icons:

• The background color can change, so don’t include parts of the background when capturing
• A “hover” effect can change how the icon looks when hovered by a mouse pointer, for instance showing a brighter or darker version. This can usually be handled by closing all open windows as part of the test run.
• A “selected” or “opened” effect can change how the icon looks when selected. For instance, the browser icons in the Windows taskbar look different when a browser is open as opposed to closed. It can be solved by creating an image collection.

When searching for images on web pages

• When loading some web pages, images or parts of images can be shown before the rendering is all done in the browser. This means the images can be moved around on the screen until the web page “settles”. Usually checking the “Await no movement” property on the building block solves this problem. This will tell the image recognition engine to wait until the screen has not changed for a period of time before starting to search for the image.
• Set the area where the image should be searched for. It will speed up and facilitate recognition. Please note that the area should be larger than the picture or text. You can also adjust the desired precision for picture recognition and turn the dirty edge mode for text recognition.

It works on my own computer but not on another computer or remote environment

• In order for it to work across machines, the conditions need to be the same. If the 2 machines run with different resolution (dpi) the “same” image will be a bit different. Installed programs will actually serve different images depending on the resolution, which ruins the image recognition.
• Two things to be aware of:
  • When you design your test, remember to capture the images from the machine where the test should run (set up an environment for the machines where the test cases should run). This means you are accessing completely the same environment while designing and while running the test case.
  • Consider the state of the machine, where the test case runs. If you run the same test twice, make sure that the first test case doesn’t change the conditions for the second one – like selecting an icon, opening a couple of programs, etc.

For any clarification, please contact our Priority Support.