UiPath Computer Vision: Activities, License, API Explained

In this tutorial, you will be able to learn UiPath Computer Vision activities. It is also known as CV activities. This activity can be used whether you need to extract data from specific scanned documents, automate repetitive tasks involving UI elements, or even navigate complex visual workflows, UiPath Computer Vision Activities has got you covered.

By using this activity you can improve your workflow much better than earlier. Here we will provide a complete tutorial on this activity. So let’s go through it.

What is UiPath Computer Vision

The Computer Vision of UiPath is an effective tool that enables users to interact with UI components automatically while simulating human interaction by using neural networks. This feature consists of a number of UIAutomation activities that are included in the package.

Computer Vision Activities
Computer Vision Activities

Refactored versions of popular UIAutomation actions like Click, Type Into, and Get Text is included in the Computer Vision bundle. The usage of UiPath’s own neural network, which can recognize UI (User Interface) components like text input fields, buttons, or checkboxes without the need for selectors, makes these tasks distinct from their conventional equivalents.

Smart anchors are used to identifying the correct location of the UI (User Interface) element being interacted with in order to guarantee that the desired action is carried out effectively. Overall, the Computer Vision component of UiPath adds value to any automation process by increasing the accuracy and productivity of repeated operations involving UI elements.

Different Types of UiPath Computer Vision Activities

Now you have some basic idea of UiPath computer vision activities, now let’s check the different activities included in it. These are also available in UiPath Studio X.

  • CV Get Text
  • CV Refresh
  • CV Screen Scope
  • CV Click
  • CV Dropdown Select
  • CV Highlight
  • CV Type Into
  • CV Extract Table
  • CV Hover
  • CV Element Exists
  • CV Check

Here CV stands for Computer Vision. Now let’s check them in detail.

CV Check

UiPath CV Check activity is used to check or uncheck the check box on a website or application. While working on web automation you may be required to toggle on some check boxes so, in that case, you can use this activity.

CV Check Activity
CV Check Activity
UiPath.CV.Activities.CvCheckWithDescriptor

CV Click

This activity is used to precisely click on the intended UI element. By utilizing cutting-edge image recognition technology, CV Click is able to interact with intricate, dynamic, or difficult-to-find UI components with better accuracy and efficiency.

CV Click Activity
CV Click Activity

The package used in this activity is:

UiPath.CV.Activities.CVClickWithDescriptor

CV Dropdown Select

UiPath CV dropdown select activity is used to select a particular option in a drop-down menu.

CV Dropdown Select
CV Dropdown Select
UiPath.CV.Activities.CvSelectItemWithDescriptor

CV Element Exists

The CV Element Exists activity is used to search for a particular UI (User Interface) element on the screen in the background with help of a Computer Vision neural network. This activity returns a particular Boolean variable that indicates whether the specified UI element is exist or not.

CV Element Exist
CV Element Exist

This activity makes it possible to rapidly and precisely verify whether a given UI element is visible on the screen, which is very helpful for automating repeated operations that call for the identification of particular items.

UiPath.CV.Activities.CVElementExistsWithDescriptor

CV Extract Table

The CV Extract Table activity is used to capture a table that is visible on the computer screen and save its information into a DataTable variable. This information can then be used for further processing or analysis within a workflow.

CV Extract Table Activity
CV Extract Table Activity
UiPath.CV.Activities.CvExtractDataTableWithDescriptor

CV Get Text

Users can extract text from a chosen UI element with the help of “CV Get Text“. Users who use this activity can choose their preferred technique for getting the target text from a drop-down menu.

CV Get Text Activity
CV Get Text Activity

For automating the extraction of text data from UI components like scanned documents or web pages, CV Get Text provides a quick and effective solution.

UiPath.CV.Activities.CVGetTextWithDescriptor

CV Highlight

You may use this activity to visually draw attention to a particular UI component inside your automation workflows.

CV Highlight Activity
CV Highlight Activity

After the element has been recognized, the activity will highlight it in a way that makes it visually stand out, making it simple for you to validate that the relevant element has been targeted. This activity can help you When you need to verify whether the selected UI Element is the right one or not.

UiPath.CV.Activities.CVHighlightWithDescriptor

CV Hover

UiPath Computer Vision Hover activity allows you to interact with a targeted UI element by hovering over it.

CV Hover Activity
CV Hover Activity
UiPath.CV.Activities.CvHoverWithDescriptor

CV Type Into

CV Type into activity is used to type any specific text or values into a text box or window. These activities are mostly used in data entry jobs where the data form excel has to be read and add it to the data entry form.

UiPath CV Type Into Activity
UiPath CV Type Into Activity
UiPath.CV.Activities.CVTypeIntoWithDescriptor

CV Refresh

A CV refresh refers to refreshing the scope of an application or a window. During the design process, you may manually accomplish this by using the Refresh button in the activity’s body.

CV Refresh Activity
CV Refresh Activity

The system checks previously-stored screens using an algorithm to determine whether changes have been made, and it only updates the server with new data in those cases.

UiPath.CV.Activities.CVRefresh

CV Screen Scope

UiPath CV Screen Scope is an essential action that launches the neural network for UiPath Computer Vision and performs a complete examination of the designated window.

The flexibility to choose which OCR engine to use while extracting text from the target application is one of the main advantages of this activity.

CV Screen Scope Activity
CV Screen Scope Activity

Users have the option to manually change the OCR engine that is used by default from UiPath Screen to one of their choosing.

Due to this flexibility, customers are able to customize the accuracy and productivity of their OCR-based automation processes and select the OCR engine that best meets their demands.

The package used in this activity is :

UiPath.CV.Activities.CVScope

What is Computer Vision Recorder

UiPath Computer Vision Recorder is a tool that is available in the Computer Vision activities package. It is used to create workflows that make advantage of the capabilities of the CV activities automatically.

Computer Vision Recorder
Computer Vision Recorder

Click the Record button, then choose the program you wish to automate, to begin recording. After an application has been chosen, the UiPath Computer vision server is notified, and the recorder switches to recording mode. Apart from the window that is now selected, the entire screen is greyed out.

You may choose a text field and enter text or hotkeys in it by using the Type button on the new recorder interface. The Get button takes text from a designated location, whereas the Click-related button allows you to select any UI element. The Save & Quit button ends the recording and creates the steps necessary to carry out the recorded actions.

The wizard is not closed when you use the “Stop” Recording button to cease recording operations. When the target window or application changes and a new CV server request has to be issued, you can utilize the Refresh button. If you wish to switch to automate another application, you may choose another application or window using the Change Application button.

UiPath Computer Vision License or API Key

To use a CV in your UiPath Studio, UiPath Computer Vision API Key is required else when you try to access it may show some errors. The license key is free for community users but has some specific limit. To enable the license to follow the below steps:

  1. Go to cloud.uipath.com on your web browser and log in with your credentials.
  2. Go to the “Admin” section.
  3. In the admin section click on “License“.
  4. Now click on “Robots & Services” and scroll to the bottom
  5. Now copy the API Key.
Computer Vision License Key
Computer Vision License Key
  1. Now open your uipath studio and go to project settings.
UiPath Studio Project Settings
UiPath Studio Project Settings
  1. Now on project settings, click on “computer vision” and scroll to the bottom. Now paste the copied API key in it and click the “Ok” button to save your settings.
Computer Vision API Key
Computer Vision API Key

The Specific Computer Vision Server Could Not Be Reached

Sometimes while using computer vision you may face these errors. I also face these errors the first time when I used it. The main reason for it is an improper internet connection. When this activity tries to find it due to poor network it is unable to find it.

The Specific Computer Vision Server Could Not Be Reached
Computer Vision Server Could not be reached

Conclusion

Till now we have discussed different varieties of Computer vision activities and recorders that can be used in your uipath studio for various automation processes. Additionally, we have also included the hope you can add the API key to your computer vision project settings.

Faq on UiPath Computer Vision

  1. What is UiPath computer vision?

    The Computer Vision of UiPath is an effective tool that enables users to interact with UI components automatically while simulating human interaction by using neural networks.

  2. Is UiPath computer vision free?

    Yes, UiPath Computer Vision is free for the community version but has certain limits.

  3. Is computer vision an AI?

    Computer Vision is part of an AI that helps to extract or capture data from images and video data.

Leave a Reply

Your email address will not be published. Required fields are marked *