UiPath Computer Vision: Activities, License, API Explained

In this tutorial, you will be able to learn UiPath Computer Vision activities. It is also known as CV activities. This activity can be used whether you need to extract data from specific scanned documents, automate repetitive tasks involving UI elements, or even navigate complex visual workflows, UiPath Computer Vision Activities has got you covered.

By using this activity you can improve your workflow much better than earlier. Here we will provide a complete tutorial on this activity. So let’s go through it.

What is UiPath Computer Vision

The Computer Vision of UiPath is an effective tool that enables users to interact with UI components automatically while simulating human interaction by using neural networks. This feature consists of a number of UIAutomation activities that are included in the package.

Refactored versions of popular UIAutomation actions like Click, Type Into, and Get Text is included in the Computer Vision bundle. The usage of UiPath’s own neural network, which can recognize UI (User Interface) components like text input fields, buttons, or checkboxes without the need for selectors, makes these tasks distinct from their conventional equivalents.

Smart anchors are used to identifying the correct location of the UI (User Interface) element being interacted with in order to guarantee that the desired action is carried out effectively. Overall, the Computer Vision component of UiPath adds value to any automation process by increasing the accuracy and productivity of repeated operations involving UI elements.

Different Types of UiPath Computer Vision Activities

Now you have some basic idea of UiPath computer vision activities, now let’s check the different activities included in it. These are also available in UiPath Studio X.

• CV Get Text
• CV Refresh
• CV Screen Scope
• CV Click
• CV Dropdown Select
• CV Highlight
• CV Type Into
• CV Extract Table
• CV Hover
• CV Element Exists
• CV Check

Here CV stands for Computer Vision. Now let’s check them in detail.

CV Check

UiPath CV Check activity is used to check or uncheck the check box on a website or application. While working on web automation you may be required to toggle on some check boxes so, in that case, you can use this activity.

UiPath.CV.Activities.CvCheckWithDescriptor

CV Click

This activity is used to precisely click on the intended UI element. By utilizing cutting-edge image recognition technology, CV Click is able to interact with intricate, dynamic, or difficult-to-find UI components with better accuracy and efficiency.

The package used in this activity is:

UiPath.CV.Activities.CVClickWithDescriptor

CV Dropdown Select

UiPath CV dropdown select activity is used to select a particular option in a drop-down menu.

UiPath.CV.Activities.CvSelectItemWithDescriptor

CV Element Exists

The CV Element Exists activity is used to search for a particular UI (User Interface) element on the screen in the background with help of a Computer Vision neural network. This activity returns a particular Boolean variable that indicates whether the specified UI element is exist or not.

This activity makes it possible to rapidly and precisely verify whether a given UI element is visible on the screen, which is very helpful for automating repeated operations that call for the identification of particular items.

UiPath.CV.Activities.CVElementExistsWithDescriptor

CV Extract Table

The CV Extract Table activity is used to capture a table that is visible on the computer screen and save its information into a DataTable variable. This information can then be used for further processing or analysis within a workflow.

UiPath.CV.Activities.CvExtractDataTableWithDescriptor

CV Get Text

Users can extract text from a chosen UI element with the help of “CV Get Text“. Users who use this activity can choose their preferred technique for getting the target text from a drop-down menu.

For automating the extraction of text data from UI components like scanned documents or web pages, CV Get Text provides a quick and effective solution.

UiPath.CV.Activities.CVGetTextWithDescriptor

CV Highlight

You may use this activity to visually draw attention to a particular UI component inside your automation workflows.

After the element has been recognized, the activity will highlight it in a way that makes it visually stand out, making it simple for you to validate that the relevant element has been targeted. This activity can help you When you need to verify whether the selected UI Element is the right one or not.

UiPath.CV.Activities.CVHighlightWithDescriptor

CV Hover

UiPath Computer Vision Hover activity allows you to interact with a targeted UI element by hovering over it.

UiPath.CV.Activities.CvHoverWithDescriptor

CV Type Into

CV Type into activity is used to type any specific text or values into a text box or window. These activities are mostly used in data entry jobs where the data form excel has to be read and add it to the data entry form.

UiPath.CV.Activities.CVTypeIntoWithDescriptor

CV Refresh

A CV refresh refers to refreshing the scope of an application or a window. During the design process, you may manually accomplish this by using the Refresh button in the activity’s body.

The system checks previously-stored screens using an algorithm to determine whether changes have been made, and it only updates the server with new data in those cases.

UiPath.CV.Activities.CVRefresh

CV Screen Scope

UiPath CV Screen Scope is an essential action that launches the neural network for UiPath Computer Vision and performs a complete examination of the designated window.

The flexibility to choose which OCR engine to use while extracting text from the target application is one of the main advantages of this activity.

Users have the option to manually change the OCR engine that is used by default from UiPath Screen to one of their choosing.

Due to this flexibility, customers are able to customize the accuracy and productivity of their OCR-based automation processes and select the OCR engine that best meets their demands.

The package used in this activity is :

UiPath.CV.Activities.CVScope

What is Computer Vision Recorder

UiPath Computer Vision Recorder is a tool that is available in the Computer Vision activities package. It is used to create workflows that make advantage of the capabilities of the CV activities automatically.

Click the Record button, then choose the program you wish to automate, to begin recording. After an application has been chosen, the UiPath Computer vision server is notified, and the recorder switches to recording mode. Apart from the window that is now selected, the entire screen is greyed out.

You may choose a text field and enter text or hotkeys in it by using the Type button on the new recorder interface. The Get button takes text from a designated location, whereas the Click-related button allows you to select any UI element. The Save & Quit button ends the recording and creates the steps necessary to carry out the recorded actions.

The wizard is not closed when you use the “Stop” Recording button to cease recording operations. When the target window or application changes and a new CV server request has to be issued, you can utilize the Refresh button. If you wish to switch to automate another application, you may choose another application or window using the Change Application button.

UiPath Computer Vision License or API Key

To use a CV in your UiPath Studio, UiPath Computer Vision API Key is required else when you try to access it may show some errors. The license key is free for community users but has some specific limit. To enable the license to follow the below steps:

1. Go to cloud.uipath.com on your web browser and log in with your credentials.
2. Go to the “Admin” section.
3. In the admin section click on “License“.
4. Now click on “Robots & Services” and scroll to the bottom
5. Now copy the API Key.

6. Now open your uipath studio and go to project settings.

7. Now on project settings, click on “computer vision” and scroll to the bottom. Now paste the copied API key in it and click the “Ok” button to save your settings.

The Specific Computer Vision Server Could Not Be Reached

Sometimes while using computer vision you may face these errors. I also face these errors the first time when I used it. The main reason for it is an improper internet connection. When this activity tries to find it due to poor network it is unable to find it.

Conclusion

Till now we have discussed different varieties of Computer vision activities and recorders that can be used in your uipath studio for various automation processes. Additionally, we have also included the hope you can add the API key to your computer vision project settings.

Faq on UiPath Computer Vision