Improving Object Recognition by using UI Automation Technology
The UI Automation Technology has been developed by Microsoft to provide a unified object model that enables all user interface (UI) frameworks to expose complex and rich functionality in an accessible and easily automated manner.
UI Automation is designed to provide robust functionality for automated testing, it is an accessibility framework that enables Microsoft Windows applications to provide and consume programmatic information about user interfaces (UIs). It provides programmatic access to most UI elements on the desktop.
WinTask x64 supports the UI Automation technology and can use UI Automation to obtain information on UI elements of the application to automatize. Properties of buttons, check boxes, menus and menu items, data grids, list boxes and other controls can be retrieved using UI Automation technology. The UI Automation technology works only on applications which have implemented the UI Automation framework. Obviously, Microsoft applications such as Office 2010 or 2013 and WPF applications do support UI Automation.
WinTask x64 addresses any desktop element exposed to UI Automation framework by its UIA descriptor. The UIA descriptor identifies uniquely one element displayed on desktop. The UIA descriptor is not generated automatically by Recording mode and you need to identify manually a UI element using Spy tool (if Spy tool finds that the object is addresses by UI Automation framework). Then you can use the pasted UIA descriptor as parameter of any WinTask x64 UIA functions.
In this version, three functions are available:
ClickUIA, clicks the object specified by its UIA descriptor
WriteUIA, writes a text into the field specified by its UIA descriptor
GetUIAProperty, retrieves the content of the specified property for the object specified by its UIA descriptor.
"[APP='iexplore.exe',CLASS='DirectUIHWND',AUTOID='IENotificationBar']" 'In Internet Explorer, many elements outside the page itself belongs to the window whose CLASS is DirectUIHWND and so AutoID tells that it belongs to the small notification bar at the bottom.
"[APP='Application.exe',CLASS='WindowsForms10.Window*',AUTOID='dataGridView1'] 'In a WinForms application, the spyeded element belongs to a grid whose AUTOID property value is dataGridView1
"[APP='Application.exe',CLASS='WindowsForms10.BUTTON*',TITLE='Delete']" 'In a WinForms application, there is a Delete button, this button is in itself a top window with the Delete TITLE word
Note that Spy tool in its Window Information part shows the APP, CLASS and TITLE parameters.
The other nodes are specified using some of their UI Automation properties, for example:
"[APP='iexplore.exe',CLASS='DirectUIHWND',AUTOID='IENotificationBar'][NAME='Save']" 'Button named Save in the IE Dowload window (class DirectUIHWND)
"[APP='Application.exe',CLASS='WindowsForms10.Window*',AUTOID='dataGridView1'][NAME='Gender Row 1'] 'Cell in the row 1 for the column called Gender belonging to a grid in a WinForms application
Always build the UIA descriptor of an element using WinTask x64 Spy tool.