Is it possible to use AI to understand a screenshot of a webpage - and return click coordinates?

zmike

Client
Регистрация
24.10.2019
Сообщения
137
Благодарностей
8
Баллы
18
When a page is dynamic or don't have fixed location for certain buttons or checkbox.

Is it possible to use Zenno Poster to take a full screenshot, then let AI to understand the screenshot of the webpage, by calling the AI using API., then let AI to return click coordinates?

Finally let Zenno Poster to click that x, y coordinates?
 

lokiys

Moderator
Регистрация
01.02.2012
Сообщения
4 858
Благодарностей
1 193
Баллы
113
I dont see why not, but you probably have to test how your chosen AI gives you the coordinates, and if they are right and match with zenno.
But I'm sure there are much easier ways to click where you want.

1. You don't need any fixed coordinates by default as you search h element by its parameters, which it tag, id etc. Whatever where element is.
2. If for any reason you don't want to do that, you can still find element by those same parameters and get x and y coordinate points. Add topInBrowser in value for y and leftInBrowser for x coordinate point and then, using those coordinates, you can click where you want.




130310



3. If you again for some reason can't do any of this, you can try using Search by picture feature in zenno.


130309
 

Alex91

Активный пользователь
Регистрация
15.08.2024
Сообщения
760
Благодарностей
210
Баллы
43
When a page is dynamic or don't have fixed location for certain buttons or checkbox.

Is it possible to use Zenno Poster to take a full screenshot, then let AI to understand the screenshot of the webpage, by calling the AI using API., then let AI to return click coordinates?

Finally let Zenno Poster to click that x, y coordinates?
 

VladZen

Administrator
Команда форума
Регистрация
05.11.2014
Сообщения
22 572
Благодарностей
5 951
Баллы
113
You should use Search by picture option to click buttons or elements with variable position and attributes.
 

zmike

Client
Регистрация
24.10.2019
Сообщения
137
Благодарностей
8
Баллы
18
Thanks for your reply.
 

Кто просматривает тему: (Всего: 1, Пользователи: 0, Гости: 1)