How to Simulate Text Selection : Get the ChatGPT answer

[Pierre Paul Jacques]

Hello ZennoPoster Community and Happy Sunday!

Im currently working on a project where I need to programmatically select and copy the text of specific responses generated by an ChatGPT in a browser environment. Each response is distinct, yet I aim to consistently select the text block that contains the AI's reply to a query.
( as you can see in the screenshot)

My challenge is to devise a method to pinpoint the start of the ChatGPT response and simulate a mouse click and drag to highlight the entire block of text, irrespective of the content length or structure :

I'm exploring strategies to programmatically determine the XY coordinates of the starting point of the desired text within the browser. The goal is to store these coordinates in a variable, and similarly for the endpoint of the response text. With accurate start and end coordinates, we could then instruct the ZennoPoster's mouse simulation features to perform precise actions, effectively highlighting the text block containing ChatGPT's response.

This approach would require a reliable method for identifying the XY coordinates that correspond to the specific text block's boundaries on the page. I'm considering image recognition or pattern matching techniques as possible solutions. However, I am open to suggestions from the community on how to effectively implement this or if there are alternative methods to achieve the same outcome.

I look forward to any advice or shared experiences regarding mouse simulation based on dynamic text selection and thank you in advance for your contributions to solving this challenge.


So far, I've attempted various C# and JavaScript approaches within ZennoPoster's integrated browser, but I've encountered obstacles, particularly with dynamically locating the start of the response text for selection.

I would greatly appreciate any insights or strategies on how to achieve this, especially in a way that adapts to the variable lengths and content of the responses.

Thank you for your time and assistance!

 

[Ахилес]

take the entire DIV block as innerhtml and parse the necessary data from it.

 

[myndeswx]

Hello ZennoPoster Community and Happy Sunday!

Im currently working on a project where I need to programmatically select and copy the text of specific responses generated by an ChatGPT in a browser environment. Each response is distinct, yet I aim to consistently select the text block that contains the AI's reply to a query.
( as you can see in the screenshot)

My challenge is to devise a method to pinpoint the start of the ChatGPT response and simulate a mouse click and drag to highlight the entire block of text, irrespective of the content length or structure :

I'm exploring strategies to programmatically determine the XY coordinates of the starting point of the desired text within the browser. The goal is to store these coordinates in a variable, and similarly for the endpoint of the response text. With accurate start and end coordinates, we could then instruct the ZennoPoster's mouse simulation features to perform precise actions, effectively highlighting the text block containing ChatGPT's response.

This approach would require a reliable method for identifying the XY coordinates that correspond to the specific text block's boundaries on the page. I'm considering image recognition or pattern matching techniques as possible solutions. However, I am open to suggestions from the community on how to effectively implement this or if there are alternative methods to achieve the same outcome.

I look forward to any advice or shared experiences regarding mouse simulation based on dynamic text selection and thank you in advance for your contributions to solving this challenge.


So far, I've attempted various C# and JavaScript approaches within ZennoPoster's integrated browser, but I've encountered obstacles, particularly with dynamically locating the start of the response text for selection.

I would greatly appreciate any insights or strategies on how to achieve this, especially in a way that adapts to the variable lengths and content of the responses.

Thank you for your time and assistance!



Посмотреть вложение 115530

What is the reason for not just using the API?

 

[Pierre Paul Jacques]

Thank for Both for your reply!

I already made a bloc to extract in html the content, but what i want in this case is to select the text from the "front end"
to get the "generate html"( i am beginner so i am not sure about the term),
because after i need to copy it straight into a site ( not mine) where i CANNOT write in html and got also his own rule for markdown...
So for the moment the best way to copy a text in this site and still get the title, bold, link... i need to copy it straight from GPT result without any transformation.

 

[myndeswx]

Goot it, well just FIY you can ask the for markdown in the API response, and it does a pretty good job, same for HTML responses

 

[Pierre Paul Jacques]

It is what i did first,
but in the case of website where i want to post the if i copy from Markdown it is not working :
like only 50% of the Markdown rules the website got his "own rules"
( and HTML is not allowed)

I can send you my ZP project to show you,
but is look like is not possible to send private message or i didn't find it,)

 

[Pierre Paul Jacques]

I can send you my ZP project to show you,
but is look like is not possible to send private message or i didn't find it,)

 

[myndeswx]

Sounds like dealing with medium.. had similar headaches with it recently lol )
Well, here's what I came up with, give it a try, it's not tested on chatgpt, but I think it should give you a rough idea of what is possible, also be warned that clipboard is shared between threads!!!

    Tab tab = instance.MainTab;

           //finding the element here             
    HtmlElement he = tab.MainDocument.FindElementByAttribute("h1", "class", "h1-main", "regexp", 0);
    if (he.IsVoid) return -1;

    //finding coordinates where it starts
    Point point = he.DisplacementInBrowser;

    // guessing where it ends based on the begining coords and elements width and heigth
    int endOfX = he.DisplacementInBrowser.X + he.Width;
    int endOfY = he.DisplacementInBrowser.Y + he.Height;


    project.SendErrorToLog(point.ToString());
instance.ActiveTab.MouseClick(he.DisplacementInBrowser.X,he.DisplacementInBrowser.Y,"left","down",true);
    instance.ActiveTab.MouseMove(he.DisplacementInBrowser.X,he.DisplacementInBrowser.Y,endOfX,endOfY,false,false);
    instance.ActiveTab.KeyEvent("c", "press", "ctrl");

 

[Pierre Paul Jacques]

Sounds like dealing with medium.. had similar headaches with it recently lol )
Well, here's what I came up with, give it a try, it's not tested on chatgpt, but I think it should give you a rough idea of what is possible, also be warned that clipboard is shared between threads!!!

    Tab tab = instance.MainTab;

           //finding the element here           
    HtmlElement he = tab.MainDocument.FindElementByAttribute("h1", "class", "h1-main", "regexp", 0);
    if (he.IsVoid) return -1;

    //finding coordinates where it starts
    Point point = he.DisplacementInBrowser;

    // guessing where it ends based on the begining coords and elements width and heigth
    int endOfX = he.DisplacementInBrowser.X + he.Width;
    int endOfY = he.DisplacementInBrowser.Y + he.Height;


    project.SendErrorToLog(point.ToString());
instance.ActiveTab.MouseClick(he.DisplacementInBrowser.X,he.DisplacementInBrowser.Y,"left","down",true);
    instance.ActiveTab.MouseMove(he.DisplacementInBrowser.X,he.DisplacementInBrowser.Y,endOfX,endOfY,false,false);
    instance.ActiveTab.KeyEvent("c", "press", "ctrl");

Thank for you code i will check this asap it's look great,
this is not for medium but for another site like you are right,)