Hello,
I'm creating a template on an eCommerce site which has 100000s of products listed. I want to scrape all the data. It's built using AJAX where when a user scrolls down, it will add new results/products to the page and would still retain the old products at the top of the page. So, if I scroll till the end, I will have 10000s of products in a single page. I want to scrape all the data. I recorded and modified a template which works like a charm. But the issue I am facing is on the scrolling part.
I used keyword simulation with down key + page down + end key random combinations. But nothing worked.
then I tried to click on an empty space using search by image and then used keyword simulation just like above, but it still didn't work.
Then in debugger mode I tested it and found that irrespective of the fact whether we click on some empty area and then press page down, down or end keys, it won't scroll. It will only scroll if I press the scroll bar once and then if I use page down, down or end keys those start working fine.
So, the conclusion is that I need to click on the scroll bar at least once in order to scroll through the page. I'm really not sure how to achieve this. I want to click on the scroll bar once.
P.S. - Even while recording the template, or debugging it, the page scrolling works in the similar fashion. i.e first click on the scroll bar and then use the down keys or keep on scrolling.
Any help?
Thanks
I'm creating a template on an eCommerce site which has 100000s of products listed. I want to scrape all the data. It's built using AJAX where when a user scrolls down, it will add new results/products to the page and would still retain the old products at the top of the page. So, if I scroll till the end, I will have 10000s of products in a single page. I want to scrape all the data. I recorded and modified a template which works like a charm. But the issue I am facing is on the scrolling part.
I used keyword simulation with down key + page down + end key random combinations. But nothing worked.
then I tried to click on an empty space using search by image and then used keyword simulation just like above, but it still didn't work.
Then in debugger mode I tested it and found that irrespective of the fact whether we click on some empty area and then press page down, down or end keys, it won't scroll. It will only scroll if I press the scroll bar once and then if I use page down, down or end keys those start working fine.
So, the conclusion is that I need to click on the scroll bar at least once in order to scroll through the page. I'm really not sure how to achieve this. I want to click on the scroll bar once.
P.S. - Even while recording the template, or debugging it, the page scrolling works in the similar fashion. i.e first click on the scroll bar and then use the down keys or keep on scrolling.
Any help?
Thanks
