How to grab information from one website - Scrapping

[VincentS]

Hello all,

I am trying to do a template in order to grab informations on one website. To get the information from product pages it's quite simple (thanks Zennoposter !), I did a template for that, but now my issue is with the crawler ...

Do I need another template to get all Urls product pages or integrate to the parser template. In multithreading I guess that the best way is another template that write the URLs of product pages into a txt file (but how to navigate through category pages ?), and the other read the URLs and parse them (and remove the URL once it's done).

According to your experience, what is your best practize for that ?

Thank you for your help

Best regards,

Vincent

 

[VincentS]

Nobody to point me in the right direction ?

Cheers,

Vincent

 

[copper12]

Hello,
Parsing can be performed using regular expressions
Using regular expressions
For saving/removing urls in lists you can use operations with list
List processing