Scraper Video Tutorials

[seo101]

Hi,

Can we have video tutorials on scraping ? Perhaps, you can use google as an example. Also, is it possible if google show the anti-bot (http://www.google.com/sorry/) we can type in the captcha so we can continue scraping using the same session.

Thanks

 

[LightWood]

Scarping is easy. Use the source code and regular expressions.
And Google has the usual recapture. Look for lessons and reviews.

 

[Perropoly]

I'm working in a little scraper for Gamespot forums with Zennoposter, the logic seems ok, the regexp is working fine in the regular expression builder, but when I debug the template it only scrape the first title on every page.

The regexp is as follows:

(?<=\<TD class\=topic\>\<A class\=\".*\" href\=\"http:\/\/www\.gamespot\.com\/pages\/forums\/show_msgs\.php\?topic_id\=\d+&amp;tag\=topics%3Btitle\"\>).*(?=\<\/A\> \<SPAN)

 

[Perropoly]

A friend suggest to add "-|-all-" and it works fine now.

 

[dongle132]

May I ask where you've placed the "-|-all-" ?!
I have the same problem and tried it in the parameters field with the regex but it does not work.
THX

 

[LightWood]

reg.exp-|-all-
search in dom or source

 

[dongle132]

Sorry but it does not work for me:

I've tried:

(?<=\>\<A href\=\").*(?=\" target\=_blank\>http)-|-all-
(?<=\>\<A href\=\").*(?=\" target\=_blank\>http-|-all-)

Maybe you can have a look at the example template I've created to show me where to set the -|-all- correct?!
Thanks in advance.

Посмотреть вложение Get_ALL_DOM_Text.xml

 

[step85]

Sorry but it does not work for me:

I've tried:

(?<=\>\<A href\=\").*(?=\" target\=_blank\>http)-|-all-
(?<=\>\<A href\=\").*(?=\" target\=_blank\>http-|-all-)

Maybe you can have a look at the example template I've created to show me where to set the -|-all- correct?!
Thanks in advance.

After regex
(?<=\>\<A href\=\").*(?=\" target\=_blank\>http)-|-all

 

[dongle132]

oh lord ... one simple "-" to much

MY (WRONG): (?<=\>\<A href\=\").*(?=\" target\=_blank\>http)-|-all-
YOU (CORRE): (?<=\>\<A href\=\").*(?=\" target\=_blank\>http)-|-all

THANKS!!

 

[seo101]

Thanks guys for sharing this. At least I have something to start with now.

 

[dongle132]

If you can master zenno and know the GUI take a look at this video here:

http://www.youtube.com/watch?v=hGjn-iWhkMY

He is scraping keywords from Amazon.com

 

[jp1]

After regex
(?<=\>\<A href\=\").*(?=\" target\=_blank\>http)-|-all

Is this NOT regex? If so what is it?

Or is it the regex that goes in the 'This goes after the required text''?

 

[step85]

Is this NOT regex? If so what is it?

Or is it the regex that goes in the 'This goes after the required text''?

This is regex, after regex add -|-all