More pinterst 'big page' issues.

[Harambulus]

The problem for me is not the dom thing, i don't think it is anyhow.

The problem is that I am continually scrolling down and it is eating up my system memory by the second.

I only have 3k followers at the moment and already Im runing out of resources before i get to the bottom of the page.

The problem is I cant do it in chunks I have to get the full page text of followers so that I can regex the list of people Im following against it to decide whether to unfollow them if they arent following back.

This stupid web 2.0 'feature' of one infiinite page is really holding me back.

I dont see how you can get around it, since as I said I need that whole page of usernames to be complete so I can regex it and the more followers I get the larger its going to get day by day. Any ideas?

With facebook they are kind enough to allow you to export all your friends in a text file but doubt there is anything like that for pinterest

 

[shabbysquire]

I know what you mean re infinite scrolling.

When I scraped another site, I had the same issue but found other sources of data, i.e. JSON. This was much easier to work with.

 

[MatthewSher]

Agreed with shabby

 

[rostonix]

Try the same with standard browser. And see how much resources will it use when you do this infinite scroll.

 

[LexxWork]

If you can't parse data in low level, wait for my fiddler article with multithreading g+ group parser example or just send pm.

 

[Harambulus]

How do you go about finding JSON data?

 

[Harambulus]

@rostonix how do you mena standard browser? I already have it on mobile version and smallest reoslution and didnt make much difference in terms of size.