SERP scrapper memory problems

  • Автор темы Автор темы zenrast
  • Дата начала Дата начала

zenrast

Client
Регистрация
20.03.2013
Сообщения
57
Реакции
1
Баллы
8
Hi,

i have a Google SERP scrapper that uses keyboard emulation to write the keyword into the "search box" on google.com.

I have a loop that gets the results and continues with the following keywords. All working perfectly.

My problem is that after a couple of hours of work the memory consumption for the "FireFox instance" gets really high (up to 2GB). And at that time i also get high CPU usage and problems beggin.

Is there a way to "reset/clear" the memory that is assigned to "FireFox instance"?
After all i only enter this KW in search box and get the results. Not sure why is the memory consumption so high after some time.
 
You should restart instancet regularly in your project.
Restart_instance.png
 
What does "restarting the instance" mean?
Does this restart all running threads of the instance?
Are cookies of threads automatically saved so i dont need to set the Google settings again?
 
thread and instance is the same.
Base can have several instances. If u set it in settings.

What does "restarting the instance" mean?
Does this restart all running threads of the instance?
Are cookies of threads automatically saved so i dont need to set the Google settings again?
You are right.

Usually I just finish an instance after some loops.
I don't like endless loop.
 
Hi,

One of most common newbie mistakes, endless loops in the template. :)
zenrast If You loop in Your templates like 100+ times then there are no wonders resources will be high. Try searching 100+ times speedy in firefox, it will be dead I guess :D

My suggestion is to save and load cookies, this way You will not lose settings and You can freely allow ZennoPoster to take care of repeats/loops You need. :)
 
  • Оценить
Реакции: zenfreak
Look for instance.Reload(); if I'm not mistaken.

But real restart of instance happens when You develop templates in a way they starts and ends without endless loops. To make same actions in loop You have to use zennoposter and executions.
 
  • Оценить
Реакции: zenfreak

Кто просматривает тему: (Всего: 0, Пользователи: 0, Гости: 0)