problem scraping data

[bill]

so i'm having a problem that doesn't even make sense. it was working fine until today when it randomly stopped working.

I'm scraping data on a website and putting it to a file. but when i run multithreads, it randomly stops outputting the data to the file. It works fine on a single thread, but on multithread for no apparent reason it now just stops scraping data.

I have added pauses to the file it binds to and before running the TAKE DOM action and STILL it stops outputting data. it really makes no sense and i'm wondering if there is something wrong with ZP.

please advise on whats the best next step to take

 

[bill]

ok too add to the mystery,

multithread works WITHOUT task manager, but i run the job from task manager... it stops scraping.


i need to use task manager so i need to fix this

it doesn't even make sense, it scrapes fine at first for a couple times then it just stops working!

 

[bill]

its still doing this and it doesn't make ANY sense.... i need some answers

 

[rostonix]

how do you work with data? what actions do you use? List of File processing?

 

[bill]

take dom text using regex, add to list... list is binded to a file

it worked fine until yesterday when for no clear reason it stopped working in task manager, yet still works outside of the task manager.

 

[bill]

this problem is still occuring since the update. There is no good reason i can see why it works out of task manager but not in task manager.

 

[bill]

by the way, it worked for a couple minutes then stopped working.

 

[lokiys]

Tell me please why you use task manager ? What is your requirements to run scrappers ?

Also please send your template with Diagnostic.exe to http://support.zennolab.com with reference to this thread...

Cheers

 

[lokiys]

i haven't seen any big problems yet either besides a bug i'm having with task manager.

i haven't seen any big problems yet either besides a bug i'm having with task manager.

Can not tell anything about task manager, I'm not using it at all. You can make much better task scheduling by your own templates...

not quite sure i understand how you do task scheduling in my own templates, can you explain? is there some action?

not quite sure i understand how you do task scheduling in my own templates, can you explain? is there some action?

Everything depends on what is your needs.
Better tell me what you can not schedule with ZennoTask scheduler and I will give you some tips. how to achieve that.

In short you have to keep one template infinite loop what is checking time and date, and adding executions for your other templates.
Another way is to create your own task list. Like example:

Template1.xmlz=12:00=14:00=16:00=18:00=20:00
Template2.xmlz=00:00=02:00=04:00=06:00=08:00
Template3.xmlz=12:00=14:00=16:00=18:00=20:00
and so on. And in you template you set some small snippet what will check time every 10 minutes and if time fit with you from task list then just run your template, if not then just go out and do nothing...

Look bro, if you have more questions open new thread about it, and I will move posts over. Lets keep bug thread for bugs

Cheers

 

[bill]

Tell me please why you use task manager ? What is your requirements to run scrappers ?

Also please send your template with Diagnostic.exe to http://support.zennolab.com with reference to this thread...

Cheers

i need to use task manager because i'm working with faulty proxies so i setup a job to check proxies in between the other jobs. The first job ends, it outputs a text file, which activates the first proxy job, which tests if proxy is alive, and if it isn't grabs a new one from the proxy checker, then outputs the next text file.. after this the 2nd job runs, outputs a text file and then the 2nd proxy job runs and so on.

The task manager is essential because i can't run this job automatically if i can't use it to check the proxies. I have 1 proxy set to multiple threads, and i can't use the the GET PROXY action without it using a different proxy for each thread.

i'm kind of wasting my breath saying all this, because its not a problem of me not knowing what to do, I have a pretty firm grasp of this setup. The problem is the task manager is, for whatever reason, abruptly no longer outputting data,whereas, running the same job outside of the task manager scrapes fine.

I want to be able too setup multiple jobs using this same setup. so the idea of just abandoning the task manager altogether when it should work fine doesn't sit well.

 

[rostonix]

Im not sure i understand your problem. Any instructions how to reproduce it?

 

[bill]

if necessary I can send you details of project but this is a bug with ZP as far as i can tell at this point. so like i said job uses take dom action with regex and table is binded to a file. it outputs all data when run from zennoposter, but when i run it from task manager it will stop outputting data to text file after a short time.

here's photos

this is the empty text the table is binded too

so i run job normally(outside of task manager)

everything runs fine, file files up with data.

i'm clearing the file for testing

now im running same job in task manager, the job is activated by a text file.

heres where the problem occurs. it just stops scraping data.

 

[bill]

what should i do to get this resolved asap?

 

[rostonix]

Why it has stop icon instead of sheduler icon?

 

[bill]

Why it has stop icon instead of sheduler icon?

because the job/task finished. not sure if thats what you meant, because i don't know what you meant by scheduler icon

 

[rostonix]

send us your fill projects with files and information how they are organized in task manager to [email protected]

 

[bill]

send us your fill projects with files and information how they are organized in task manager to [email protected]

done