How can I run projects together, but wait for others to finish?

[nycdude]

OK, I didn't know a better way to title this post but here goes ...

I have 12 projects that I need to run in a certain order but I can't control them how I want.
Project A needs to run once to scrape URLs.
Projects B1-B10 will run in loops at once to get the data from the URLs
Project C processes all the data.

Project A is my search engine scraper, I can set it up to create a trigger file to start the project Bs but it's not very dependable.
Projects B1-B10 will run together looping through the URLs collecting all my data.
Project C needs to wait until the Bs are done to start processing the data.

I'm trying to figure out how to make C wait until the Bs are done. I looked through the forum and saw some ideas but not sure if they would work for this. I looked into nesting projects but they only run at a time, I need the 10 Bs to all run at once.

Any help appreciated.

 

[lokiys]

Hi.

I'm very sure that You do not need that much templates to accomplish what You want, but anyway. In your case I would make my own scheduler template what will run in loop and check how all other templates are doing.
There are lot of options in c# to make own scheduler. Example: https://help.zennolab.com/en/v5/zennoposter/5.17.0.0/webframe.html#topic668.html

 

[nycdude]

Hi.

I'm very sure that You do not need that much templates to accomplish what You want, but anyway. In your case I would make my own scheduler template what will run in loop and check how all other templates are doing.
There are lot of options in c# to make own scheduler. Example: https://help.zennolab.com/en/v5/zennoposter/5.17.0.0/webframe.html#topic668.html

Ok thanks I'll check it out. I got so many scrapers working at once to complete the job faster, when I use a lot of threads on one project it messes up my output files so I keep them all separate.

Thanks for the help, I'm not well versed in C# yet so it' looks daunting. Are there any already made samples of this use I may study?

 

[lokiys]

Ok thanks I'll check it out. I got so many scrapers working at once to complete the job faster, when I use a lot of threads on one project it messes up my output files so I keep them all separate.

Thanks for the help, I'm not well versed in C# yet so it' looks daunting. Are there any already made samples of this use I may study?

All your posts smells fishy You obvious do some wrong logic there.
There should not be any problems with files and data even when you work with 200 threads.
To get job done faster and if that's related to scrapping then study POST/GET requests https://zennolab.com/wiki/en:actions:http and make just one template with all logic in it, and forget about 20 templates for one task.

 

[nycdude]

All your posts smells fishy You obvious do some wrong logic there.
There should not be any problems with files and data even when you work with 200 threads.
To get job done faster and if that's related to scrapping then study POST/GET requests https://zennolab.com/wiki/en:actions:http and make just one template with all logic in it, and forget about 20 templates for one task.

Haha, fishy? Yeah it's true. I'm new so learning as I go. I'm using GET to scrape but I'm also going individually to each scraped image online and checking it's dimensions and resolution if I want it or not. They scrape google bing yahoo twitter youtube and more and check the source links as it goes.

With time I'll get better, having a ball.

 

[lokiys]

Haha, fishy? Yeah it's true. I'm new so learning as I go. I'm using GET to scrape but I'm also going individually to each scraped image online and checking it's dimensions and resolution if I want it or not. They scrape google bing yahoo twitter youtube and more and check the source links as it goes.

With time I'll get better, having a ball.

You have to disable browser for scrapping, what will make your bots work much better, faster. Then You will be able to do all from one template.

 

[morpheus93]

You have to disable browser for scrapping, what will make your bots work much better, faster. Then You will be able to do all from one template.

by the way, what does "Browser w/out content view" mean in this settings box?

 

[zortexx]

by the way, what does "Browser w/out content view" mean in this settings box?

It means that browser runs as usual but without generation video raw. In this case a bit of cpu, ram and video resources could be saved in theory
On practice there is a lot of reasons why I cannot recommend to use this feature.