How to keep fresh links while deleting processed ones.

[britalic]

Hey all, I hope this is easy for someone, because I am stuck.

- I have a master list of urls that have been processed already.
- I now get new urls from the internet, for processing.

From this list of "new" urls I need to delete urls that I have processed already, these urls are the ones in the master list.

In other words I need to delete urls from new list, only urls that are in master list, keeping the unprocessed ones.


Thanks for any help

 

[VladZen]

Not sure what you mean exactly... but if some fresh links are the same as processed ones, use action Lists>List processing>Remove duplicates.

 

[britalic]

imagine you are posting to blogs but you only want to post ONCE per domain. You have posted to a billion domains already and are looking for new domains. So you get new links from the net. You now want to take the billion domain AWAY from the list you just got from the internet so you are left with the blogs you have NOT posted yet.

it's very easy if you think of it. I have a tool that does this and have to do it by hand

 

[britalic]

imagine you are posting to blogs but you only want to post ONCE per domain. You have posted to a billion domains already and are looking for new domains. So you get new links from the net. You now want to take the billion domain AWAY from the list you just got from the internet so you are left with the blogs you have NOT posted yet.

it's very easy if you think of it. I have a tool that does this and have to do it by hand

there is a block that gets close to it indeed is the one that removes duplicates. It needs a line that says: "remove all lines as found in ____ " and then browse to a file containing the urls, in list B to be removed from list A

and every time you get new urls you keep on processing them to make sure none of these urls are already contained in the masterlist.

 

[Tobbe]

Make sure to update the Master List (bind and all that) after you've taken a new line from New List and processed it.
I wouldn't trust zenno to handle a list with a billion lines tho.

 

[britalic]

Make sure to update the Master List (bind and all that) after you've taken a new line from New List and processed it.
I wouldn't trust zenno to handle a list with a billion lines tho.

Hey you did it, that's very good, thanks!
I am not using it with domains, that was just the best example I could come up with as I guess every marketer has the prob of sifting new url from rough lists and so I am now trying to find out just about how this thing works.. so far it beats me.
I mean why would one need to bind all data removing all spaces is beyond my understanding right now.
But it does work and that's all I needed, just need a bit to come round to this way of doing things.
Thanks man if you pm me your paypal I'll buy you a beer.

 

[Tobbe]

@britalic

Think of it like this. When you don't bind the file you just make a copy of the text file and work with the copy in zenno, leaving the original file untouched.
If you wanna process the same data each time the template is run this is a good solution.

Example:
You have 10 urls in a text file you'd like to visit every time the template is run.
Put those into a list -> Take one line and put it into a variable -> Delete the line -> Visit Url from variable.
If you loop the underlined steps you'd soon run out of lines (urls) and the list would be empty, meaning you've visited all urls, processed the lines one at the time, and the process is done.

- If the list was not binded.
Next time you run the template it will make an internal copy of that file again and use as a list, the original file was never changed. So you can load that data how many times you want.

- If the list was binded.
Then, it would have worked with the original source and not a copy of it. So when it reached 'Delete the line' step it also removed that line from the original source/text file.
So next time the template is run the text file with 10 urls would now be empty, as the loop would've deleted the lines one at the time.

If you wanna give something back, checkout my signature.

 

[britalic]

thanks for your help Tobbe, I did follow your signature and posted you a note (asking for paypal email really) but page went dead