Synchronize Web-API with local Database

[morpheus93]

Hi,

I'd like to synchronize a proxy-list from an web-api, that gives me a JSON format with my local MySQL database. As a result all proxies that are in the output of the JSON web-api shall stay in the DB (if they are already contained), otherwise they should be added. Additionally all proxies that are already in the DB but NOT contained in the JSON output of the web-api shall be updated in the DB (set table-columns "active" to "offline") or deleted.

Any help is much appreciated!

Thank you!

 

[EtaLasquera]

This is not a question, you want to hire a job.

 

[morpheus93]

This is not a question, you want to hire a job.

I already parsed the JSON API in the right format and thought I could save it to a list. Then take line after line from the list and check if the IP is already in the table of the DB. If yes, set the column "active" to 1 if not to 0. And then vice versa take all table entries and check if they are in the list (from the JSON API) and set active appropriate.

Shouldn't be much effort.

But maybe it's even easier with updating multiple rows at once and/or INSERT ... ON DUPLICATE KEY UPDATE...

 

[PHaRTnONu]

im kinda baffled by this... its litteraly so basic. i dont mean this as a disrespectful way. but you could literally just process that w a query command in your database...
IDK what DB your running as you provided no information but google or udemy will teach you this info for free.
Your going to have to batch it either way in the way you described

 

[morpheus93]

im kinda baffled by this... its litteraly so basic. i dont mean this as a disrespectful way. but you could literally just process that w a query command in your database...
IDK what DB your running as you provided no information but google or udemy will teach you this info for free.
Your going to have to batch it either way in the way you described

Yes, was just a peanut. I'm just at the beginning to working with databases in Zenno and so it looked more complicated at the beginning. Btw, using MySQL, as you asked.

 

[PHaRTnONu]

i assume the key in the DB is the IP address?

 

[morpheus93]

i assume the key in the DB is the IP address?

Yes, I have a unique key as a combination of proxy-IP and proxy-source columns (because sometimes it could be that the same IP comes from another proxy-source and I want this entries to be allowed in the table).

 

[PHaRTnONu]

huh... well here is 1 solution when you "update" your list just turn EVERY value to zero and update the valid ones if hte ip matches the ip as the key to 1
thats the fastest and quickest solution for you