I have a script that receives json data from various sources and processes it. I have a list in a database and also as a text file of known good sources. The list has thousands of records.
Before processing I want to compare the source value from json with the source value in the list. Data is received every 10 sec. The list does not change often.
At the moment I can make this work either by querying the database for the sources list or read the list from a text file, however it seems redundant to do this every 10 sec upon receiving json since the list is going to be the same 99% of the time.
The question is – what is the good way to do this?
Advertisement
Answer
Assuming this DB is something you have more than read access – you mentioned the database records do not change often, you could add a trigger on the DB for any changes. Have the trigger update a single row in a new table called “listUpdated” to True. Load the list into an array in your PHP and use that to bump your data against. Every 10 seconds you can just check if the “listUpdated” field has been set to True. If it is, update your array and change the value back to False.