Im guessing its php cURL, but Whats the best way to make a loop to scrape the DOM for info from a webpage that uses id’s in the URL Query like (?ProductId=103) There is about 1200 pages. I need to find the innerHTML of the 9th span on each page. This info will just get stored in a mySQL table (id->value) for future scraping of this site.
Advertisement
Answer
Well curl might be faster (not sure), but if it is a one off thing, then I would just use file_get_contents
for($x=0;$x<1200;$x++){ $f = file_get_contents(URL . '?productId='.$x); #do stuff to $f }