basically, I am trying to scrape webpages with php but I want to do so after the initial javascript on a page executes – I want access to the DOM after initial ajax requests, etc… is there any way to do this?
Advertisement
Answer
Short answer: no.
Scraping a site gives you whatever the server responds with to the HTTP request that you make (from which the “initial” state of the DOM tree is derived, if that content is HTML). It cannot take into account the “current” state of the DOM after it has been modified by Javascript.