Skip to content
Advertisement

Curl Scraper working on localhost but not on online servers

I am trying to scrape https://www.gst.gov.in

echo $html = file_get_contents("https://www.gst.gov.in");

This code works perfect on localhost but not on the server.
I have tried using different serves with various curl methods with custom headers and referrers but no luck. On the server, I get a connection timeout error.

If I use any other https URL or another site it works fine. the problem is with this specific URL can anyone help in scraping this page also if anyone can tell if the remote server is blocking the request then how to bypass this.

Advertisement

Answer

It’s a government website – it’s most likely blocking any IPs from geographical region that is not local.

i.e. your server needs to be in India or have Indian proxy since you are trying to download Indian government page.

User contributions licensed under: CC BY-SA
3 People found this is helpful
Advertisement