Forums

So I have a web scrapper that needs to scrape certain websites that stops requests from a bot. I was able to go through the it using custom headers on my computer. But my flask program is not running after I uploaded it here,

 response = requests.get(url, headers=headers)

This is how my program makes a request to the site, with the headers. So does pythonanywhere change the headers somehow? if so how can I change it back to the one I want, if that's not the problem what else can it be and how do I fix that?

We do not change the request headers that you specify in your request.

I have the same problem, how can i fix that?

As Glenn said above, we don't change the headers. Your issues may also be related to the fact that you use a free account, which has restricted internet access.