Forums

Hi,

I'm scraping data from a website in a scheduled task and when I try to download it I get:

HTTPError: 403 while retrieving

I can't figure out why this is happening since I have a paid account, and when I run the same script straight from the console and the same virtualenv it works fine and downloads the data. If it helps to fetch the data I'm using urlopen but I don't see how the code can be the problem since it works fine from the console.

The only thing I could possibly think of is that the request is somehow blocked from the tasks server on pythonanywhere, is this possible? It seems unlikely since I'm pulling public data that is intended to be used for things like this...

I'm calling the task like this:

source virtualenvwrapper.sh && workon myvirtualenv && /home/myname/myfolder/scrape

Can anyone help me fix this? Thanks!

EDIT: Just realized this could be important to note, but I am running a bash script, not a python script. If I call a bash script from a scheduled python script to run the command could that possibly help? Or should I try importing subprocess and try calling the bash command from a python script?

Can you see what the 403 response body says? In addition to the 403 message.

It is possible that you are hitting that website too frequently and that website had banned the task server IP from access it.

Bash script/python script would not be relevant.