Forums

I am a paying customer (custom account), and I am trying to use BeautifulSoup to scrap the website sports-reference.com. This works on my own web server, but not on PythonAnywhere.

The error message given is: "2016-03-06 22:48:07,904 :urllib2.HTTPError: HTTP Error 403: Forbidden"

Trying to scrape the website badcomputer.org (a website on the whitelist) does work, so I'm assuming I just need sports-reference.com to be added to the whitelist.

As you don't have a free account, your requests don't go through the proxy. It is likely that Sports-Reference.com is blanket banning requests that come from Amazon Web Services. Sorry.

Hmm, it might not be that bad. @tommyvaltin, I see that you upgraded to a paid account very shortly before you posted this message. Are you trying to run the code from a console that you had open from before you upgraded? Or have you reloaded your web app since you upgraded? If it's the former, try a new console. If it's the latter, reload the app and try again.