Forums

Hi, I'm trying to use Python requests to web scrape certain data from Spotify. It works fine on my local machine but I receive a long error when I try to run this as a task on PythonAnywhere.

Probably the most telling parts of the error:

MaxRetryError: HTTPSConnectionPool(host='open.spotify.com', port=443): Max retries exceeded with url: /artist/1WO1hFAkFbeo9tV3uVX7Dy (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f34346f5050>: Failed to establish a new connection: [Errno 111] Connection refused')))

I'm not sure if this is a technical error with my code or me needing to upgrade to a paid account. I see that 'accounts.spotify.com' and 'api.spotify.com' are both whitelisted domains. Does this mean that only data that comes up on those specific pages are available or should I also be able to access data on 'open.spotify.com'?

If open.spotify.com isn't on the whitelist, then you won't be able to access it. However, if it's part of an official public API, we can potentially add it -- we'd need a link to a place where it is mentioned in the the official API documentation in order to do that, though.

Thanks for your reply. I'm not quite sure I know what you mean by how the documentation would explicitly say different domains that you're allowed to use? But I have had a look and found multiple cases where they reference it and use it as an example of what can be done so I think this is enough.

API Documentation, instances in the example code and elsewhere

Thanks for your time

No problem, we've added open.spotify.com to our whitelist.