Forums

Hi, I have the hacker plan ($5/month). I am building an API web app (no virtual env) on PythonAnywhere that scrapes marketplace websites and am using Selenium/Chrome driver/BeautifulSoup. My code works well on my localhost but when hosted on PythonAnywhere, it would perform really inconsistently. For every call that I make, I am expecting a returned JSON but it would only return about 40-50% of the time (no errors though). Can someone please help with this.

It might be that the site you're scraping is returning different results at different times -- sometimes sites notice that you're accessing them from a cloud computing environment and handle things differently. Your best bet would be to log what is coming back when you make your requests and seeing what you get when it is not the expected JSON.