Forums

I have a selenium program whose job is to log into a website, wait for the page to load, and then get the page_source.

It works fine when run on my local machine (Ubuntu), but when I run it on Python Anywhere, after I make selenium click the login button, the main page fails to actually load, even if I wait a couple minutes.

Is there any reason why this would be the case? When I run the selenium browser offline using the same code, using the same arguments (--headless, etc), it works.

The browser setup I'm using is:

chrome_options = selenium.webdriver.ChromeOptions()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
driver = webdriver.Chrome(options=chrome_options)

I also tried running after setting up a display with pyvirtualdisplay, but that doesn't help.

It's possible that the site in question is blocking login attempts from cloud computing environments. Many people who are running sites don't want their site to be scraped, so they put in place systems to block that -- they check the IP address for incoming connections, and then make the site behave differently if they suspect that the access is coming from a script rather than a real human.