Forums

Is there anyway to use PhantomJS with splinter/selenium on PythonAnywhere?

Also, when trying to use Firefox, example:

import splinter

browser = splinter.Browser("firefox")

browser.visit("http://google.com")

print browser.html

I get the error message:

File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py", line 90, in _wait_until_connectable                                                                                                           
    self._get_firefox_output())                                                                                                                                                                                                             
selenium.common.exceptions.WebDriverException: Message: "The browser appears to have exited before we could connect.      The output was: \n(process:20484): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed\nError: 
no display specified\n"

Does this mean I can't use splinter/selenium at all?

Perhaps this might help?

https://www.pythonanywhere.com/forums/topic/1192/#id_post_8360

It's worth trying 'selenium' in the forum Search box.

Thanks,

Is there any chance of Phantomjs working in the future?

We're actually putting some infrastructure changes in place that should make it possible in future, yes. But it'll be months, not weeks, before we'll be able to start testing things like this...

That's good to hear, is there anywhere I can signup to receive a notification of big changes like that? Or do you send out emails by default for things like that?

I've got your name down against the ticket, so we'll email you if we manage to get it working. We might even include it in some kind of newsletter, if it's part of a wider set of improvements...

Great, thanks for your help.

@harry, I'm interested in phantomjs too. please let me know when you have something working.

noted. but, like I say, this is months away, not weeks.

@harry, I'm interested in following progress on phantomjs support too. Thanks.

Cool. I've added you to the list.

Thanks, Glenn.

@harry Please add me to the list of people to be notified of PhantomJS progress.

will do.

Hey guys,

Just to clarify, as some people are getting confused about what works and doesn't work, splinter + pyvirtualdisplay should work out of the box, as should selenium + pyvirtualdisplay. However, we have no support for phantomjs or ghost as of right now.

Just for a quick test, take the code from the front page of splinter and run it with pyvirtualsdisplay:

from pyvirtualdisplay import Display
from splinter import Browser

display = Display(visible=0, size=(800, 600))
display.start()

try:
    with Browser() as browser:
        # Visit URL    
        url = "http://www.google.com"
        browser.visit(url)
        browser.fill('q', 'splinter - python acceptance testing for web applications')
        # Find and click the 'search' button
        button = browser.find_by_name('btnG')
        # Interact with elements 
        button.click()
        if browser.is_text_present('splinter.cobrateam.info'):
            print "Yes, the official website was found!"
        else:
            print "No, it wasn't found... We need to improve our SEO techniques"
finally:
    display.stop()

I can run it directly and get as output:

Yes the official website was found!

Similarly, selenium + pyvirtualdisplay also works. If you are running into any problems, please make sure you are stopping your displays and not letting them persist (which is what happens if display.stop() doesn't get executed). Dig through the forums for more info.

[edit by admin: dead link removal]

@conrad

Yep, that works for me.

I don't normally use the word, but that's awesome - I hadn't realised things had got that easy!

Many thanks, Jim

Hi all, for anyone still interested in running phantomjs on PythonAnywhere, we're currently beta testing a new type of sandboxing for consoles based on docker. More info in this blog post on docker consoles.

If you'd like to try out docker consoles (and thus phantomjs), give us a shout and we'll switch it on for you!

NB - consoles only at the moment, so it won't work in scheduled tasks or web apps.

A note for anyone trying to get phantomjs running. Here is how I did it.

Let's say I am currently working in /home/conrad/my-site

First run npm install phantomjs, which will create a node_modules folder.

Then, from a python script, something like this should work:

from selenium import webdriver
driver = webdriver.PhantomJS(executable_path='/home/conrad/my-site/node_modules/phantomjs/bin/phantomjs')

Hi , i have try get phantomjs running as instructed above (run the npm install phantomjs). But i still get error "Cant connect to ghostdriver"

i try to run phantomjs binary.. and it crashed

Hi wrathgeek, you missed half the story in this thread I'm afraid! You'll need the experimental docker consoles feature to get phantomjs working. Have a read of the blog post linked above, then let us know if you want us to switch it on for you.

Thanks Harry. My bad for missing half the story. can you switch it on for me? Thanks in advance!

OK, that's active now. Any new consoles you create will be docker consoles under the hood. You shouldn't notice any major differences, apart from phantomjs working hopefully. But it is a beta feature, so let us know if you spot anything strange...

Could I have docker consoles activated on this account? Also, are there any updates on docker consoles? Specifically, when they might be available in scheduled tasks or web apps?

@firstline all the new consoles that you open should be docker based now. You can check this by running a ps on it. Not sure about timeline for scheduled tasks or webapps yet.

Hi, Please activate docker consoles on this account . Thanks in advance!

done- any new consoles you open should be docker consoles now.

Hi, can you activate docker console for this account too? Is there any way to get phantomjs working in webapps or scheduler tasks ?

Hi there, no phantomjs in web apps or scheduled tasks I'm afraid, no. You should be aware that docker-enabled consoles are slower to start up (although once they've started they perform similarly to old-style consoles). Would you still like me to switch them on for you?

Hi, what is the latest update on this? I would also like to use phantomjs, can you please turn on the docker console for me?

Thanks!!

Sure, I've switched that on. Please note, though, that it will only work in consoles -- not scheduled tasks, or from web apps.

Hi. Could someone please switch on docker consoles for me, too? Thanks!

@resultsrobot, that's done now. Any new consoles you create will be able to run phantomjs...

Hi turn on docker console for me too :)

Done!

thanks, now i am able to run phantomjs, but i am unable to remove error/exception: "available via screen"

i am unable to scrap from websites, with https protocol actually, when i try, i get error

File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Screenshot: available via screen

The exception suggests that it's timing out waiting for something. Perhaps you can find what it's waiting for in the rest of the traceback.

well the same code is working in my laptop using pycharm

Different sites sometimes behave differently based on where they're being accessed from. Perhaps you could try running curl in a bash console to try to access the same URL -- it's possible that they're blocking incoming connections from cloud servers because they don't want people to scrape the site.

thanks, but one thing more, but now i am able to run phantomjs through consoles but the same code using phantomjs is not working when i add that file in the schedule tasks Thankyou

Unfortunately PhantomJS only works in Docker consoles -- not in scheduled tasks or in web apps.

I want to use PhantomJS. Could someone please switch on docker consoles for me?

okay- we've switched on docker consoles for you. Any new consoles you start should be docker ones. You can check by doing a ps (there shouldn't be any permission errors).

I need to use PhantomJS

Will this be from inside consoles on PythonAnywhere? Or from a website's code, or scheduled tasks?

console

OK -- I've switched you over to Docker-based consoles; if you start a new console, PhantomJS should work inside it.

I've got the same problem of using Phantomjs, can anyone help me? I've already install phantomJs but still got error

selenium.common.exceptions.WebDriverException: Status code was: -11

we don't really support phantomjs. if you want we can switch you to our experimental docker consoles, and then it might work, but only in consoles, not in web apps or scheduled tasks, so it's unlikely you'd be able to use it for anything reliable. let us know if you still want to give it a go, bearing in mind those caveats.

Hi Pythonanywhere moderators, I need Docket-based consoles feature to be switched on on my account, I really need to run phantomjs , Besides that, I need to run it with external proxy ip:port mode, please let me know is it possible Thanks

OK, that's switched on for your account now. I don't know about how to set up the proxy stuff, though -- but if it works from your own local machine, it should work from our servers.

Hello! I'm trying to utilize Phantom JS for my script. These are all the simple steps that i've taken:

1. Open console (without activating virtualenv)
2. Install PhantomJS (pip install phantomjs)
3. Select it as webdriver (driver = webdriver.PhantomJS(executable_path="/home/MTonskins/<main_folder>/<site _name>/node_modules/phantomjs/bin/phantomjs")

After executing third step, this error is given:

selenium.common.exceptions.WebDriverException: Message: Service /home/MTonskins/<main_folder>/<site_name>/node_modules/phantomjs/bin/phantomjs exited. Status code was: 0

Do i need to enable Docker-based consoles as well? Or the source of the problem might be something else?

Also, do web applications support PhantomJS from now?

Thanks!

hi there, phantomjs will only work with the experimental docker consoles, yes. But it doesn't work in webapps or scheduled tasks. Let me know if you're still interested in switch console types.

I'm using Django 2.2.5, Bokeh 1.4 and trying WeasyPrint 50 because pythonanywhere doesn't support Pdfkit, now I try to embed a graph in a template view for render a pdf document but is not working because it requires PhantomJS and selenium, and for for my bad luck PhantomJS doesn't install :( Can I try docker and Pdfkit? I'm really appreciate suggestions or options.

No problem. I've switched your account to use docker consoles. Any new consoles you start will now use docker.

Hello i would like to use PhantomJS by and selenium in console script. Can you switch the consoles in my account ? Regards, Paolo.

We no longer support docker consoles, but our new process namespaces virtualisation should work for you. I have enabled it for your account. It will be used in any new consoles you start.