Forums

Hey Folks,

I have a Flask application working on my local machine that I am having trouble porting over to PythonAnywhere. It is an NFL data app. The app takes inputs from a URL, fills in an ipython notebook template with the values from the URL, runs the ipython notebook, converts the new notebook to html, and serves it.

The function is here:

import IPython
from IPython import nbformat
from IPython.nbconvert.preprocessors.execute import ExecutePreprocessor
from IPython import nbconvert
import os
import sys

path = '/home/gitzman/FootballAnalysis/'

sys.path.append(path)

def editNotebook(start,end=7):
nb = nbformat.read(path+"3danalysis.ipynb", as_version=4)

nb.cells[0]["source"]=nb.cells[0]["source"].format(start,end)
nb.cells[3]["source"]=nb.cells[3]["source"].format(start,end,end+1)

pp = ExecutePreprocessor()
pp.timeout = 30  # seconds
pp.interrupt_on_timeout = True

nb_executed, resources = pp.preprocess(nb, resources={})
directory="static/"
filename=str(start)+str(end)+"export"


with open(path+directory+filename+".ipynb", 'w') as f:
    f.write(nbformat.v4.writes_json(nb_executed))

os.system('ipython nbconvert {0} --to html --output {1}'.format(path+directory
+ filename+".ipynb" ,path+directory+ filename+".html"))

return filename+".html"

Something is happening in this line:

nb_executed, resources = pp.preprocess(nb, resources={})

It looks like there is something odd happening with kernel creation when I try to host this as a website.

This is the error message:

2015-11-02 20:11:35,956 :Exception on /3d/3 [GET]
Traceback (most recent call last):
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/flask/app.py", line 1381, in      handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/gitzman/FootballAnalysis/flasky.py", line 17, in thirdd
return app.send_static_file(makenotebook.editNotebook(start))
File "/home/gitzman/FootballAnalysis/makenotebook.py", line 22, in editNotebook
nb_executed, resources = pp.preprocess(nb, resources={})
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/IPython/nbconvert/preprocessors/execute.py",    line 55, in preprocess
cwd=path)
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/IPython/kernel/manager.py", line 420, in   start_new_kernel
km.start_kernel(**kwargs)
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/IPython/kernel/manager.py", line 240, in start_kernel
**kw)
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/IPython/kernel/manager.py", line 189, in _launch_kernel
return launch_kernel(kernel_cmd, **kw)
File "/home/gitzman/.virtualenvs/football/local/lib/python2.7/site-packages/IPython/kernel/launcher.py", line 213, in  launch_kernel
proc = Popen(cmd, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

I hope that somebody can help me to get this working, as I've been trying to deploy this for a few days and I feel very close at this point. If you need any additional code, like the flask app that calls this function, or the ipython notebook being edited, please let me know.

Thank you!

-Incredibly Appreciative!

Hi there,

I am able to get a version of your code working using our docker consoles (beta) and having installed juptyer instead of the older ipython (eg. there are deprecation warnings when you try to from IPython import nbformat). I did not try the older version. Unfortunately, this is not available for webapps yet.

We are actually hoping to offer runnable ipython notebooks soon, so when that happens, you can just open your ipython notebook from pythonanywhere and be able to chg/run it in real time. You could then have a scheduled task that does the value updating behind the scenes as well.

Let us know if you want to try our docker consoles beta in the meantime.

Hey Conrad, thank you very much for your reply.

I am trying to host this as a webapp where people can enter in the URL week #'s and then see the template updated to include data for those week #'s. The end goal will be a website with automated NFL analysis. I am trying to get this up and running as a Pilot so that I can then build more templates, and include additional functionality like team highlighting.

I also have the code working from my PythonAnywhere bash, but I guess you are saying it just will not work as a PythonAnywhere webapp?

I have been trying to deploy this app for a decent amount of time and I am struggling. Is there any chance this will be runnable as a webapp at PythonAnywhere in the near future?

If not, do you know of a hosting service that would be flexible enough to run this? I tried Heroku, but my slug size is too big because of all the dependencies. PythonAnywhere seemed great because it looks and feels exactly like my local environment. Let me know. Thanks!

Oh- the solution I hacked up was workable in docker bash but not webapps, but if you got it working from bash then it seems as though it should also work from webapp then.

Thoughts about the kernel issue: maybe you can specify which kernel to use (eg: in your preprocessor etc)? I know that for what we are doing in our implementation of ipython notebooks, we make the kernels and link to them.

Also keep in mind that your webapp is run from /home/gitzman, while your bash code is run from wherever your current working directory is, and run with perhaps env variables etc loaded from your bashrc etc. Hope that is useful for you to debug.

Thank you very much! This certainly seems like the right road to go down. I'll plug away at this after work tonight and I'll let you know if I make any progress (or don't).

One thing that will help me debug, is when I run my flask file via bash, is there a way for me to open a virtual browser to see the browser version of my app's output? Apologies is this is a FAQ.

Thanks again Conrad!

btw- is it possible to just manually run it from the console every week to update the numbers, and serve the html file generated from your flask app? that sounds like it would simplify things a lot.

Just to clarify, are you running your flask app from the console page with a random port? You can take a look at our blog post here about getting selenium to work. (you can then take screenshots or just download the html body and view it from your files page) Please note that for running your flask app from the console, you won't be able to access it externally, and your port number will not be guaranteed to be free.

Ha I think I'll just do that and serve all static files. I guess I just felt it was cooler to dynamically generate them, but it will be so much easier to generate them from bash and then serve a bunch of different static files.

I'll do that, and then I can work on dynamic generation at a much slower pace.

Thanks!