Forums

upgraded classic to glastonbury - subsequent error loading web2py & BeautifulSoup from scheduled task/script with fix

[edit]

Need to do some more work on this...
THIS HAS NOT WORKED - THE SCRIPT ALREADY HAD THE CORRECT LINE

When I ran the command manually from the bash console it worked fine

$ python web2py/web2py.py -S 'APP_NAME' -M -R applications/APP_NAME/private/SCRIPT_NAME.py
web2py Web Framework
Created by Massimo Di Pierro, Copyright 2007-2022
Version 2.14.6-stable+timestamp.2016.05.10.00.21.47
Database drivers available: psycopg2, pymysql, imaplib, MySQLdb, sqlite3, pg8000, pyodbc, pymongo, mysqlconnector

However the correct line was already in the script being run
It looks like the relative path reference is changed or broken... or there is a permissions issue??
[edit]


INITIAL POST...

Hopefully this helps someone else... The required image upgrade (moving from classic to glastonbury) caused an error in a scheduled script that loaded web2py and automatically ran another script (see solution below)

log output before change from classic to glastonbury

web2py Web Framework
Created by Massimo Di Pierro, Copyright 2007-2022
Version 2.14.6-stable+timestamp.2016.05.10.00.21.47
Database drivers available: psycopg2, pymysql, imaplib, MySQLdb, sqlite3, pg8000, pyodbc, pymongo, mysqlconnector
2022-10-25 09:10:29 -- Completed task, took 23.51 seconds, return code was 0.

log output after change from classic to glastonbury

web2py Web Framework
Created by Massimo Di Pierro, Copyright 2007-2022
Version 2.14.6-stable+timestamp.2016.05.10.00.21.47
Database drivers available: sqlite3, imaplib, pyodbc, pymysql, pg8000
Traceback (most recent call last):
  File ".../web2py/gluon/shell.py", line 263, in run
    execfile(startfile, _env)
  File ".../private/pa_message_queue_processor.py", line 9, in <module>
    from BeautifulSoup import BeautifulSoup
  File "/home/.../web2py/gluon/custom_import.py", line 95, in custom_importer
    return base_importer(pname, globals, locals, fromlist, level)
ImportError: No module named BeautifulSoup
2022-10-26 09:10:22 -- Completed task, took 14.91 seconds, return code was 0.

** I thought the solution was in updating the web2py.py path reference in the initial one line scheduled script
from: python web2py.py -S 'APP_NAME' -M -R applications/APP_NAME/private/SCRIPT_NAME.py
to: python web2py/web2py.py -S 'APP_NAME' -M -R applications/APP_NAME/private/SCRIPT_NAME.py

Is there anyone there from pythonanywhere that can throw some light on this for me?

You have not installed BeautifulSoup in the python version/virtualenv that you are using to run the scheduled task: https://help.pythonanywhere.com/pages/InstallingNewModules/

May thanks for the response Glen!

It leaves me with a couple of follow up questions (I haven't spent much time on virtual environments and don't recall specifically installing one)...

  • Are virtual environments created automatically by the three default Python versions selected on the system image page?
  • How do I identify/address the relevant virtual environment?

Additional info (may be repeating some from previous post)...
- the interactive db is still running fine
- running the script from the bash prompt works fine
- the script just isn't working from the scheduler

In a bash console there is no output when I try to list environments..

00:06 ~ $ lsvirtualenv                                                                                                                                                        
00:06 ~ $

However I can run python and it loads 2.7.6 and I can load BeautifulSoup without issue...

$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from BeautifulSoup import BeautifulSoup
>>> dir()
['BeautifulSoup', '__builtins__', '__doc__', '__name__', '__package__', 'readline', 'rlcompleter']

Incidentally the web2py version is an old one but aslo indicates it is running on Python 2.7.6

2.14.6-stable+timestamp.2016.05.10.00.21.47
(Running on Unknown, Python 2.7.6)

(I see Python 2.7.18 is available/running on the eu servers so now I am wonder if the above versions are as expected)

Are virtual environments created automatically by the three default Python versions selected on the system image page?

If you use virtualenvwrapper (mkvirtualenv command) you may want to add the --python option specifying Python command that should be used (e.g. python3.8). When I'm using it without this flag it seems to use the default Python command.

How do I identify/address the relevant virtual environment?

Again, if you use virtualenvwrapper it will create virtual envs in ~/.virtualenvs by default.

If you want to use a venv in a scheduled task, see this help page.

Thanks glen/pafk for the responses and sorry for the delay in responding but wanted to update/close this off in case anyone else is having similar problems.

The issues arose while upgrading web2py app version, moving it from Python2 to Python3 environment and also moving from US to EU servers (for anyone doing the same its up and running now and could have been so much easier had I paid more attention to the details).

The problems... - I set the three default Python versions in the system image page to 3.7 not realising/noticing the web app has its own default python version option on the web page which was set to 3.9 so while in bash I was trying to install packages to the wrong environment. - To compound things one of the packages I wanted to install is called daiquiri (a logging helper) but I was actually installing daquiri ( "a nuts and bolts included framework for scientific data acquisition").

Thanks for sharing that, @phadmin!