Forums

Is this error somehow related to a site not been whitelisted? I thought the entire Yahoo domain are whitelisted? My app is supposed to maintain a server connection <YFconn = httplib.HTTPConnection('finance.yahoo.com')> while it iterates over links contained within the Yahoo domain with < YFconn.request('GET','/q/cp?s=%5EDJI+Components')> and < YFconn.request('GET','/q/ks?s='+tick+'+Key+Statistics')>. My method works locally, but how come it doesn’t work when ran on Pythonanywhere?

PythonAnywhere uses a proxy to filter requests from non-paying users. I would guess that the YFconn library is not respecting the proxy settings.

Have you tried using the requests or urllib libraries to access Yahoo and to check it works?

I think @rcs1000 is right.

python3.4 -c "import requests; print(requests.get('http://finance.yahoo.com/q/cp?s=%5EDJI+Components'))"

gives a 200 when I test it on a free account, so I think it's the library you're using...

I use the httplib library, is this not supported? What other options do I have to maintain a server connection while iterating over its links?

You can use the httplib library if you want, but it's a really low-level library -- the second sentence of its documentation says "[i]t is normally not used directly". It requires you to write special code to go through a proxy -- if you really must use it for some reason then this Stack Overflow post gives some hints. But I really really recommend you try alternative libraries.

The third sentence of the httplib docs says that "[t]he Requests package is recommended for a higher-level http client interface", and that's what I would use if I were you. Requests works transparently with a proxy like the one free PythonAnywhere accounts use, so the same code will work whether you're running it on your own machine, on PythonAnywhere, or pretty much anywhere else.

Thanks for the help everyone. I installed requests library, and it works fine locally, but when I ran it on Pythonanywhere, I got the following:

ImportError at /
No module named requests

I have no idea what is causing this. The version I have is 2.9.1 by the way.

Just a quick check- did you install requests locally or install requests on PythonAnywhere? (ie. by starting a bash console on PythonAnywhere and installing it successfully)

I only installed the request lib on my machine. Do I have to do it on PythonAnywhere too? If so, what's the proper way of doing it?

http://help.pythonanywhere.com/pages/InstallingNewModules

Is requests not installed by default on Python Anywhere?

It is, unless you're using a virtualenv for your web app.

I install the library under a virtualenv folder following the tutorial, and I got the following installation message:

(my-virtualenv) $pip install requests
Collecting requests
/home/cqcum6er/.virtualenvs/my-virtualenv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading requests-2.9.1-py2.py3-none-any.whl (501kB)
    100% |████████████████████████████████| 503kB 71kB/s 
Installing collected packages: requests
Successfully installed requests-2.9.1

However, after reload my app, it still gave me the same ImportError message. I tried to look up where the library was installed using the shell command but still without success:

>>> import requests
Traceback (most recent call last):
  File "<console>", line 1, in <module>
ImportError: No module named requests

I tried to look for the library within my folders but could not find it. Any idea what is going on with the requests installation?

Are you sure you're using the that virtualenv for the install, your web app and for the shell command where you're testing the import?

When I created my web app, I created a virtualenv folder with the command:

$virtualenv --python=python2.7 myvenv

The folder has the path: > home > cqcum6er > my-first-blog > myvenv. Later I made another folder following another tutorial when installing requests:

$mkvirtualenv my-virtualenv

With the path: > home > cqcum6er > .virtualenvs > my-virtualenv. Are there differences between the two beside being installed at different levels? Do they run into conflict with each other?

They wouldn't conflict with each other, no. Although it can be a bit confusing to have virtualenv folders scattered around the place -- you tend to forget which one is which.

We tend to use a tool called "virtualenvwrapper". It's a set of little helpers for virtualenvs. By default it stores all your virtualenvs in a hidden folder in your home directory.

To start a new virtualenv, you do

mkvirtualenv --python=/usr/bin/python3.4 project1-env

Then, later, when you want to work on it, you can do

workon project1-env   # this tab-completes btw

I think I solve the problem. The error disappeared after I used "pip2.7" instead of "pip" for the installation. But I thought the default pip uses Python version 2.7?

If your virtualenv is active, and it's a Python 2.7 virtualenv, then both "pip" and "pip2.7" should be the same thing.

Outside the virtualenv, they will both refer to the system-wide pip, also for python 2.7. But this one will give you an error unless you use the --user flag.

You just have to be clear about when your virtualenv is active and when it is not. you can see it in the Bash prompt:

$

means it is not active

(my-virtualenv)$

means it is active...