Forums

Mongodb Error

https://stackoverflow.com/questions/41759254/pythonanywhere-flask-500-internal-whenever-post

I am having this same issue, but I have a paid account. Any solutions?

Beloved premium user jcrowe | 8 posts | Sept. 8, 2017, 3:14 p.m. | permalink

Make sure that your mongo instance is visible to the internet. Also, search the forums - there are a number of people that have worked through various issues with mongo and some of them may apply to you.

Staff glenn | 9498 posts | PythonAnywhere staff | Sept. 8, 2017, 3:58 p.m. | permalink

The mongo instance is visible on the internet. Works fine with the development environment. Everything I have seen with this issue points to not having a paid account, which I do. Is there any assistance you can offer?

Beloved premium user jcrowe | 8 posts | Sept. 8, 2017, 4:15 p.m. | permalink

We don't interfere in any way with outgoing network for paid accounts.

If you're running the code in a console, you will need to do it in a new console (new since the upgrade). If you're running it in a webapp, you'll need to reload the web app after the upgrade.

Otherwise search the forums for more mongo-specific info.

Staff glenn | 9498 posts | PythonAnywhere staff | Sept. 8, 2017, 5:08 p.m. | permalink

Both the console and the webapp have been restarted.

I have searched the forums and there is nothing that addresses my connection problem. The Stack Overflow link talks directly to the issue.

Can you please provide some documentation about how to connect to MongoDB from PA?

Beloved premium user jcrowe | 8 posts | Sept. 8, 2017, 6:40 p.m. | permalink

Connecting to MongoDB from a PythonAnywhere paid account should be exactly the same as connecting from, say, your own computer. At a low level, what's happening is identical -- packets go from our servers to the Mongo instance and back again just the same way as they would from your machine.

But clearly if you can connect from your own machine but not from our site, there's something different happening, perhaps non-obviously.

A few questions:

  • Are you, like the person in the Stack Overflow post, trying to connect from a Flask app? Or some other kind of web framework?
  • If it's from a web framework, can you try connecting from a console to see if that behaves differently (if you haven't already)?
  • Could you post the exact traceback that you're getting? There might be non-obvious differences between the one that person is getting and the one are that might help debug?
Staff giles | 11800 posts | PythonAnywhere staff | Sept. 9, 2017, 3:09 p.m. | permalink

Nice site!

deleted-user-2824895 | 1 post | Sept. 11, 2017, 6:51 p.m. | permalink

Here is my actual error:

2017-09-11 20:50:58,639: /home/jcrowe/.local/lib/python2.7/site-packages/pymongo/topology.py:145: UserWarning: MongoClient opened before fork. Create MongoClient with connect=False, or create client after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#pymongo-fork-safe>
2017-09-11 20:50:58,639:   "MongoClient opened before fork. Create MongoClient "
2017-09-11 20:51:28,653: Error running WSGI application
2017-09-11 20:51:28,668: ServerSelectionTimeoutError: No servers found yet
2017-09-11 20:51:28,669:   File "/home/jcrowe/.local/lib/python2.7/site-packages/flask/app.py", line 2000, in __call__
2017-09-11 20:51:28,669:     return self.wsgi_app(environ, start_response)
2017-09-11 20:51:28,669: 
2017-09-11 20:51:28,669:   File "/home/jcrowe/.local/lib/python2.7/site-packages/flask/app.py", line 1991, in wsgi_app
2017-09-11 20:51:28,669:     response = self.make_response(self.handle_exception(e))
2017-09-11 20:51:28,669: 
2017-09-11 20:51:28,669:   File "/home/jcrowe/.local/lib/python2.7/site-packages/flask/app.py", line 1567, in handle_exception
2017-09-11 20:51:28,670:     reraise(exc_type, exc_value, tb)
2017-09-11 20:51:28,670: 
2017-09-11 20:51:28,670:   File "/home/jcrowe/.local/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app
2017-09-11 20:51:28,670:     response = self.full_dispatch_request()
2017-09-11 20:51:28,670: 
2017-09-11 20:51:28,670:   File "/home/jcrowe/.local/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request
2017-09-11 20:51:28,670:     rv = self.handle_user_exception(e)
2017-09-11 20:51:28,671: 
2017-09-11 20:51:28,671:   File "/home/jcrowe/.local/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception
2017-09-11 20:51:28,671:     reraise(exc_type, exc_value, tb)
2017-09-11 20:51:28,671: 
2017-09-11 20:51:28,671:   File "/home/jcrowe/.local/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request
2017-09-11 20:51:28,671:     rv = self.dispatch_request()
2017-09-11 20:51:28,671: 
2017-09-11 20:51:28,671:   File "/home/jcrowe/.local/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request
2017-09-11 20:51:28,672:     return self.view_functions[rule.endpoint](**req.view_args)
2017-09-11 20:51:28,672: 
2017-09-11 20:51:28,672:   File "/home/jcrowe/Amazon_Management_Platform/app/flask_app.py", line 34, in inv
2017-09-11 20:51:28,672:     df = pd.DataFrame([x for x in db.asin_data.find()])
2017-09-11 20:51:28,672: 
2017-09-11 20:51:28,672:   File "/home/jcrowe/.local/lib/python2.7/site-packages/pymongo/cursor.py", line 1114, in next
2017-09-11 20:51:28,672:     if len(self.__data) or self._refresh():
2017-09-11 20:51:28,672: 
2017-09-11 20:51:28,673:   File "/home/jcrowe/.local/lib/python2.7/site-packages/pymongo/cursor.py", line 1036, in _refresh
2017-09-11 20:51:28,673:     self.__collation))
2017-09-11 20:51:28,673: 
2017-09-11 20:51:28,673:   File "/home/jcrowe/.local/lib/python2.7/site-packages/pymongo/cursor.py", line 873, in __send_message
2017-09-11 20:51:28,673:     **kwargs)
2017-09-11 20:51:28,673: 
2017-09-11 20:51:28,673:   File "/home/jcrowe/.local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 888, in _send_message_with_response
2017-09-11 20:51:28,673:     server = topology.select_server(selector)
2017-09-11 20:51:28,674: 
2017-09-11 20:51:28,674:   File "/home/jcrowe/.local/lib/python2.7/site-packages/pymongo/topology.py", line 214, in select_server
2017-09-11 20:51:28,674:     address))
2017-09-11 20:51:28,674: 
2017-09-11 20:51:28,674:   File "/home/jcrowe/.local/lib/python2.7/site-packages/pymongo/topology.py", line 189, in select_servers
2017-09-11 20:51:28,674:     self._error_message(selector))

I have not run the connection from a bash prompt yet, but the error seems to talk to the idea of forking being the problem.

Beloved premium user jcrowe | 8 posts | Sept. 11, 2017, 8:55 p.m. | permalink

I am trying to connect from a flask app.

Beloved premium user jcrowe | 8 posts | Sept. 11, 2017, 8:55 p.m. | permalink

Hi there, we don't support threads in our web apps, and I think the latest version of the pymongo client library tries to force you to use threads. Try setting maxPoolSize=1 ?

Staff harry | 2710 posts | PythonAnywhere staff | Sept. 12, 2017, 12:46 p.m. | permalink

I ran a quick test in the console and didn't have any problems. The code ran as expected.

I have changed the maxPoolSize to 1. That didn't change anything.

Beloved premium user jcrowe | 8 posts | Sept. 12, 2017, 1:42 p.m. | permalink

have you tried googling the error message? eg this thread suggests setting connect=False https://stackoverflow.com/questions/44133435/mongoengine-and-dealing-with-userwarning-mongoclient-opened-before-fork-creat

Staff harry | 2710 posts | PythonAnywhere staff | Sept. 12, 2017, 4:58 p.m. | permalink

well... connect=False

Was the first thing I tried and it didn't work.

maxPoolSize=1

Also didn't work.

connect=False, maxPoolSize=1

Does work!

Thanks, Jason

Beloved premium user jcrowe | 8 posts | Sept. 13, 2017, 8:57 p.m. | permalink

Excellent! Glad you worked it out :-)

Staff giles | 11800 posts | PythonAnywhere staff | Sept. 14, 2017, 11:06 a.m. | permalink

Here's a help page about MongoDB on PythonAnywhere summarises that: hopefully it will be useful for anyone with questions about this in the future. Any suggestions for extra stuff we should put on there much appreciated!

Staff giles | 11800 posts | PythonAnywhere staff | April 6, 2018, 7:14 p.m. | permalink

Hi guys, I'm having a similar issue. Trying to get my Flask webapp to run, connecting to a mlab hosted MongoDB, but keep getting the "MongoClient opened before fork" error. The code is something like this:

from flask import Flask
from models import db
#where db = MongoEngine()

app = Flask(__name__)
app.config.from_pyfile('settings.cfg')

db.init_app(app)
db.connect(connectTimeoutMS=30000, socketTimeoutMS=None, socketKeepAlive=True, connect=False, maxPoolsize=1)

[...]

if __name__ == "__main__":
    app.run()

Tried all suggestions floating around here and stackoverflow, but nothing seems to help. It runs perfectly fine locally, so I'm sure it's related to the webabb. Below is my error log:

2018-05-19 10:10:07,277: /home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py:149: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#is-pymongo-fork-safe
2018-05-19 10:10:07,277:   "MongoClient opened before fork. Create MongoClient only "
2018-05-19 10:10:09,672: Error running WSGI application
2018-05-19 10:10:09,688: ServerSelectionTimeoutError: No servers found yet
Beloved premium user tingi | 7 posts | May 19, 2018, 10:28 a.m. | permalink

I think you need to do the db.connect inside your views rather than at the global module level -- could you try that and let us know if it helps? We can add that to the docs if it does.

Staff giles | 11800 posts | PythonAnywhere staff | May 19, 2018, 3:15 p.m. | permalink

It's related to initiating the .connect() command yes. With views, I assume you mean the Flask re-routing to load the page? I've tried the following instead with no succes:

@app.route("/")
def index():
    db.connect(connectTimeoutMS=30000, socketTimeoutMS=None, socketKeepAlive=True, connect=False, maxPoolsize=1)

return render_template('index.html')

The above code gives the same forking error. Whenever I omit the db.connect() command completely the error is reduced to:

2018-05-19 15:30:17,468: Error running WSGI application
2018-05-19 15:30:17,474: ServerSelectionTimeoutError: No servers found yet
Beloved premium user tingi | 7 posts | May 19, 2018, 3:39 p.m. | permalink

Could you give the full traceback for the forking error when you put the db.connect inside the index view?

Staff giles | 11800 posts | PythonAnywhere staff | May 19, 2018, 5:26 p.m. | permalink

Edit: removed because of double post

Beloved premium user tingi | 7 posts | May 19, 2018, 6:28 p.m. | permalink

Here's the full traceback, though I cut out some of the irrelevant script errors.

2018-05-20 06:54:31,274: /home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py:149: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#is-pymongo-fork-safe
2018-05-20 06:54:31,275:   "MongoClient opened before fork. Create MongoClient only "
2018-05-20 06:54:41,442: /home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py:149: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#is-pymongo-fork-safe
2018-05-20 06:54:41,442:   "MongoClient opened before fork. Create MongoClient only "
2018-05-20 06:55:01,290: Error running WSGI application
2018-05-20 06:55:01,310: ServerSelectionTimeoutError: No servers found yet
2018-05-20 06:55:01,310:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
2018-05-20 06:55:01,310:     return self.wsgi_app(environ, start_response)
2018-05-20 06:55:01,310: 
[...]
2018-05-20 06:55:01,315:     selector, server_timeout, address)
2018-05-20 06:55:01,315: 
2018-05-20 06:55:01,315:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 199, in _select_servers_loop
2018-05-20 06:55:01,315:     self._error_message(selector))
2018-05-20 06:55:11,459: Error running WSGI application
2018-05-20 06:55:11,464: ServerSelectionTimeoutError: No servers found yet
[...]
2018-05-20 06:55:11,469:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 199, in _select_servers_loop
2018-05-20 06:55:11,470:     self._error_message(selector))
Beloved premium user tingi | 7 posts | May 20, 2018, 7 a.m. | permalink

Could you post the full traceback? If it has private information in it that you don't want to post on the forums, you can send it to us via support@pythonanywhere.com. I'd really like to see the details from the incoming request all the way through to the raising of the exception, it's hard to help debug with only partial information.

Staff giles | 11800 posts | PythonAnywhere staff | May 20, 2018, 2:15 p.m. | permalink

Sorry there. It's not private, but I tried yesterday and my post got blocked awaiting for admin approval (I guess too long?). I just tried running it again, and this is the full error log:

2018-05-20 14:39:18,217: /home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py:149: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#is-pymongo-fork-safe
2018-05-20 14:39:18,217:   "MongoClient opened before fork. Create MongoClient only "
2018-05-20 14:39:22,817: /home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py:149: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#is-pymongo-fork-safe
2018-05-20 14:39:22,818:   "MongoClient opened before fork. Create MongoClient only "
2018-05-20 14:39:48,239: Error running WSGI application
2018-05-20 14:39:48,262: ServerSelectionTimeoutError: No servers found yet
2018-05-20 14:39:48,262:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
2018-05-20 14:39:48,263:     return self.wsgi_app(environ, start_response)
2018-05-20 14:39:48,263: 
2018-05-20 14:39:48,263:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
2018-05-20 14:39:48,263:     response = self.handle_exception(e)
2018-05-20 14:39:48,263: 
2018-05-20 14:39:48,263:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception
2018-05-20 14:39:48,263:     reraise(exc_type, exc_value, tb)
2018-05-20 14:39:48,263: 
2018-05-20 14:39:48,263:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
2018-05-20 14:39:48,263:     response = self.full_dispatch_request()
2018-05-20 14:39:48,264: 
2018-05-20 14:39:48,264:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
2018-05-20 14:39:48,264:     rv = self.handle_user_exception(e)
2018-05-20 14:39:48,264: 
2018-05-20 14:39:48,264:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
2018-05-20 14:39:48,264:     reraise(exc_type, exc_value, tb)
2018-05-20 14:39:48,264: 
2018-05-20 14:39:48,264:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
2018-05-20 14:39:48,264:     rv = self.dispatch_request()
2018-05-20 14:39:48,264: 
2018-05-20 14:39:48,264:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
2018-05-20 14:39:48,265:     return self.view_functions[rule.endpoint](**req.view_args)
2018-05-20 14:39:48,265: 
2018-05-20 14:39:48,265:   File "./flask_app.py", line 23, in index
2018-05-20 14:39:48,265:     for listing in Listing.objects().fields(date_str=1).order_by('-date_str').limit(1):
2018-05-20 14:39:48,265: 
2018-05-20 14:39:48,265:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/queryset/manager.py", line 37, in __get__
2018-05-20 14:39:48,265:     queryset = queryset_class(owner, owner._get_collection())
2018-05-20 14:39:48,265: 
2018-05-20 14:39:48,265:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/document.py", line 192, in _get_collection
2018-05-20 14:39:48,265:     db = cls._get_db()
2018-05-20 14:39:48,265: 
2018-05-20 14:39:48,266:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/document.py", line 181, in _get_db
2018-05-20 14:39:48,266:     return get_db(cls._meta.get('db_alias', DEFAULT_CONNECTION_NAME))
2018-05-20 14:39:48,266: 
2018-05-20 14:39:48,266:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/connection.py", line 235, in get_db
2018-05-20 14:39:48,266:     db.authenticate(conn_settings['username'], conn_settings['password'], **auth_kwargs)
2018-05-20 14:39:48,266: 
2018-05-20 14:39:48,266:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/database.py", line 1167, in authenticate
2018-05-20 14:39:48,266:     connect=True)
2018-05-20 14:39:48,266: 
2018-05-20 14:39:48,266:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 583, in _cache_credentials
2018-05-20 14:39:48,266:     writable_preferred_server_selector)
2018-05-20 14:39:48,267: 
2018-05-20 14:39:48,267:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 224, in select_server
2018-05-20 14:39:48,267:     address))
2018-05-20 14:39:48,267: 
2018-05-20 14:39:48,267:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 183, in select_servers
2018-05-20 14:39:48,267:     selector, server_timeout, address)
2018-05-20 14:39:48,267: 
2018-05-20 14:39:48,267:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 199, in _select_servers_loop
2018-05-20 14:39:48,267:     self._error_message(selector))
2018-05-20 14:39:52,837: Error running WSGI application
2018-05-20 14:39:52,844: ServerSelectionTimeoutError: No servers found yet
2018-05-20 14:39:52,845:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
2018-05-20 14:39:52,845:     return self.wsgi_app(environ, start_response)
2018-05-20 14:39:52,845: 
2018-05-20 14:39:52,845:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
2018-05-20 14:39:52,845:     response = self.handle_exception(e)
2018-05-20 14:39:52,846: 
2018-05-20 14:39:52,846:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception
2018-05-20 14:39:52,846:     reraise(exc_type, exc_value, tb)
2018-05-20 14:39:52,846: 
2018-05-20 14:39:52,846:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
2018-05-20 14:39:52,846:     response = self.full_dispatch_request()
2018-05-20 14:39:52,846: 
2018-05-20 14:39:52,847:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
2018-05-20 14:39:52,847:     rv = self.handle_user_exception(e)
2018-05-20 14:39:52,847: 
2018-05-20 14:39:52,847:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
2018-05-20 14:39:52,847:     reraise(exc_type, exc_value, tb)
2018-05-20 14:39:52,847: 
2018-05-20 14:39:52,847:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
2018-05-20 14:39:52,847:     rv = self.dispatch_request()
2018-05-20 14:39:52,848: 
2018-05-20 14:39:52,848:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
2018-05-20 14:39:52,848:     return self.view_functions[rule.endpoint](**req.view_args)
2018-05-20 14:39:52,848: 
2018-05-20 14:39:52,848:   File "./flask_app.py", line 23, in index
2018-05-20 14:39:52,848:     for listing in Listing.objects().fields(date_str=1).order_by('-date_str').limit(1):
2018-05-20 14:39:52,848: 
2018-05-20 14:39:52,848:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/queryset/manager.py", line 37, in __get__
2018-05-20 14:39:52,849:     queryset = queryset_class(owner, owner._get_collection())
2018-05-20 14:39:52,849: 
2018-05-20 14:39:52,849:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/document.py", line 192, in _get_collection
2018-05-20 14:39:52,849:     db = cls._get_db()
2018-05-20 14:39:52,849: 
2018-05-20 14:39:52,849:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/document.py", line 181, in _get_db
2018-05-20 14:39:52,849:     return get_db(cls._meta.get('db_alias', DEFAULT_CONNECTION_NAME))
2018-05-20 14:39:52,850: 
2018-05-20 14:39:52,850:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/connection.py", line 235, in get_db
2018-05-20 14:39:52,850:     db.authenticate(conn_settings['username'], conn_settings['password'], **auth_kwargs)
2018-05-20 14:39:52,850: 
2018-05-20 14:39:52,850:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/database.py", line 1167, in authenticate
2018-05-20 14:39:52,850:     connect=True)
2018-05-20 14:39:52,850: 
2018-05-20 14:39:52,851:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 583, in _cache_credentials
2018-05-20 14:39:52,851:     writable_preferred_server_selector)
2018-05-20 14:39:52,851: 
2018-05-20 14:39:52,851:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 224, in select_server
2018-05-20 14:39:52,851:     address))
2018-05-20 14:39:52,851: 
2018-05-20 14:39:52,851:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 183, in select_servers
2018-05-20 14:39:52,852:     selector, server_timeout, address)
2018-05-20 14:39:52,852: 
2018-05-20 14:39:52,852:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 199, in _select_servers_loop
2018-05-20 14:39:52,852:     self._error_message(selector))
2018-05-20 14:40:22,177: Error running WSGI application
2018-05-20 14:40:22,184: ServerSelectionTimeoutError: No servers found yet
2018-05-20 14:40:22,185:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
2018-05-20 14:40:22,185:     return self.wsgi_app(environ, start_response)
2018-05-20 14:40:22,185: 
2018-05-20 14:40:22,185:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
2018-05-20 14:40:22,185:     response = self.handle_exception(e)
2018-05-20 14:40:22,186: 
2018-05-20 14:40:22,186:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception
2018-05-20 14:40:22,186:     reraise(exc_type, exc_value, tb)
2018-05-20 14:40:22,186: 
2018-05-20 14:40:22,187:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
2018-05-20 14:40:22,187:     response = self.full_dispatch_request()
2018-05-20 14:40:22,187: 
2018-05-20 14:40:22,187:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
2018-05-20 14:40:22,187:     rv = self.handle_user_exception(e)
2018-05-20 14:40:22,187: 
2018-05-20 14:40:22,187:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
2018-05-20 14:40:22,187:     reraise(exc_type, exc_value, tb)
2018-05-20 14:40:22,187: 
2018-05-20 14:40:22,188:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
2018-05-20 14:40:22,188:     rv = self.dispatch_request()
2018-05-20 14:40:22,188: 
2018-05-20 14:40:22,188:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
2018-05-20 14:40:22,188:     return self.view_functions[rule.endpoint](**req.view_args)
2018-05-20 14:40:22,188: 
2018-05-20 14:40:22,188:   File "./flask_app.py", line 23, in index
2018-05-20 14:40:22,188:     for listing in Listing.objects().fields(date_str=1).order_by('-date_str').limit(1):
2018-05-20 14:40:22,189: 
2018-05-20 14:40:22,189:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/queryset/manager.py", line 37, in __get__
2018-05-20 14:40:22,189:     queryset = queryset_class(owner, owner._get_collection())
2018-05-20 14:40:22,189: 
2018-05-20 14:40:22,189:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/document.py", line 192, in _get_collection
2018-05-20 14:40:22,190:     db = cls._get_db()
2018-05-20 14:40:22,190: 
2018-05-20 14:40:22,190:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/document.py", line 181, in _get_db
2018-05-20 14:40:22,190:     return get_db(cls._meta.get('db_alias', DEFAULT_CONNECTION_NAME))
2018-05-20 14:40:22,190: 
2018-05-20 14:40:22,190:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/mongoengine/connection.py", line 235, in get_db
2018-05-20 14:40:22,190:     db.authenticate(conn_settings['username'], conn_settings['password'], **auth_kwargs)
2018-05-20 14:40:22,191: 
2018-05-20 14:40:22,191:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/database.py", line 1167, in authenticate
2018-05-20 14:40:22,191:     connect=True)
2018-05-20 14:40:22,191: 
2018-05-20 14:40:22,191:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 583, in _cache_credentials
2018-05-20 14:40:22,191:     writable_preferred_server_selector)
2018-05-20 14:40:22,191: 
2018-05-20 14:40:22,191:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 224, in select_server
2018-05-20 14:40:22,192:     address))
2018-05-20 14:40:22,192: 
2018-05-20 14:40:22,192:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 183, in select_servers
2018-05-20 14:40:22,192:     selector, server_timeout, address)
2018-05-20 14:40:22,192: 
2018-05-20 14:40:22,192:   File "/home/tingi/.virtualenvs/scrapy27/local/lib/python2.7/site-packages/pymongo/topology.py", line 199, in _select_servers_loop
2018-05-20 14:40:22,193:     self._error_message(selector))
Beloved premium user tingi | 7 posts | May 20, 2018, 2:43 p.m. | permalink

Ah, sorry about that! I should have checked the spam queue before posting, you're quite right -- your post with the full traceback was there. Investigating now.

Staff giles | 11800 posts | PythonAnywhere staff | May 20, 2018, 4:15 p.m. | permalink

OK, so the traceback isn't as helpful as I was hoping, but I think what's happening is that Flask is trying to initialize the database connection at the module level, despite the fact that you've put the db.connect inside your view -- perhaps when it does the db.init_app(app). That would make some kind of sense.

So, if my suspicion is correct, what you need to do is put all of the db initialization code into something that's only called post-fork. Something like this might do the job:

db = None
def get_db():
    global db
    if db is None:
        db.init_app(app)
        db.connect(connectTimeoutMS=30000, socketTimeoutMS=None, socketKeepAlive=True, connect=False, maxPoolsize=1)
    return db

...and then, in your views, instead of using db directly, you'd do db = get_db() and then use the local variable.

Staff giles | 11800 posts | PythonAnywhere staff | May 20, 2018, 4:21 p.m. | permalink

Awesome, it worked! Only had to make some small changes. I import db from another script, which is a MongoEngine() class. So when I set it to None, I cannot use its inherited functions anymore, like .init_app(). Instead I use the following code:

from flask_mongoengine import MongoEngine
[...]
db = None

def get_db():
    global db
    if db is None:
        db = MongoEngine()
        db.init_app(app)
        db.connect(connectTimeoutMS=30000, socketTimeoutMS=None, socketKeepAlive=True, connect=False, maxPoolsize=1)
    return db

@app.route("/")
def index():
    db = get_db()
    return render_template('index.html')

Thanks a lot for the help Giles! Small edit: added index view to code

Beloved premium user tingi | 7 posts | May 20, 2018, 4:38 p.m. | permalink

Ah, good point! I missed the import. Glad you got it working :-)

Staff giles | 11800 posts | PythonAnywhere staff | May 20, 2018, 7:09 p.m. | permalink