hi folks, this started happening a few minutes ago. basically the app either doesn't respond or has a burst of responses then moments of silence. the culprit in the log files seems to be: pthread_create(): Resource temporarily unavailable [core/utils.c line 3816]
also, hitting my app's url yields a 502 backend error
i tried restarting the app a bunch of times but to cigar.
help?
edited to reflect that i'm also getting 502 backend error
Did you take a look at the errors in your error log?
Yes, I did write "the culprit in the log files is ..." In my post right? :) The details are up there as far as I can find them
Yes, I did write "the culprit in the log files is ..." In my post right? :) The details are up there as far as I can find them
Hi there, can you post more details of the error in your logs?
We don't support threads in web apps. Also, if your app is spawning subprocesses, you'll eventually hit a limit (of 64 IIRC) if they keep trying to start concurrently.
if you need to do lots of heavy/background processing in your app, check out these tips: http://help.pythonanywhere.com/pages/AsyncInWebApps
thanks for your response. to clarify, my app doesn't spawn any threads (that i know of at least). the only recent change i can think of is that i'm using a pip package called natty which calls into java but i don't think it's spawning threads.
as to logs: my error log hasn't seen anything new in hours but my server log is spewing the following every 2 minutes or so:
conrad
|
4232
posts
|
PythonAnywhere staff
|
2017-08-21 18:02:33 *** Starting uWSGI 2.0.14 (64bit) on [Mon Aug 21 18:02:32 2017] ***
2017-08-21 18:02:33 compiled with version: 5.4.0 20160609 on 30 June 2017 17:32:30
2017-08-21 18:02:33 os: Linux-4.4.0-83-generic #106-Ubuntu SMP Mon Jun 26 17:54:43 UTC 2017
2017-08-21 18:02:33 nodename: harry-liveweb7
2017-08-21 18:02:33 machine: x86_64
2017-08-21 18:02:33 clock source: unix
2017-08-21 18:02:33 pcre jit disabled
2017-08-21 18:02:33 detected number of CPU cores: 2
2017-08-21 18:02:33 current working directory: (unreachable)/etc/uwsgi/vassals
2017-08-21 18:02:33 detected binary path: /usr/local/bin/uwsgi
2017-08-21 18:02:33 *** dumping internal routing table ***
2017-08-21 18:02:33 [rule: 0] subject: path_info regexp: \.svgz$ action: addheader:Content-Encoding:gzip
2017-08-21 18:02:33 *** end of the internal routing table ***
2017-08-21 18:02:33 chdir() to /home/shie/
2017-08-21 18:02:33 limiting number of processes to 15...
2017-08-21 18:02:33 your processes number limit is 15
2017-08-21 18:02:33 your memory page size is 4096 bytes
2017-08-21 18:02:33 detected max file descriptor number: 123456
2017-08-21 18:02:33 building mime-types dictionary from file /etc/mime.types...
2017-08-21 18:02:33 536 entry found
2017-08-21 18:02:33 lock engine: pthread robust mutexes
2017-08-21 18:02:33 thunder lock: disabled (you can enable it with --thunder-lock)
2017-08-21 18:02:33 uwsgi socket 0 bound to UNIX address /var/sockets/shie.pythonanywhere.com/socket fd 3
2017-08-21 18:02:33 Python version: 3.4.3 (default, Nov 17 2016, 01:12:14) [GCC 4.8.4]
2017-08-21 18:02:33 Set PythonHome to /home/shie/.virtualenvs/my-virtualenv
2017-08-21 18:02:33 *** Python threads support is disabled. You can enable it with --enable-threads ***
2017-08-21 18:02:33 Python main interpreter initialized at 0x8036b0
2017-08-21 18:02:33 your server socket listen backlog is limited to 100 connections
2017-08-21 18:02:33 your mercy for graceful operations on workers is 60 seconds
2017-08-21 18:02:33 setting request body buffering size to 65536 bytes
2017-08-21 18:02:33 mapped 333952 bytes (326 KB) for 1 cores
2017-08-21 18:02:33 *** Operational MODE: single process ***
2017-08-21 18:02:33 initialized 35 metrics
2017-08-21 18:02:33 OpenJDK 64-Bit Server VM warning:
2017-08-21 18:02:33 Can't detect initial thread stack location - find_vma failed
2017-08-21 18:02:33 WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x8036b0 pid: 4853 (default app)
2017-08-21 18:02:33 *** uWSGI is running in multiple interpreter mode ***
2017-08-21 18:02:33 gracefully (RE)spawned uWSGI master process (pid: 4853)
2017-08-21 18:02:33 spawned uWSGI worker 1 (pid: 4903, cores: 1)
2017-08-21 18:02:33 metrics collector thread started
2017-08-21 18:02:33 pthread_create(): Resource temporarily unavailable [core/utils.c line 3816]
2017-08-21 18:02:33 !!! unable to spawn stats pusher thread !!!
2017-08-21 18:02:33 pthread_create(): Resource temporarily unavailable [core/utils.c line 3816]
2017-08-21 18:02:33 unable to start offload thread 0 for worker 1 !!!
2017-08-21 18:02:33 spawned 0 offload threads for uWSGI worker 1
2017-08-21 18:02:33 chdir(): No such file or directory [core/uwsgi.c line 1590]
2017-08-21 18:02:33 VACUUM: unix socket /var/sockets/shie.pythonanywhere.com/socket removed.
2017-08-21 18:02:34 *** Starting uWSGI 2.0.14 (64bit) on [Mon Aug 21 18:02:33 2017] ***
2017-08-21 18:02:34 compiled with version: 5.4.0 20160609 on 30 June 2017 17:32:30
2017-08-21 18:02:34 os: Linux-4.4.0-83-generic #106-Ubuntu SMP Mon Jun 26 17:54:43 UTC 2017
2017-08-21 18:02:34 nodename: harry-liveweb7
2017-08-21 18:02:34 machine: x86_64
2017-08-21 18:02:34 clock source: unix
2017-08-21 18:02:34 pcre jit disabled
2017-08-21 18:02:34 detected number of CPU cores: 2
2017-08-21 18:02:34 current working directory: (unreachable)/etc/uwsgi/vassals
2017-08-21 18:02:34 detected binary path: /usr/local/bin/uwsgi
2017-08-21 18:02:34 *** dumping internal routing table ***
2017-08-21 18:02:34 [rule: 0] subject: path_info regexp: \.svgz$ action: addheader:Content-Encoding:gzip
2017-08-21 18:02:34 *** end of the internal routing table ***
2017-08-21 18:02:34 chdir() to /home/shie/
2017-08-21 18:02:34 limiting number of processes to 15...
2017-08-21 18:02:34 your processes number limit is 15
2017-08-21 18:02:34 your memory page size is 4096 bytes
2017-08-21 18:02:34 detected max file descriptor number: 123456
2017-08-21 18:02:34 building mime-types dictionary from file /etc/mime.types...
2017-08-21 18:02:34 536 entry found
2017-08-21 18:02:34 lock engine: pthread robust mutexes
2017-08-21 18:02:34 thunder lock: disabled (you can enable it with --thunder-lock)
2017-08-21 18:02:34 uwsgi socket 0 bound to UNIX address /var/sockets/shie.pythonanywhere.com/socket fd 3
2017-08-21 18:02:34 Python version: 3.4.3 (default, Nov 17 2016, 01:12:14) [GCC 4.8.4]
2017-08-21 18:02:34 Set PythonHome to /home/shie/.virtualenvs/my-virtualenv
2017-08-21 18:02:34 *** Python threads support is disabled. You can enable it with --enable-threads ***
2017-08-21 18:02:34 Python main interpreter initialized at 0xc806b0
2017-08-21 18:02:34 your server socket listen backlog is limited to 100 connections
2017-08-21 18:02:34 your mercy for graceful operations on workers is 60 seconds
2017-08-21 18:02:34 setting request body buffering size to 65536 bytes
2017-08-21 18:02:34 mapped 333952 bytes (326 KB) for 1 cores
2017-08-21 18:02:34 *** Operational MODE: single process ***
2017-08-21 18:02:34 initialized 35 metrics
2017-08-21 18:02:34 OpenJDK 64-Bit Server VM warning:
2017-08-21 18:02:34 Can't detect initial thread stack location - find_vma failed
2017-08-21 18:02:34 WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xc806b0 pid: 4906 (default app)
2017-08-21 18:02:34 *** uWSGI is running in multiple interpreter mode ***
2017-08-21 18:02:34 gracefully (RE)spawned uWSGI master process (pid: 4906)
2017-08-21 18:02:34 spawned uWSGI worker 1 (pid: 4944, cores: 1)
2017-08-21 18:02:34 metrics collector thread started
2017-08-21 18:02:34 pthread_create(): Resource temporarily unavailable [core/utils.c line 3816]
2017-08-21 18:02:34 !!! unable to spawn stats pusher thread !!!
2017-08-21 18:02:34 chdir(): No such file or directory [core/uwsgi.c line 1590]
2017-08-21 18:02:34 VACUUM: unix socket /var/sockets/shie.pythonanywhere.com/socket removed.
2017-08-21 18:02:34 pthread_create(): Resource temporarily unavailable [core/utils.c line 3816]
2017-08-21 18:02:34 unable to start offload thread 0 for worker 1 !!!
2017-08-21 18:02:34 spawned 0 offload threads for uWSGI worker 1
[changed by admin: formatting]
sorry for the poor outcome with pasting the log. i'm uncertain how to format it properly
what's the expected resolution time (if at all) for issues like that on pythonanywhere? i'm asking as this has been open for >18 hours i think with no useful input or progress
thank you for any help!
Hi there, we usually try to respond within 24 hours.
Those logs look like your webapp constantly restarting. Are you running some sort of Java code?
Hi Conrad, My app uses a pip module called natty, which in turn calls into java and needs JAVA_HOME specified. is that a problem?
having said that, the code that calls java is not executed until you actually make requests - which i'm not making and bad stuff still happens
A quick look at the natty code indicates that it tries to start up a JVM when you import it. At present, the isolation mechanism that we use on PythonAnywhere does not work with Java.
that's too bad :( natty is a thing of beauty
thank you for confirming
