Forums

Spacy model load preventing Flask API from functioning

Hi all, I am trying to set up a simple website that hosts a spacy text classification model. My issue is that despite the actual model working fine, for whatever reason when it is loaded, any API requests time out. Here's the code of the flask app:

import flask
import os
from flask import Flask, request
import pickle
import spacy
classifier = spacy.load("/home/evankozierok/orpheus-ai/combo")
print(classifier("hello world").cats) // works perfectly fine, prints to server log
# spacy.load('en_core_web_md') # this line when uncommented doesn't break anything

app = Flask(__name__)

random_var = 3.14
@app.route('/stuff', methods=['GET']) # dummy endpoint for testing
def stuff():
    return {'genre': random_var}

@app.route('/predict', methods=['POST'])
def handle_predict():
    lyrics = request.json['lyrics']
    genres = classifier(lyrics)
    return genres


@app.route('/')
def index():
    """ Displays the index page accessible at '/'
    """
    return flask.render_template('index.html')


if __name__ == '__main__':
    app.debug=True
    app.run()

The print line that comes directly after the imports works perfectly well (it's the dictionary of music genres in the server log), but after that both the /stuff and /predict API calls time out instead of doing something. Below is my server log:

2023-04-19 13:41:52 announcing my loyalty to the Emperor...
2023-04-19 13:41:55 announcing my loyalty to the Emperor...
2023-04-19 13:41:58 Wed Apr 19 13:41:58 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /predict (ip 10.0.0.83) !!!
2023-04-19 13:41:58 Wed Apr 19 13:41:58 2023 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /predict (10.0.0.83)
2023-04-19 14:54:21 Wed Apr 19 14:54:21 2023 - received message 0 from emperor
2023-04-19 14:54:21 SIGINT/SIGTERM received...killing workers...
2023-04-19 14:54:36 worker 1 buried after 15 seconds
2023-04-19 14:54:36 worker 2 buried after 15 seconds
2023-04-19 14:54:36 goodbye to uWSGI.
2023-04-19 14:54:36 VACUUM: unix socket /var/sockets/evankozierok.pythonanywhere.com/socket removed.
2023-04-19 14:55:18 *** Starting uWSGI 2.0.20 (64bit) on [Wed Apr 19 14:54:39 2023] ***
2023-04-19 14:55:18 compiled with version: 9.4.0 on 22 July 2022 18:35:26
2023-04-19 14:55:18 os: Linux-5.15.0-1021-aws #25~20.04.1-Ubuntu SMP Thu Sep 22 13:59:08 UTC 2022
2023-04-19 14:55:18 nodename: green-liveweb3
2023-04-19 14:55:18 machine: x86_64
2023-04-19 14:55:18 clock source: unix
2023-04-19 14:55:18 pcre jit disabled
2023-04-19 14:55:18 detected number of CPU cores: 4
2023-04-19 14:55:18 current working directory: /home/evankozierok
2023-04-19 14:55:18 detected binary path: /usr/local/bin/uwsgi
2023-04-19 14:55:18 *** dumping internal routing table ***
2023-04-19 14:55:18 [rule: 0] subject: path_info regexp: \.svgz$ action: addheader:Content-Encoding:gzip
2023-04-19 14:55:18 *** end of the internal routing table ***
2023-04-19 14:55:18 chdir() to /home/evankozierok/
2023-04-19 14:55:18 your processes number limit is 256
2023-04-19 14:55:18 your memory page size is 4096 bytes
2023-04-19 14:55:18 detected max file descriptor number: 123456
2023-04-19 14:55:18 building mime-types dictionary from file /etc/mime.types...
2023-04-19 14:55:18 567 entry found
2023-04-19 14:55:18 lock engine: pthread robust mutexes
2023-04-19 14:55:18 thunder lock: disabled (you can enable it with --thunder-lock)
2023-04-19 14:55:18 uwsgi socket 0 bound to UNIX address /var/sockets/evankozierok.pythonanywhere.com/socket fd 3
2023-04-19 14:55:18 Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0]
2023-04-19 14:55:18 *** Python threads support is disabled. You can enable it with --enable-threads ***
2023-04-19 14:55:18 Python main interpreter initialized at 0x563f1a0ddd20
2023-04-19 14:55:18 your server socket listen backlog is limited to 100 connections
2023-04-19 14:55:18 your mercy for graceful operations on workers is 60 seconds
2023-04-19 14:55:18 setting request body buffering size to 65536 bytes
2023-04-19 14:55:18 mapped 501384 bytes (489 KB) for 2 cores
2023-04-19 14:55:18 *** Operational MODE: preforking ***
2023-04-19 14:55:18 initialized 54 metrics
2023-04-19 14:55:18 2023-04-19 14:54:46.896508: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-04-19 14:55:18 2023-04-19 14:54:46.896530: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2023-04-19 14:55:18 2023-04-19 14:54:57.199686: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2023-04-19 14:55:18 2023-04-19 14:54:57.199709: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2023-04-19 14:55:18 2023-04-19 14:54:57.199737: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (green-liveweb3): /proc/driver/nvidia/version does not exist
2023-04-19 14:55:18 /usr/local/lib/python3.8/dist-packages/torch/autocast_mode.py:162: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling#012  warnings.warn('User provided device_type of \'cuda\', but CUDA is not available. Disabling')
2023-04-19 14:55:18 {'pop': 0.8147894144058228, 'rock': 0.9454762935638428, 'country': 0.34297677874565125, 'folk': 0.0846322625875473, 'jazz': 0.9878403544425964, 'hip hop': 0.2699604630470276, 'rap': 0.6814014911651611, 'soul': 0.061875514686107635, 'blues': 0.5190078616142273, 'metal': 0.8359938263893127}
2023-04-19 14:55:18 WSGI app 0 (mountpoint='') ready in 39 seconds on interpreter 0x563f1a0ddd20 pid: 1 (default app)
2023-04-19 14:55:18 *** uWSGI is running in multiple interpreter mode ***
2023-04-19 14:55:18 gracefully (RE)spawned uWSGI master process (pid: 1)
2023-04-19 14:55:18 spawned uWSGI worker 1 (pid: 19, cores: 1)
2023-04-19 14:55:18 huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
2023-04-19 14:55:18 To disable this warning, you can either:
2023-04-19 14:55:18 #011- Avoid using `tokenizers` before the fork if possible
2023-04-19 14:55:18 #011- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
2023-04-19 14:55:18 spawned 2 offload threads for uWSGI worker 1
2023-04-19 14:55:18 announcing my loyalty to the Emperor...
2023-04-19 14:55:18 spawned uWSGI worker 2 (pid: 22, cores: 1)
2023-04-19 14:55:18 metrics collector thread started
2023-04-19 14:55:18 huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
2023-04-19 14:55:18 To disable this warning, you can either:
2023-04-19 14:55:18 #011- Avoid using `tokenizers` before the fork if possible
2023-04-19 14:55:18 #011- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
2023-04-19 14:55:18 spawned 2 offload threads for uWSGI worker 2
2023-04-19 14:55:23 announcing my loyalty to the Emperor...

Any help is much appreciated!

evankozierok | 2 posts | April 19, 2023, 3:24 p.m. | permalink

It looks like you are trying to use cuda, but there are no GPUs on our machines, so that will not work. Also, I see that there are warnings about forks, so you should probably not run that code in a web app: https://help.pythonanywhere.com/pages/AsyncInWebApps/

Staff glenn | 9498 posts | PythonAnywhere staff | April 19, 2023, 5:28 p.m. | permalink