My webapp is a natural language playlist generator. It takes in a description of a playlist, like:
"midwest emo songs to cry to in the shower because my girlfriend broke up with me"
and converts it into an embedding generated by a NLP transformer model and does recommender system stuff to return songs in a playlist for a user.
My website hangs after the user has submitted their description, and I get a 504 load balancer error after 5 minutes. After tracing where the code hangs, it seems to stop during model.encode(text), which runs the user's query through the ML model to get the embedding.
This code runs no problem on my local machine, and when I run it in the console it also has no problem processing the text through the ML model.
What should I do? Add more workers? Free up space in the program? Let me know.
Below are my server logs after model.encode() is run.
2022-11-26 07:53:26 entered the get embedding function
2022-11-26 07:53:27 announcing my loyalty to the Emperor...
2022-11-26 07:54:11 Sat Nov 26 07:54:10 2022 - *** HARAKIRI ON WORKER 4 (pid: 18, try: 1) ***
2022-11-26 07:54:11 Sat Nov 26 07:54:10 2022 - HARAKIRI !!! worker 4 status !!!
2022-11-26 07:54:11 Sat Nov 26 07:54:10 2022 - HARAKIRI [core 0] 10.0.0.75 - POST / since 1669448649
2022-11-26 07:54:11 Sat Nov 26 07:54:10 2022 - HARAKIRI !!! end of worker 4 status !!!
2022-11-26 07:54:11 DAMN ! worker 4 (pid: 18) died, killed by signal 9 :( trying respawn ...
2022-11-26 07:54:11 Respawned uWSGI worker 4 (new pid: 33)
2022-11-26 07:54:11 spawned 2 offload threads for uWSGI worker 4
2022-11-26 08:03:28 Sat Nov 26 08:03:27 2022 - *** HARAKIRI ON WORKER 3 (pid: 15, try: 1) ***
2022-11-26 08:03:28 Sat Nov 26 08:03:27 2022 - HARAKIRI !!! worker 3 status !!!
2022-11-26 08:03:28 Sat Nov 26 08:03:27 2022 - HARAKIRI [core 0] 10.0.0.75 - POST / since 1669449206
2022-11-26 08:03:28 Sat Nov 26 08:03:27 2022 - HARAKIRI !!! end of worker 3 status !!!
2022-11-26 08:03:28 DAMN ! worker 3 (pid: 15) died, killed by signal 9 :( trying respawn ...
2022-11-26 08:03:28 Respawned uWSGI worker 3 (new pid: 36)
2022-11-26 08:03:28 spawned 2 offload threads for uWSGI worker 3