Forums

One of my apps is not accessible right now, is there any outage happening?

Update: It seems all my requests are leading to:

2016-01-22 16:55:01 spawned 2 offload threads for uWSGI worker 1

2016-01-22 17:00:03 Fri Jan 22 17:00:02 2016 - *** HARAKIRI ON WORKER 1 (pid: 18197, try: 1) ***

2016-01-22 17:00:03 Fri Jan 22 17:00:02 2016 - HARAKIRI !!! worker 1 status !!!

2016-01-22 17:00:03 Fri Jan 22 17:00:02 2016 - HARAKIRI [core 0] 10.169.229.133 - GET /app2/plot? freq=pma1&check_boxes=list,8,10&range=__float__828&output_id=sine_wave_plot& since 1453481701

2016-01-22 17:00:03 Fri Jan 22 17:00:02 2016 - HARAKIRI !!! end of worker 1 status !!!

2016-01-22 17:00:03 DAMN ! worker 1 (pid: 18197) died, killed by signal 9 :( trying respawn ...

2016-01-22 17:00:03 Respawned uWSGI worker 1 (new pid: 18366)

2016-01-22 17:00:03 spawned 2 offload threads for uWSGI worker 1

2016-01-22 17:00:03 Fri Jan 22 17:00:03 2016 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /app2/html?freq=pma1&check_boxes=list,8,10&range=__float__0&output_id=custom_html& (ip 10.169.229.133) !!!

2016-01-22 17:00:03 Fri Jan 22 17:00:03 2016 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 287] during GET /app2/html?freq=pma1&check_boxes=list,8,10&range=__float__0&output_id=custom_html& (10.169.229.133)

2016-01-22 17:00:03 ERROR:root:IOError: write error

2016-01-22 17:00:03 ERROR:root:RuntimeError: generator ignored GeneratorExit 2016-01-22 17:00:03 announcing my loyalty to the Emperor...

Update 2: I figured out what is causing the app to crash. In my app there is some data that is retrieved from a file in my account or from a webapi, when data is retrieved from my files, everything works fine, however whenever it has to contact the API the app crashed all together and I have to manually restart it. I didn't change anything from my app and it was working fine before, any clues on what is happening?

Update 3: Ok, it has nothing to do with pythonanywhere, the service I am calling from the API is offline and somehow the requests don't go ahead, maybe the app crashes because it never releases a worker that is stuck.

No, there are no outages -- the "HARAKIRI" message is telling you that a worker spent a long time (normally a few minutes) trying to process a request, so it was killed. Perhaps the web API you're trying to connect to is down, or is running particularly slowly?

Thanks Giles, that was exactly what was happening, it is fixed now. Just a couple questions, does the app restart automatically after an harakiri, and if so how long does it take?

It should automatically restart within a second or two -- the "announcing my loyalty to the Emperor" is saying that. But from what you wrote earlier, it sounds like even if they are restarting, they're not responding to requests -- is that right?

One possibility -- I see you only have one worker thread associated with your web app. So, if someone hits a view that makes a request to the web API and that takes a while to respond, your app will block until either the API responds, or the worker is killed. It will then work again, right up until someone hits the web API view again, at which point it will block again. If you were trying to debug while someone else happened to be hitting the view that was talking to the broken API, you'd see it continuously locking up.

Thanks again Giles. Yeah I think I may upgrade to have at least another worker.

Probably a good idea. Optimally, if you think the web API is going to be unreliable, you should put some kind of timeout around your calls to it so that your view can respond to the user with a "a service we use is down" kind of error rather than locking up a worker for several minutes and then generating an unhelpful internal server error.