Forums

Handling application shut down in the middle of handling a user request

Suppose my application has handled a user request and charged him. As it is about to return a response to the user, my app shuts down either in an expected way (e.g. I disable it to perform maintenance of the database or PA closes for maintenance) or an unexpected one (e.g. power outage). In this case, I would have charged the user without giving him the service he paid for. Is there anything I can do besides manually detecting the affected users and making restitutions to handle such situations gracefully?

Just record that they have paid so that, when they reload the page, they get whatever it was that they paid for.

In my case, the user pays (i.e. credits are subtracted from his balance) for searching a data collection that my application stores in AWS S3 on the user's behalf. When the user refreshes the page, he will not get the search results, because the shutting down of the application would have resulted in the death of the search request.

I don't think that there's any way to avoid some kind of manual element to handling error cases like that, but perhaps you can make it easier to detect. One algorithm that comes to mind would be:

  • Before deducting the credits, log the thing that they are paying for in a database with a state of something like "Pending" and a timestamp
  • When the deduction is make, change the state to something like "Paid"
  • When the state is "Paid", allow the user to make the request.
  • (Optional) if this is a payment for a one-off query, make the state "Compete"

Then a scheduled task looking for requests older than some particular time (depending on your product) that are in the "Pending" or the "Paid" state and notifying you about any would allow you to pick up on any issues like this.

There should be a way at least for the case of expected maintenance, such as when PA announces one in advance...

Sure. The way that Giles suggested would work for that.

I did not understand how the algorithm solved the problem. However, I guess the solution is to have the client submit another request notifying that it has received the response.

For scheduled maintenance, I ended up storing an entry in the database about the next down period and implementing a middleware that returns a special response during that period. This lets me stop accepting requests a few minutes before my app goes down.