Forums

Hi guys,

I am doing some google API requests and use them to get a result. Its about 50 API Calls and these results get stored in a list where i do some statistic operations.

The task takes a while on my local machine ( 5 - 10 mins ) but here it runs into a timeout?

2021-02-10 18:31:14,646: OSError: write error

Error code: 504-loadbalancer

What would be the easiest way to get my program working?

Yes, we've got 5 min timeout set per web app worker. If you need to do some heavy processing in your web app, check this help page -- https://help.pythonanywhere.com/pages/AsyncInWebApps .

Unfortunatly, I need around 10 - 15 mins runtime.

Is there a way to get that? Or is there another provider you work with, which provides that possibly?

The way to achieve that on PythonAnywhere is described on the help page I posted above -- you'd need to upgrade your account though.

The API I use needs around 10 - 15 secs to respond and I have a loop of 40 API requests. I am not sure how to do that with async easily. So I would upgrade my account if it will be possible to extend the timeout limit?

The idea is to separate the request and the processing. You'd need paid account to use Always-on task feature. You can use free Scheduled task feature but it would process your requests only once per day.

Here's a high-level overview of what a solution could look like:

• register the user's request for work somewhere by storing the details of the request somewhere, eg on the filesystem, or in a table in your database

• respond immediately to the user and let them know the request is now in state "pending"

• set up an Always-on task (if you have a paid account). You can use a Scheduled task instead, if you don't have a paid account, but your queue will only be processed when the scheduled task is scheduled. The job of the task is to monitor your task queue (eg the database), and pick jobs off one by one. Include some code to update the job status (eg, pending, under way, complete...)

• give the user a way of checking on the progress of the job, either by asking them to refresh the page, or perhaps setting up an Ajax polling system.Here's a high-level overview of what a solution could look like (everyone's requirements are different, so adapt this to your own neeeds):