Forums

Hi, I have my Django-based app hosted in PA but some jobs take longer than 5' to process, so I need a queue or an scheduled task to avoid restarting after those 5'.

Locally I have my Django project running with Celery and RabbitMQ, and it works perfectly well, but I've seen that PA doesn't support them. Am I right?

The alternative I've read is the one that appears here:

https://help.pythonanywhere.com/pages/AsyncInWebApps

However, if this is the only option I can use, I would be interested in passing directly the task to the tab "Always-on task" from the code, rather than having to create it manually from the collected data. The idea is to automate all the process so that I don't need to manually create the task.

Thanks in advanced,

PPMCLAB

The page you link to is the only way to get that working. However, you only need to create the task once -- it should be a single always-on task that just keeps running all the time, picking up jobs as they're created by your website's code. As you're using Django I'd recommend the Django Background Tasks module that we mention on that page.

If I've misunderstood what you meant about manually creating the task, and instead you were thinking of automating your deployment, then you can also use our API to programmatically create always-on tasks.

Yes, the thing is that I have a python script running back that usually takes longer than 5min, and it has multiple arguments. Those arguments change every time the code is run so I'm afraid I cannot describe an static always-on-task command. The idea is to run the script in a queue until it's done and send the results to a fixed link. The Django Background Tasks descriptions use the @background decorator, but it doesn't seem to be able to execute the code in the back-end as I wish. What is the best way to proceed?

Django Background Tasks can definitely handle tasks with different arguments; what problem are you having when you try to use it?

The problem is that when I create my task from my views.py file and I call it later such:

from background_task import background
if n > 5000: #If n is too large
        @background(schedule=10) #Create background task
        def task_function(arg1): #Testing function
            with open("test.txt","wt") as out:
                out.write(arg1)
        task_function(arg1)

And I run the command:

python manage.py process_tasks

I don't see any file created or any response from the process. It seems that the task is saved in the db.sqlite3 database but it is not executed. But it works when I call the function in synchronous mode like:

task_function.now(arg1)

So I don't get why the code is not executed asyncrhonously while I have the process_tasks command running (like if it was an always-on-task command).

If you're using a sqlite database, make sure that your web app and your command line client are using the same database file and that the file that you're writing to is the same as the one that you're checking: https://help.pythonanywhere.com/pages/NoSuchFileOrDirectory/

The problem is not the file itself, I used it just as an example for testing if the process_tasks command was working. Let's put it this way, from my views.py:

from background_task import background
if n > 5000: #If n is too large
        @background(schedule=10) #Create background task
        def task_function(arg1): #Testing function
            print(arg1)
        task_function("test") #Save it to db

When I execute the web app, if n > 5000, the task is saved in the db.sqlite database (I see the how the modified date is changing on my filesystem), but I don't see any print statement when executing the process_tasks command. In the console I don't see any error message either. I'm concerned about why the code is not being executed (not locally neither on PA).

If your concern is that there's nothing printed into the console while you run process_tasks management command, try running it with log-std option, as docs suggest.