Forums

I'm currently testing a small django webapp that uses Django-Q (as it is recommended here: https://help.pythonanywhere.com/pages/AsyncInWebApps ) to execute a background task. Currently it uses a scheduled task, but I want to eventually upgrade the account and use it as an always on task. Everything works great, except that the Django Q cluster is using up a lot of CPU even when not actively doing anything. I rand 3 tasks, which took about 20 CPUs, and then let the cluster run until it was eventually killed 4 hours later. In that time it used nearly 400 CPUs, even though I did not execute any tasks (and the webapp was not accessed at all). That is not really sustainable with any upgraded account, and while being in the tarpit has not affected me negatively yet (as the task is not time sensitive, as long as it finishes within 10 minutes), I am a bit weary of it becoming a problem down he road.

I implemented the Django Q task by using async_task(task, arg) to start the process inside a view, and configured it to use the default database (SQLite) as message broker in my setings.py file like this:

Q_CLUSTER = {
  'name': 'DjangORM',
  'workers': 1,
  'timeout': 300,
  'retry': 400,
  'queue_limit': 50,
  'max_attempts': 1,
  'bulk': 5,
  'orm': 'default'
}

The scheduled task command is:

cd /home/CosmoAltmann/App && source virtualenvwrapper.sh && workon my-virtualenv && python /home/CosmoAltmann/App/manage.py qcluster

Is there any way to stop the cluster from using so much CPUs while not executing a task?

I'm not sure, but maybe setting guard-cycle to higher number could help https://django-q.readthedocs.io/en/latest/configure.html#guard-cycle