Forums

Hello. I'm trying to setup huey task queue to use it on pythonanywhere.

Following the guid on huey docs I have one file with a task:

# demo.py

from huey import SqliteHuey

huey = SqliteHuey(filename='/tmp/demo.db')

@huey.task()
def add(a, b):
    return a + b

And another file that sends this task to the queue:

 from demo import add
 r = add(1, 2)

Now, when I run the task queue in the separate console with

huey_consumer.py demo.huey

The first run of the file with task works well, but every time after the second run of the file I get the following error:

 2021-05-20 02:48:29,667] ERROR:huey.consumer.Worker:Worker-2:Error reading from queue
 Traceback (most recent call last):
   File "/home/mkondratyev85/.virtualenvs/my-virtualenv/lib/python3.7/site-packages/huey/consumer.py", line       109, in loop
     task = self.huey.dequeue()
   File "/home/mkondratyev85/.virtualenvs/my-virtualenv/lib/python3.7/site-packages/huey/api.py", line 306, in      dequeue
data = self.storage.dequeue()
 File "/home/mkondratyev85/.virtualenvs/my-virtualenv/lib/python3.7/site-packages/huey/storage.py", line 734, in dequeue
'order by priority desc, id limit 1', (self.name,))
 sqlite3.DatabaseError: database disk image is malformed
 [2021-05-20 02:48:30,329] ERROR:huey.consumer.Worker:Worker-1:Error reading from queue

This example works fine on my local computer, but fails on pythonanywhere.

Is there the way to run huey task queue on pythonanywhere?

Do you run it in the web app? Take a look at https://help.pythonanywhere.com/pages/AsyncInWebApps/

No. I run it in two separate consoles. One is for the task itself and another is calling this task. The error is has to do with sqlite3.DatabaseError as you see in the previous message.

sqlite is not designed to work when it's being accessed from multiple processes. Only one process can write to sqlite at a time: https://www.sqlite.org/faq.html#q5

But when I run my example on my local PC it works fine. And this is example is from the guide of the huey package. So the the code I'm trying to run on pythonanywhere is correct.

Your local PC filesystem is much faster than PythonAnywhere one. Better to use MySQL or Postgresql that can easily handle cases like that on PythonAnywhere.