Forums

Hi

I've got a scheduled task running that triggers at every hour. This task has become progressively more complex to the point where I was using all my allocated seconds and ending up in the tarpit regularly. I have since refactored the code and included some internal logging to measure and benchmark performance going forward.

The purpose of the task is to query an SQLite DB, load to a Pandas dataframe, and then iteratively query the pd dataframe and generate plots using pyplot.

Now that I have redeployed the scheduled task, I have noted a significant difference in performance of the code when run in a bash shell vs when run via the scheduled task, with execution time taking 2.5 to 3 x longer when run as past of the scheduled task.

Should there be any substantive difference in performance that accounts for what I am seeing below?

Sample performance logging below.

When run via bash shell:

29-09-2022 23:50:28: Total execution time: 86.1
29-09-2022 23:50:28: SQL time: 1.0
29-09-2022 23:50:28: Pandas time: 1.5
29-09-2022 23:50:28: Plot time: 83.32

When run via scheduled task:

30-09-2022 01:08:38: Total execution time: 180.4
30-09-2022 01:08:38: SQL time: 2.55
30-09-2022 01:08:38: Pandas time: 3.73
30-09-2022 01:08:38: Plot time: 174.13

30-09-2022 02:08:31: Total execution time: 180.4
30-09-2022 02:08:31: SQL time: 1.39
30-09-2022 02:08:31: Pandas time: 3.41
30-09-2022 02:08:31: Plot time: 175.61

30-09-2022 03:08:36: Total execution time: 200.1
30-09-2022 03:08:36: SQL time: 2.19
30-09-2022 03:08:36: Pandas time: 3.3
30-09-2022 03:08:36: Plot time: 194.57

Sometimes, when you run things as a scheduled task and you're in the tarpit, the tarpitting code will not cut in correctly. So if you're getting better performance running it that way it might be because you're getting resources that you should not. The tarpit behaviour in consoles will be the correct performance level given your resource usage.