Hi
I've got a scheduled task running that triggers at every hour. This task has become progressively more complex to the point where I was using all my allocated seconds and ending up in the tarpit regularly. I have since refactored the code and included some internal logging to measure and benchmark performance going forward.
The purpose of the task is to query an SQLite DB, load to a Pandas dataframe, and then iteratively query the pd dataframe and generate plots using pyplot.
Now that I have redeployed the scheduled task, I have noted a significant difference in performance of the code when run in a bash shell vs when run via the scheduled task, with execution time taking 2.5 to 3 x longer when run as past of the scheduled task.
Should there be any substantive difference in performance that accounts for what I am seeing below?
Sample performance logging below.
When run via bash shell:
29-09-2022 23:50:28: Total execution time: 86.1
29-09-2022 23:50:28: SQL time: 1.0
29-09-2022 23:50:28: Pandas time: 1.5
29-09-2022 23:50:28: Plot time: 83.32
When run via scheduled task:
30-09-2022 01:08:38: Total execution time: 180.4
30-09-2022 01:08:38: SQL time: 2.55
30-09-2022 01:08:38: Pandas time: 3.73
30-09-2022 01:08:38: Plot time: 174.13
30-09-2022 02:08:31: Total execution time: 180.4
30-09-2022 02:08:31: SQL time: 1.39
30-09-2022 02:08:31: Pandas time: 3.41
30-09-2022 02:08:31: Plot time: 175.61
30-09-2022 03:08:36: Total execution time: 200.1
30-09-2022 03:08:36: SQL time: 2.19
30-09-2022 03:08:36: Pandas time: 3.3
30-09-2022 03:08:36: Plot time: 194.57