Forums

Hi there,

I have two scripts effectively polling two different APIs at different intervals and am using the threading module.

I notice that my first script has been running perfectly for over a day now, but my second one just doesn't run beyond the first iteration of the thread (10 seconds) and just stops.

It all works fine locally, so was wondering what I can do differently to make it run on PA? I have a hacker account, and am well below daily CPU usage limits.

The logic is roughly:

Function 1:
Check API 1 at 10 min intervals

Function 2:
Check API 2 at 10 second intervals 
Do stuff if conditions matched

Actually, I am not sure these are actually threads but rather just using the threading.timer() method to repeatedly call my two functions.

Edit - also noticed that my processes are running again but in a very jagged manner not following my timing rules, running every 30 seconds or so pretty randomly?

It's fairly likely that the functions you're using are erroring in some way and you're just not seeing the error. That could be because you're catching all exceptions and not handling them properly or because the errors are not passed out of the threads. The timing gaps may be because each iteration of the timer is taking longer to run than you expect. That may introduce jitter in the start time of the next iteration.

Thank you glenn. Just wanted to make sure it wasn't an issue with me using the timer() method.

I'll look into the code and debug.

Hmm, I looked into it and here's how I'm handling the general case to alert me if something goes wrong:

except Exception as e:
    print(sys.exc_info())
    raise

it seems like my first script is still running well (and is in it's 60,000th iteration), but when I try to re-run the first script in another terminal it still jitters. Am I potentially running out of memory here?

Running out of memory would terminate your process and the limit is pretty high. Unless you're doing something really memory hungry, you're unlikely to hit it.

Try adding some timing records to see where it's getting stuck.

Thanks glenn, just did that and it looks like the timing of execution between functions is basically consistent, which means it's running well. Both functions take about 0.1 seconds each to execute. I started logging the terminal output using

python3.5 WTT.py | tee WTTlog.txt

However, the output in the terminal seems to be delayed a lot which makes me believe it is jittery / something else is going on. Any idea what might be causing this? Just looked into the logged output at WTTlog.txt and it seems quite consistent.

The timing of output to the console is pretty meaningless for any kind of useful diagnostic. There are so many pipes and buffers between you and the program that the timing of output is mostly dependent on them, rather than when they actually executed.

bonjour les amis