Forums

RecursionError: maximum recursion depth exceeded while calling a Python object

I have a website that allows users to enter large bodies of text, then processes it and returns metrics on that text (https://sappho.pythonanywhere.com/ ). I seem to have hit the limit in how much text can be entered and I would like to understand what is causing that limit and what I can do about it. I can enter about 2600 words into my program and it works fine, but once I have more than that I get a RecursionError: maximum recursion depth exceeded while calling a Python object:

2017-10-08 07:36:07,102: RecursionError: maximum recursion depth exceeded while calling a Python object
2017-10-08 07:36:07,102:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2000, in __call__
2017-10-08 07:36:07,102:     return self.wsgi_app(environ, start_response)
2017-10-08 07:36:07,102: 
2017-10-08 07:36:07,102:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1991, in wsgi_app
2017-10-08 07:36:07,102:     response = self.make_response(self.handle_exception(e))
2017-10-08 07:36:07,103: 
2017-10-08 07:36:07,103:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1567, in handle_exception
2017-10-08 07:36:07,103:     reraise(exc_type, exc_value, tb)
2017-10-08 07:36:07,103: 
2017-10-08 07:36:07,103:   File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 33, in reraise
2017-10-08 07:36:07,103:     raise value
2017-10-08 07:36:07,103: 
2017-10-08 07:36:07,103:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1988, in wsgi_app
2017-10-08 07:36:07,104:     response = self.full_dispatch_request()
2017-10-08 07:36:07,104: 
2017-10-08 07:36:07,104:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1641, in full_dispatch_request
2017-10-08 07:36:07,104:     rv = self.handle_user_exception(e)
2017-10-08 07:36:07,104: 
2017-10-08 07:36:07,104:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1544, in handle_user_exception
2017-10-08 07:36:07,104:     reraise(exc_type, exc_value, tb)
2017-10-08 07:36:07,104: 
2017-10-08 07:36:07,104:   File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 33, in reraise
2017-10-08 07:36:07,105:     raise value
2017-10-08 07:36:07,105: 
2017-10-08 07:36:07,105:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1639, in full_dispatch_request
2017-10-08 07:36:07,105:     rv = self.dispatch_request()
2017-10-08 07:36:07,105: 
2017-10-08 07:36:07,105:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1625, in dispatch_request
2017-10-08 07:36:07,105:     return self.view_functions[rule.endpoint](**req.view_args)
2017-10-08 07:36:07,106: 
2017-10-08 07:36:07,106:   File "/home/sappho/mysite/flask_app.py", line 71, in analyze
2017-10-08 07:36:07,106:     print(results, file=sys.stderr)
2017-10-08 07:36:07,106: 
2017-10-08 07:36:07,106:   File "/usr/lib/python3.5/logging/__init__.py", line 1308, in error
2017-10-08 07:36:07,106:     self._log(ERROR, msg, args, **kwargs)
2017-10-08 07:36:07,106: 
2017-10-08 07:36:07,106:   File "/usr/lib/python3.5/logging/__init__.py", line 1415, in _log
2017-10-08 07:36:07,106:     self.handle(record)
2017-10-08 07:36:07,107: 
2017-10-08 07:36:07,107:   File "/usr/lib/python3.5/logging/__init__.py", line 1425, in handle
2017-10-08 07:36:07,107:     self.callHandlers(record)
2017-10-08 07:36:07,107: 
2017-10-08 07:36:07,107:   File "/usr/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
2017-10-08 07:36:07,107:     hdlr.handle(record)
2017-10-08 07:36:07,107: 
2017-10-08 07:36:07,107:   File "/usr/lib/python3.5/logging/__init__.py", line 855, in handle
2017-10-08 07:36:07,108:     self.emit(record)
2017-10-08 07:36:07,108: 
2017-10-08 07:36:07,108:   File "/usr/lib/python3.5/logging/handlers.py", line 912, in emit
2017-10-08 07:36:07,108:     self.handleError(record)
2017-10-08 07:36:07,108: 
2017-10-08 07:36:07,108:   File "/usr/lib/python3.5/logging/__init__.py", line 907, in handleError
2017-10-08 07:36:07,108:     sys.stderr.write('--- Logging error ---\n')
2017-10-08 07:36:07,108: 
2017-10-08 07:36:07,109:   File "/usr/lib/python3.5/logging/__init__.py", line 1308, in error
2017-10-08 07:36:07,109:     self._log(ERROR, msg, args, **kwargs)
2017-10-08 07:36:07,109: 
2017-10-08 07:36:07,109:   File "/usr/lib/python3.5/logging/__init__.py", line 1415, in _log
2017-10-08 07:36:07,109:     self.handle(record)
2017-10-08 07:36:07,109: 
2017-10-08 07:36:07,109:   File "/usr/lib/python3.5/logging/__init__.py", line 1425, in handle
2017-10-08 07:36:07,110:     self.callHandlers(record)
2017-10-08 07:36:07,110: 
2017-10-08 07:36:07,110:   File "/usr/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
2017-10-08 07:36:07,110:     hdlr.handle(record)

And the same error messages just keep repeating in a loop. However, when I try to run even 100,000 words on my offline version it works perfectly. Is there something I can do to fix this or is this a limitation of the (free) PythonAnywhere infrastructure? If I make my code more efficient, will that allow more text to be entered? I would like to allow about 50 times more text, so I don't think I can make the difference up in efficiencies.

sappho | 9 posts | Oct. 8, 2017, 7:47 a.m. | permalink

Interesting. I don't think it's a limit of a PythonAnywhere free account. I think it might be something about the data that you're printing in this line:

2017-10-08 07:36:07,106:   File "/home/sappho/mysite/flask_app.py", line 71, in analyze
2017-10-08 07:36:07,106:     print(results, file=sys.stderr)

...triggering a bug of some kind in our error logging system.

What exactly are you trying to print? Or, if you don't know exactly what it is (because it's not printing) what do you get if you write it out to a temp file, by replacing the print with something like 

with open("/tmp/mylog", "a") as f:
    f.write("{!r}\n".format(results))
Staff giles | 11800 posts | PythonAnywhere staff | Oct. 8, 2017, 4:56 p.m. | permalink

Line 71 is a call to the part of speech tagger, which is computationally expensive but seems to be working fine when offline. As an example, if you copy and paste the first two chapters of Pride and Prejudice (https://www.gutenberg.org/files/1342/1342-0.txt) (with or without the preamble) the program runs fine. The third chapter also works. But if you try to include the first three chapters it runs into this error (online when online; the offline version works).

sappho | 9 posts | Oct. 9, 2017, 5:34 a.m. | permalink

something very strange is going on there. try changing that line 71 from

print(results, file=sys.stderr)

to

print(repr(results), file=sys.stderr)

and see if that works?

Staff harry | 2710 posts | PythonAnywhere staff | Oct. 9, 2017, 1:51 p.m. | permalink

Thanks for the suggestion. I tried that and got the same result.

2017-10-24 13:12:11,522: RecursionError: maximum recursion depth exceeded while calling a Python object
2017-10-24 13:12:11,522:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2000, in __call__
2017-10-24 13:12:11,522:     return self.wsgi_app(environ, start_response)
2017-10-24 13:12:11,523: 
2017-10-24 13:12:11,523:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1991, in wsgi_app
2017-10-24 13:12:11,523:     response = self.make_response(self.handle_exception(e))
2017-10-24 13:12:11,523: 
2017-10-24 13:12:11,523:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1567, in handle_exception
2017-10-24 13:12:11,523:     reraise(exc_type, exc_value, tb)
2017-10-24 13:12:11,523: 
2017-10-24 13:12:11,523:   File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 33, in reraise
2017-10-24 13:12:11,523:     raise value
2017-10-24 13:12:11,524: 
2017-10-24 13:12:11,524:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1988, in wsgi_app
2017-10-24 13:12:11,524:     response = self.full_dispatch_request()
2017-10-24 13:12:11,524: 
2017-10-24 13:12:11,524:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1641, in full_dispatch_request
2017-10-24 13:12:11,524:     rv = self.handle_user_exception(e)
2017-10-24 13:12:11,524: 
2017-10-24 13:12:11,524:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1544, in handle_user_exception
2017-10-24 13:12:11,524:     reraise(exc_type, exc_value, tb)
2017-10-24 13:12:11,525: 
2017-10-24 13:12:11,525:   File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 33, in reraise
2017-10-24 13:12:11,525:     raise value
2017-10-24 13:12:11,525: 
2017-10-24 13:12:11,525:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1639, in full_dispatch_request
2017-10-24 13:12:11,525:     rv = self.dispatch_request()
2017-10-24 13:12:11,525: 
2017-10-24 13:12:11,525:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1625, in dispatch_request
2017-10-24 13:12:11,525:     return self.view_functions[rule.endpoint](**req.view_args)
2017-10-24 13:12:11,526: 
2017-10-24 13:12:11,526:   File "/home/sappho/mysite/flask_app.py", line 71, in analyze
2017-10-24 13:12:11,526:     print(repr(results), file=sys.stderr)
2017-10-24 13:12:11,526: 
2017-10-24 13:12:11,526:   File "/usr/lib/python3.5/logging/__init__.py", line 1308, in error
2017-10-24 13:12:11,526:     self._log(ERROR, msg, args, **kwargs)
2017-10-24 13:12:11,526: 
2017-10-24 13:12:11,526:   File "/usr/lib/python3.5/logging/__init__.py", line 1415, in _log
2017-10-24 13:12:11,527:     self.handle(record)
2017-10-24 13:12:11,527: 
2017-10-24 13:12:11,527:   File "/usr/lib/python3.5/logging/__init__.py", line 1425, in handle
2017-10-24 13:12:11,527:     self.callHandlers(record)
2017-10-24 13:12:11,527: 
2017-10-24 13:12:11,527:   File "/usr/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
2017-10-24 13:12:11,527:     hdlr.handle(record)
2017-10-24 13:12:11,527: 
2017-10-24 13:12:11,527:   File "/usr/lib/python3.5/logging/__init__.py", line 855, in handle
2017-10-24 13:12:11,528:     self.emit(record)
sappho | 9 posts | Oct. 24, 2017, 1:14 p.m. | permalink

do you think you could explain how the print statement works? ie. you said it's actually computationally intensive- is it because results is lazy/not evaluated until you try to print it?

also some really dumb things to check- are you running the same version of python on PythonAnywhere vs locally? also are all the python libraries the same version? (eg: do you use a virtualenv + requirements.txt installaion)

Staff conrad | 4232 posts | PythonAnywhere staff | Oct. 25, 2017, 10:27 a.m. | permalink

I said it was computationally intensive but that was actually line 71 from the function it calls, not the print statement - my mistake. I played with it some more and couldn't get it to work but once I commented out line 71 it started working perfectly. I'm satisfied with that for now. Thanks for all the help.

sappho | 9 posts | Oct. 25, 2017, 11:56 a.m. | permalink