Forums

My site runs through Cloudflare, hosted here with PythonAnywhere. I have a common task to crawl my site for an indexer. This keeps timing out with no specific error. I look at my error and server log nothing noticeable besides this:

2023-03-23 10:24:08 Thu Mar 23 10:24:08 2023 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /search-index....

It times out with this message from Cloudflare: The connection to the origin web server was made, but the origin web server timed out before responding. The likely cause is an overloaded background task, database, or application, stressing the resources on your web server. To resolve this, please work with your hosting provider or web development team to free up resources for your database or overloaded application.

What is going on here?

The URL that is being accessed is taking longer than the timeout that CloudFlare have set for making the connection to the site. We have a help page to help you to identify what may be making your site slow: https://help.pythonanywhere.com/pages/MySiteIsSlow/

Funny thing is, some of the work is actually done before the request drops. I have up to 10 workers I will think that should be enough to handle the case.

Figured it out, it is a Cloudflare problem.

Thanks for letting us know.