Forums

Hi, Django allows several type of caching:

• Memcached
• Cache in Db
• Filesystem cache
• Local memory

I think we can't use Memcached on pythonanywhere, which cache do you use and recommend to use for best performances?

Hm. I don't think we have a good answer for that. We don't use any caching ourselves on pythonanywhere, because so much of the site is dynamically generated, and we find that a combination of normal browser caching for static assets, plus the caching/query optimisation that the database does all by itself, is enough for our needs...

What makes you think you need caching?

And/or, maybe some other users have some experience to share?

Is this still the case?

I'm looking at using django-select2 V4 and it seems to assume there will be a cache. And it uses cache settings to control how long lookup results are available.

Rich.

Hi,

Is there someone who does use caching? Can you please shed some light? I'm still a bit confused after reading the django docs. It's pretty straight forward for development using local memory caching, but using caching for production is still a bit unclear:

1. Memcached is the fastest according to the django docs but it seems like a pain to install on windows and even if you mange to install it, you might not be able to use it on PA as kaeria mentioned above
2. The other options seem ok but there is no explanation on how to approach caching when you have 2 settings files (local, production)

Many thanks George

You will prob want to figure out why you are caching before deciding on which level to cache at (in db vs filesystem vs in memory). ie. some stuff won't make sense to be cached on the file system level, some won't make sense to be cached in memory etc.

Thanks bfg. I'm mostly interested in caching a couple of views due to the long time they take to load. I installed django-debug-toolbar found the expensive queries and tried to optimize them according to Django's performance and optimization guidelines. I did see some progress but I'm still unhappy with the load times of these views.

do you know how long it took? if it is say 30s, then I would say you want to cache to disk/to db. This is because if you cache to memory, then each webapp worker would probably need to hit it once (I believe they don't share memory) and also because it won't persist if your webapp restarts

To Rich:

I have exactly the same problem: I want to use django-select2 which requires a cache. Did you find a solution?

Thank you for your help Davy

@jacopsd

Sure, if by "solved" you mean "do nothing" and it seems to work fine.

"caching" takes on several tricky aspects on PAW's system, and one could argue is not needed given the performance specs of the underlying hardware.

I kept the default settings where I could and ignored the rest. And it seems to work fine in the project where I'm using it. I plan to use it again for a few more projects because it worked so well.

Regards, Richard Cooke (rcooke)

@Richard

When I use django-select2 on PA with 3 workers, the select2 boxes often return an error "Results cannot be loaded". I believe this is due to a caching (or lack of caching) issue, because on my dev server there is no pbm.

So I would need a django caching solution on PA.

in DB cache fails because I am using django-tenant-schemas which has an issue as well. And file based caching also didn't work. Maybe permissions issue, still investigating.

It would be better if the django-select2 caching could be disabled.

Or did you manage to have a django cache to work on PA?

Rgds DJ

"Results cannot be loaded" sounds like a config problem. The "dev" server breaks many static server rules in the name of brevity. Meaning its a terrible way to test pre-deployment projects - I have many bite marks on my butt to prove it.

I suggest you dig into the 3 server logs for clues. And/or re-read the "how to deploy static assets" docs in Django and PAW.

YMMV

Static assets are serving fine.

In this case it is not a fixed config issue, because about 60% of the time, select2 box loads fine. I guess when another worker serves a request, it has a cache issue to find back what was stored by a previous worker.

Server logs show "WARNING [django.request:170] Not Found: /select2/fields/auto.json"

Everything points to caching issue I believe.

Deployed "Memcached Cloud" from redislab, and it runs fine now. No more errors.

Monitoring performance closely...

Cool!

Hi,

I wanted to use a sorl-thumbnail package for Django and it needs some sort of key-value store. I'm now trying to set up Redis, but I don't know how to, I'll have to do lots of guesswork probably :) (I'm now looking for clues in this topic: https://www.pythonanywhere.com/forums/topic/11050/)

But maybe there is already a solution for that?

Hi again. I just used a different library, easy-thumbnails, it doesn't require a cache, so it's kind of ok for now :)

I would like to cache some query results in my django app. I haven't deployed yet. But I expect a lot of requests that will need to return filtered results.

Any hint about how to set this up on PA?

I'd suggest monitoring the site's real-world performance before implementing caching -- you may find you don't need it at all, or that the caching is needed on pages that aren't the ones you'd expect. But if and when you do find you need it, Django's built-in caching support is pretty solid. A filesystem cache, in-memory cache, or a database cache should work fine on PythonAnywhere. Memcached unfortunately won't.

To use select2: I have a django cache working fine from Redislabs, without username/pwd.

Now I want to create a new cache on a second site, but the new Redislab cache requires a username/pwd which was not the case before. How do we configure that in django?

This doesn't seem to work either: https://redislabs.com/lp/django-memcached/

Anyone has a working cache, based on Redislabs or other solution?

Thx

Got it working via module django-bmemcached:

So install it:

pip install django-bmemcached

Then in settings.py:

CACHES = {
    'default': {
        'BACKEND': 'django_bmemcached.memcached.BMemcached',
        'LOCATION': ‘<my location>',
        'OPTIONS': {
                    'username': ‘<username>',
                    'password': ‘<pwd>',
        }
    }

}

Thanks for sharing that here, @davy!

Not sure why but django_bmemcached.memcached.BMemcached uses many more open connections to the cache. vs django.core.cache.backends.memcached.MemcachedCache

This may cause "too many open connections" to your cache in case the cache does not provision sufficient connections.

I see you asked about this over email too -- perhaps we can sort it out over that channel, and then either you or we can post any solutions here.

Apparently this post: https://blog.memcachier.com/2018/10/15/django-on-pythonanywhere-tutorial/

is a guide on setting up memcache on PA. Will this work?

@betkrok -- thanks for the confidence, but we're not able to try every tutorial out there! :) It looks like an official tut for MemCachier though so I'd suppose it was working when they were writing it.