Forums

Hi. I have a free account since more than a year. I got installed a web2py app. Recently I have noticed a considerably slowdown of performance. My question is: It has changed the performance/bandwith of the free account plan?

Thanks.

We haven't deliberately changed the performance for free plans for a long time, at least a year, I think. Since when have you seen the slowdown? Are you using an SQLite database that's been growing over time, perhaps? SQLite is a bit slow on our platform (we recommend MySQL for non-testing apps) and I think it does get slower non-linearly as the database size gets larger.

Greetings!

Firstly it was great to find pythonanywhere and the fact that you have 1click install for web2py. As a novice developer I found my two favourites in one place (python and web2py) so thank you for that!

However... My experience with web2py is similar to secatel 's in that it can take seconds to save edits to a model or view (and that's when the commits work as often it just fails with 'communication error'). Similarly, selects or refreshes from the web2py app's user interface can also take inordinate amounts of time to load.

All in all this is making it extremely frustrating to try and work with it and does not bode well for using pythonanywhere for production. I am using sqlite during development (with record count for any table currently less than 30) and had hoped to use it in production but am now extremely reluctant to force populate with hundreds or thousands of records for testing performance.

Additional Information: I am based in Ireland, I have HTTPS turned on for app's user interface and I am accessing as developer and user via VPN service. Even with HTTPS & VPN turned off the experience is the same (have not yet moved static files to the 'fast access' directory but don't see how this will change anything in relation to the problems with code edits).

Is there any chance this is because I am currently on a free account? If I subscribe to a premium account will this change and if so which subscriptions will resolve these issues for me?

[edit] Perhaps there is something fundamentally wrong with how I am accessing or where I am accessing from - an edit to this post took 10 seconds to commit?

That all sounds very odd! It certainly should be much faster than that. I just edited my own post above and it took less than a second to commit the change. I'm based in London, so networkalogically (if there is such a word) we should be getting pretty much the same performance -- the servers are on the US East Coast.

Could you ping www.pythonanywhere.com from your command prompt/terminal just to see if there's some kind of weird latency issue?

With VPN...

$ ping www.pythonanywhere.com  
PING www.pythonanywhere.com (50.19.109.98) 56(84) bytes of data.  
(detailed line removed)  
--- www.pythonanywhere.com ping statistics ---  
19 packets transmitted, 18 received, 5% packet loss, time 18016ms  
rtt min/avg/max/mdev = 121.383/125.151/130.090/2.855 ms

Without VPN... (not sure if should reboot but I didn't)

$ ping www.pythonanywhere.com  
PING www.pythonanywhere.com (50.19.109.98) 56(84) bytes of data.  
(detailed lines removed)  
--- www.pythonanywhere.com ping statistics ---  
12 packets transmitted, 12 received, 0% packet loss, time 11013ms  
rtt min/avg/max/mdev = 113.822/120.917/140.994/7.882 ms

5% packet loss with VPN ON can't be helping!

When I get a chance later I am going to check the performance on another machine, can't help thinking it's time to replace my trusty 11 year old Toshiba laptop :-(

Ouch! Yes, 5% packet loss isn't going to be helping. I'd be interested to hear if switching to another machine helps. Our own site is pretty JavaScript-light, but it's possible that there's something there that's slowing things down.

2nd Machine VPN OFF...

$ ping www.pythonanywhere.com
PING www.pythonanywhere.com (50.19.109.98) 56(84) bytes of data.
--- www.pythonanywhere.com ping statistics ---
15 packets transmitted, 15 received, 0% packet loss, time 23314ms
rtt min/avg/max/mdev = 113.043/116.346/119.977/1.955 ms

This machine is an old desktop machine that is hard wired to the router and the ping shows marginally better response times, more importantly and despite still exhibiting occasional delays and communication errors, the experience from development and user point of view is much improved. (Writing code sitting on a couch using a 60" TV as the screen that is 12 feet away is not recommended! :-)

This prompted me to go back and look again at my laptop... I turned off some browser plugins (adblockers and noscript) and things show a marked improvement so much so that it is now much more acceptable at least for development purposes.

Given that I am currently on a Free Account, this experience probably matches what the pythonanywhere documentation for Beginner Account says We don't offer any guarantees about what bandwidth you'll get, but obviously it'll be enough to be usable and I really need to consider moving up a level to Hacker @ $5/month or better still Web Developer @ $12/month for people who want to use PythonAnywhere to run a normal personal or small-business website

For the record...
In case anyone else is perusing this thread re performance both of my machines are over 10 years old and run on Linux (Bless You Linus!). On the
Desktop [Ubuntu 15.04, need to check the rest] I am running Firefox and on the
Laptop [Ubuntu 12.04 LTS, Intel® Core™2 Duo CPU T5450 @ 1.66GHz × 2, 2Gb RAM, 32Bit ],
I am running both Firefox (for the user's app access) and Chromium (for the development environment).
I hear you swoon in awe at the spec!
I'm trying to remember why I started using Chromium and I think it is because it has better built in support for the web2py site editor)

Thanks for the detailed post! I agree that various browser addons is probably causing the issues here. I would guess that you definitely need to add pythonanywhere and any third party sites we use to the noscript exceptions. Otherwise certain pages may not load properly.

I have changed user name from pplin to phadmin, long story but basically because my euphoria after turning off Firefox plugins was short lived.

The next day performance had returned to depressingly slow despite having Firefox plugins turned off and also realising that I had no plugins to turn off in Chromium (used for development access) so any changes would not have affected development transactions/speed.

So frustrated again, I bit the bullet and decided I would try a premium account. I set up the bare minimum account with 1 Gb hdd space and the difference was instantaneous. No more slow loading of user pages, no more slow responses to code changes and critically it seems the only time web2py comes up with 'communications error' is if I have walked away from the keyboard for a few hours. All day yesterday it was a pleasure to work without the usual delays and failures. So to my mind the performance restriction is by design not accident.

I made an assumption about pythonanywhere (in fairness it's not in the literature that I have seen) that because it uses python so prominently in its name it would also hold certain values and display some of the munificence that is attributed to Guido Van Rossum but this appears not to be the case. The statement that non paying users can expect restricted outbound Internet access from your apps, low CPU/bandwidth seems fair in its intent but in reality it should state restricted outbound Internet access from your apps, llow CPU and unusable bandwidth

This leaves a somewhat sour taste in the mouth. I understand pythonanywhere has to make money and the charge to access a basic premium account is not debilitating but I also think the throttling of performance for free account users is severe in the extreme to the point where it is deliberately unusable.

So I put it to pythonanywhere, is this really the intent?

Hi there -- first things first, sorry about the slow posting of your comment. Our spam-checker picked it up for some strange reason, and we didn't spot the miscategorisation over the weekend. We've submitted the comment to Akismet as "ham" so hopefully it will have adjusted its settings and won't do it again.

It's probably worth explaining the difference between the machine resources provided to different accounts.

• All accounts have unlimited bandwidth available (well, at least as much as the gigabit ethernet on the loadbalancers allow, and subject to bandwidth being used across the service as a whole). The "low bandwidth" offered to free accounts means that we monitor how much you're using, and tell you that you need to upgrade if you're using too much. This is a manual process, there's no code that automatically cuts you off. This may change in the future, but under no circumstances will we reduce the bandwidth available to free accounts in a way that will make them unusable.
• Free accounts get one "worker process" to handle their website. This is generally enough for a testing site (unless you're doing very large amounts of processing on each hit). Some people have very lightweight sites, and they are able to handle even quite large numbers of hits per day on on a free account.
• Free websites "expire" after three months. This means that they stop running, but all you have to do to reactivate them is log in and click a button -- the expiry isn't a way to get people to start paying, but rather a way to stop us from having to run throwaway sites that people have created (for example, just to play with a new web framework) if the people who created them no longer want them. We send the websites' owners an email two weeks in advance so that they can log in to extend the expiry date and avoid having downtime.
• Websites on all accounts get shut down if they have had no requests whatsoever for more than 26 hours. They start up again as soon as they receive a request -- how quickly they start up depends on the framework they're using.

So, to summarise -- we strongly support the values of fairness and giving back to the community that are so important to Python's success. We host several tens of thousands of websites for free, may of which have quite a lot of traffic, and have absolutely no plans to stop doing so, or to impose silly restrictions on them to try to weasel people into paying.

Now, all that said, you clearly were seeing problems with your free account, and we definitely want to help. We weren't restricting it in a sneaky way to try to make you upgrade, but obviously something strange was going on if it started working faster once you upgraded.

Was there anything else you changed as a result of the upgrade? For example, did you upgrade shortly after you switched from one username to another? Different usernames can get assigned to different web servers, and while we're not aware of any specific problems on any specific servers right now, it's possible that the one that pplln was assigned to has some kind of performance issue that our monitoring is missing out on, while phadmin's one doesn't have that problem. The speedup might simply have been because you changed your account name, not because you upgraded.

Also, what are you using for your database? If it's SQLite, that can run slowly on our servers -- disk storage is network-based, which can slow down SQLite's locking code, for example. If, say, you'd switched from SQLite to MySQL, then that would speed things up quite a lot. (And perhaps this could even feed into the username change side of things, if the disk IO was for some reason getting maxed out due to a disk-heavy "neighbour" on the pplln server.)

And a final question -- are you still seeing the improved performance today? Our traffic across the servers as a whole is pretty flat over the course of the week, but conceivably some aspect of it might be lower at the weekend, and so it might simply have got faster because you were trying at the weekend...? That's not an issue we've heard about from anyone else, but there's always a chance.

Answers to those questions would be much appreciated. We want to make the service as good as we possibly can, both for free users and paying ones, and debugging strange issues like this is an important part of that.

Many thanks for the detailed response giles. It goes a long way in removing the sour taste I had in my mouth!

Firstly, the good news is, I have had no problems since subscribing and I have been using it quite a bit in spurts of activity over the last couple of days.

Re the info you provide - thank you,

• All accounts have unlimited bandwidth available...
  This is encouraging but before subscribing to a premium account there were severe issues when submitting requests sometimes waiting minutes for response at the user interface and often getting communications error in the web2py edit tool.

• Free accounts get one "worker process" to handle their website...
  I freely admit I have no idea what this really means but would expect that my small development is firmly in the category Some people have very lightweight sites I am the only user and it is a small db.

• Free websites "expire" after three months...
  Fully understand the requirement (have seen and used it) and appreciate and accept you comments.

• Websites on all accounts get shut down if they have had no requests...
  Was not aware of this and assume this would only impact by way of a slow startup after returning, having been away for 26 hours or more, so would not have been the case in my situation.

Re queries...

• Was there anything else you changed as a result of the upgrade?...
  This is hard to say because I did move the database between accounts and can only see that the current instance has file creation dates of Friday evening (3 days ago). I had upgraded web2py to the latest version available on pplin before moving it and I'm pretty sure was on the latest web2py version by default when I opened it on phadmin. Again I'm not certain but think I spent some time using the phadmin install before deciding to subscribe to premium because performance had not improved. The only changes at my end are as described earlier when trying to isolate the problem (i.e. I removed the Firefox plugins) and performance picked up but was short lived. Everything else is the same at my end and performance is great (with the VPN turned back on but Firefox plugins stilll off).

• Also, what are you using for your database?
  I am using SQLite and understand the encouragement to move to MySQL or other db but at the moment it keeps things simple, it only has a handful of tables and records and besides it is now working well at least while I am developing - I will load it to see what happens but that will be some time away.

• And a final question -- are you still seeing the improved performance today?
  Absolutely - have been using it all afternoon and it is a pleasure to use!

Again I appreciate the detailed response and I hope my answers are of some use despite their ambiguity.

Hmm. I'm wondering if this is the culprit:

Free accounts get one "worker process" to handle their website...

I freely admit I have no idea what this really means but would expect that my small development is firmly in the category Some people have very lightweight sites I am the only user and it is a small db.

Here's why. Requests to a web app are served up on a first-come, first-served basis. Think of it as being like one of those queues you get in banks and some supermarkets, where there's one long queue, and people at the head of the queue go forward to the first available teller or cashier.

As requests come in, they join a queue. The worker processes pull requests off the head of the queue, and process them, then get the next one from the queue, and so on.

If you have only one worker process, then that's the equivalent of one cashier in the analogy. So if processing a request takes a long time, then the queue can start growing, and everyone in the queue (every incoming request) has to wait for a long time.

So, if you have one view in your website that takes a long time to process, then it can make everything else queue up while it's doing its thing.

Once you upgraded, then you wound up with multiple worker processes -- so if you're not hitting the slow view very often, then it may be that one of the processes is handling the slow requests, while all of the other, quicker ones, might be going to the other one.

Does that sound at all plausible?

So to paraphrase, basic accounts have a single transaction queue while premium accounts have multiple queues (I have selected 3) so parallel processing can take place?

• Does that sound at all plausible?
  It's possible, though (and bear in mind that I am not coming from any great technical understanding of the background processing that takes place) I would expect this to become more of an issue when the database is loaded and any search or complex relationship would result in more sizeable processing.

Interestingly I have being using without issue for the last few days until about 30 minutes ago when I was kicked out with 'something went wrong' page which roughly coincides with the first time I gave a test user access to a copy of the database under web2py. Is that to be expected with my account settings?

Your paraphrase is broadly correct. With 3 workers, your web app can be busy processing 2 requests and still be able to handle another one coming in.

In general, the "more sizable" processing shouldn't be an issue as long as you've designed your database and queries reasonably well.

What kind of "Something went wrong" page? Some (502 and 504) indicate that your web app didn't respond in time and those are because your workers may be busy. The other "Unhandled exception" indicates that there was an error processing a request and that it was probably your code that had an issue. If it's the last one, there should be a stacktrace in your error log that will tell you what happened.

• "more sizable" processing shouldn't be an issue as long as you've designed your database and queries reasonably well.
  Absolutely no guarantees on that one, I'm a novice! I had a look at the sql log in web2py today for the first time and was surprised at the transaction counts despite the truly meagre scale of the db, so perhaps there is something there.

• What kind of "Something went wrong" page? Some (502 and 504)
  Sorry didn't keep a note - but would be a good habit to get in to.