WoodenBits

Well, I thought that my new 256MB VPS is really cool, and immediately deployed 3 mongrel clusters, 2 servers in each there, all load balanced by more or less standard apache configuration. Results? The thing was crawling. There were not enough memory to keep buffers and memory cache, so overall performance was miserable.

I realized my mistake and cut down each of those clusters to only 1 server (I know, at this point they are not really clusters, but I decided to keep it this was because it makes integration with capistrano easier, and allows for easy and quick expansion later down the road).

That provided immediate relief, but still... there were still not enough memory. Especially after I stress tested some of those mongrels with httperf, my swap grew to 150M, and things did not looked too good. I looked around on internet, and on some blogs and forums there was some information that 256MB slice was good only for 2 mongrels. Well, I am happy to report that with some tweaking 3 mongrels can be run more or less happily, with some room to spare on 256MB VPS!

Here is what I did.

First step was to realize that even that in idle state apache was not taking too much of memory (4-5k of RSS per process, couple of processes initially), during high load there were 150 of those processes! That quickly ate all the available memory.

Long story short, I added following lines to the bottom of my /etc/apache2/httpd.conf

ServerSignature Off
ServerTokens Prod

StartServers 1
MinSpareServers 1
MaxSpareServers 2
MaxClients 50
MaxRequestsPerChild 500
KeepAliveTimeout 2

Most of this means: do not start more than 50 apache processes at once, restart them after they served 500 requests (to get rid on any possible memory leaks), and do not hold connection to client open after response is served for more that 2 seconds (default is 15).

Now, that helped a LOT. Those settings are described well in other places on internet, I will not repeat that here, but tuning them helped a lot, especially under load.

One more thing that I have done to conserve even more memory is tuning rails. There is a way to tell rails to NOT LOAD frameworks that re not used. Here is how to do it in environment.rb file:

# Skip frameworks you're not going to use (only works if using vendor/rails) config.frameworks -= [ :actionwebservice, :action_mailer ]

Unfortunately, that works only if project is using it's own rails deployment, and it will not work if system-wide rails installation is being used. But, I wanted to use edge rails anyway, so it was a good excuse to do that as well.

Turning off web services in rails brought size on mongrel processes from 45MB RSS memory down to 30MB. Multiply 15MB by 3, and you will see that I gained 45MB of RAM this way!

This is the output of utility that displays amount of free memory on my system, that ran for couple days:

site ~ # free -m
             total       used       free     shared    buffers     cached
Mem:           254        251          3          0         11         40
-/+ buffers/cache:        198         56
Swap:          511          9        502

Not too shabby, right? ;)