WoodenBits

Today I got to play with my newly rebuilt VPS slice with shiny new Ubuntu 7.10 on it.

My current setup is apache and mongrels behind it, but wanted to try some new ideas and frameworks that appeared in Railscape in last couple of months.

Switchpipe looked like an interesting way to solve memory problems on small 256M VPS, where I run 3 mongrels and things feel quite a bit cramped with no more space for new rails apps.

I ran a simple performance tests using httperf hitting my server from another VPS on same local network, to eliminate artificial CPU and bandwidth related bottlenecks.

All tests used same "signup" page that is low on database usage, mostly rendering.

Participating frameworks:

• mongrel_rails 1.1.4
• thin 0.7.0
• ebb_rails 0.0.4
• switchpipe 1.0.4
• apache 2.2.4

Ok, here we go...

Naked Mongrel.

Good response rate, around 50 hits/s, but when rate of client requests went higher, it started to leak memory like a sieve, and process size went from 54M to more than 140M, and stayed there after load was long gone. Performance seriously degraded as hit rate went above 100.

Naked Thin.

Even slightly better response rate around 58 hits/second, and no memory leaks! Performance also degraded as hit rate went above 100, but not as bad as in case of mongrel.

Naked Ebb

Well, ebb was able to serve test page at whooping 1.3 hits/second. Not quite expected from server that is all about performance, but understandable given young age of the framework. Hopefully this is caused by some obvious problem that is going to be fixed soon.

Switchpipe managing 1 mongrel or 1 thin

Response rate dropped to about 20, and reason is obvious: switchpipe has loaded 100% of one CPU core, and mongrel/thin slacked at around 20% usage. Good news - no more memory leaks from mongrel any more, it feels much better when protected from overload by somebody.

Switchpipe managing 3 mongrels or 3 thins

Response rate went up to 50, with all 3 mongrels loaded at 20% and switchpipe 100%. Server also degrades nicer on hight load.

Apache proxying to switchpipe managing 3 mongrels or 3 thins

Same as above - apache seems to not introduce new bottlenecks.

Apache working as load balancer in front of 1 mongrel or thin

Pretty much same performance as naked mongrel or thin ( ~50 hits/second), but mongrel does not leak memory anymore and seems to have better degradation behavior under high load than thin.

Apache as a load balancer in front of 2 or 3 mongrels

Best performance of all - around 95 hits/second which degraded to 75 hits/second as load increased. What was interesting is that it did not mattered, 2 or 3 mongrels, performance was same. Maybe limits on the CPU usage of VPS were kicking in, maybe it was because server was low on memory with 3 mongrels (30M f of free RAM, cache included), but there were no benefit in running third mongrel.

Conclusions

Based on this experiment I decided that for my purposes combination of apache, switchpipe and mongrel or thin servers would work the best. It allows me to run as many 'test' rails apps on the box as I please, working around memory issues.

Still, switchpipe seems to be not very suitable for any heavy duty production work. If one of my server is to be slashdotted, the first thing to do is to take its configuration out of switchpipe, and run cluster of 2 mongrels under apache directly.

Also it looks like performance of thin and mongrel is pretty much the same for typical rails page, but thin is behaving better if server is to be used in a standalone fashion.