A PHP framework showdown
I’ve enjoyed working on the Zoop Framework’s Lunar branch, and it’s gotten pretty rad in the last several months. I wanted to compare it with some of the other offerings.
For some background, watch Simple is Hard, Rasmus Lerdorf’s really long talk at Drupalcon Szeged last year. Make sure to check out the presentation slides for all the gory details. If you just want to see some numbers and pretty charts, see below.
I wrote a quick Hello World in the latest versions of CakePHP, CodeIgniter and Symfony, and compared them with the a skeleton app from Zoop’s Lunar branch. I kept each of the apps as vanilla and basic as I could, so they look a lot like the ones Rasmus wrote.
For comparison, I also included a trivial PHP and HTML page with the exact same output
<html>
<head>
<title>Test</title>
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>The output looks like just like that for each of the frameworks.
Each framework was responsible for duplicating the following PHP snippet
<html>
<head>
<title><?php echo 'Title'; ?></title>
</head>
<body>
<h1><?php echo 'Hello world!'; ?></h1>
</body>
</html>Here are the good parts of the Zoop app
Controller:
<?php
class zone_default extends Zone {
function pageIndex() {
global $gui;
$gui->assign('title', 'Test');
$gui->assign('message', 'Hello world!');
$gui->display('base/hello_world.tpl');
}
}
?>And view:
<html>
<head>
<title>{$title}</title>
</head>
<body>
<h1>{$message}</h1>
</body>
</html>
</code>This kept the differences to a minimum. All of my tests were done on a virtual machine with PHP 5.2.4, and tested with 15 concurrent connections, using Siege in benchmark mode. These benchmarks essentially measure the minimum overhead of each framework while completing a trivial task. Note that it doesn’t address feature sets, functionality, ease of use, available libraries and support, or any of the other reasons you might want to use a framework… That’s another blog post for another day.
Here’s the big picture
The HTML and trivial PHP pages are obviously fast. And APC (the blue sections) is worth every cent. It’s a pretty chart, too. Raw PHP is twice as fast as CodeIgniter, which is twice as fast as Lunar. Lunar is twice as fast as Symfony, and CakePHP just crawls, even with APC enabled. Scroll to the bottom of this post to see all the numbers, if you’re into that sort of thing.
Let’s see if we can do any better
I’m not as experienced with most of these frameworks, so I couldn’t dig in and find the low hanging fruit to optimize. But I’m fairly familiar with Zoop… So I hacked it up a bit. Since this is a really cheezy hello world app, I figured direct PHP output would be quite a bit faster than getting Smarty and templates involved. It turns out that Hello World runs almost twice as fast without Smarty:
Lunar without Smarty templates, APC enabled
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 13848 hits
Availability: 100.00 %
Elapsed time: 30.46 secs
Data transferred: 2.92 MB
Response time: 0.03 secs
Transaction rate: 454.63 trans/sec
Throughput: 0.10 MB/sec
Concurrency: 14.93
Successful transactions: 13863
Failed transactions: 0
Longest transaction: 0.75
Shortest transaction: 0.00
For reference, here’s our Zoop Lunar baseline:
Lunar, vanilla, APC enabled
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 11240 hits
Availability: 100.00 %
Elapsed time: 30.11 secs
Data transferred: 8.82 MB
Response time: 0.04 secs
Transaction rate: 373.30 trans/sec
Throughput: 0.29 MB/sec
Concurrency: 14.87
Successful transactions: 11255
Failed transactions: 0
Longest transaction: 0.61
Shortest transaction: 0.00
I wanted to know if I could take it any further, so I hacked on Zoop core a bit. I completely disabled the view component. Since the controller was outputting the HTML directly, we didn’t need Smarty at all. A couple of quick snips, and Zoop no longer required or initialized the Smarty templating engine. As you can tell, this had a bit more impact:
Lunar minus View components, APC enabled
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 19716 hits
Availability: 100.00 %
Elapsed time: 30.34 secs
Data transferred: 4.15 MB
Response time: 0.02 secs
Transaction rate: 649.84 trans/sec
Throughput: 0.14 MB/sec
Concurrency: 14.91
Successful transactions: 19731
Failed transactions: 0
Longest transaction: 1.35
Shortest transaction: 0.00
And a pretty picture to finish things off
The moral of the story? Lunar’s pretty fast. And it’s getting faster. It might be worth looking into Smarty alternatives, or possibly packaging multiple template engine choices. And during the next several months we’ll keep cleaning things up, optimizing, and making Zoop an even more compelling offering.
The siege results, for your perusal
HTML and trivial PHP
HTML
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 62569 hits
Availability: 100.00 %
Elapsed time: 29.78 secs
Data transferred: 4.60 MB
Response time: 0.00 secs
Transaction rate: 2101.04 trans/sec
Throughput: 0.15 MB/sec
Concurrency: 5.99
Successful transactions: 62578
Failed transactions: 0
Longest transaction: 0.20
Shortest transaction: 0.00
Trivial PHP
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 54941 hits
Availability: 100.00 %
Elapsed time: 30.15 secs
Data transferred: 3.98 MB
Response time: 0.00 secs
Transaction rate: 1822.26 trans/sec
Throughput: 0.13 MB/sec
Concurrency: 8.08
Successful transactions: 54949
Failed transactions: 0
Longest transaction: 0.19
Shortest transaction: 0.00
Trivial PHP with APC
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 55543 hits
Availability: 100.00 %
Elapsed time: 30.41 secs
Data transferred: 4.03 MB
Response time: 0.00 secs
Transaction rate: 1826.47 trans/sec
Throughput: 0.13 MB/sec
Concurrency: 7.77
Successful transactions: 55551
Failed transactions: 0
Longest transaction: 0.30
Shortest transaction: 0.00
CakePHP 1.2.1.8004
CakePHP
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 646 hits
Availability: 100.00 %
Elapsed time: 30.14 secs
Data transferred: 0.06 MB
Response time: 0.69 secs
Transaction rate: 21.43 trans/sec
Throughput: 0.00 MB/sec
Concurrency: 14.75
Successful transactions: 646
Failed transactions: 0
Longest transaction: 1.87
Shortest transaction: 0.38
CakePHP with APC
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 3774 hits
Availability: 100.00 %
Elapsed time: 29.99 secs
Data transferred: 0.37 MB
Response time: 0.12 secs
Transaction rate: 125.84 trans/sec
Throughput: 0.01 MB/sec
Concurrency: 14.95
Successful transactions: 3774
Failed transactions: 0
Longest transaction: 0.77
Shortest transaction: 0.00
CodeIgniter 1.71
CodeIgniter
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 5248 hits
Availability: 100.00 %
Elapsed time: 30.18 secs
Data transferred: 1.08 MB
Response time: 0.09 secs
Transaction rate: 173.89 trans/sec
Throughput: 0.04 MB/sec
Concurrency: 14.84
Successful transactions: 5263
Failed transactions: 0
Longest transaction: 0.43
Shortest transaction: 0.00
CodeIgniter with APC
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 23960 hits
Availability: 100.00 %
Elapsed time: 30.48 secs
Data transferred: 4.89 MB
Response time: 0.02 secs
Transaction rate: 786.09 trans/sec
Throughput: 0.16 MB/sec
Concurrency: 14.86
Successful transactions: 23973
Failed transactions: 0
Longest transaction: 0.52
Shortest transaction: 0.00
Symfony 1.2.4
Symfony
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 1300 hits
Availability: 100.00 %
Elapsed time: 30.36 secs
Data transferred: 0.33 MB
Response time: 0.34 secs
Transaction rate: 42.82 trans/sec
Throughput: 0.01 MB/sec
Concurrency: 14.76
Successful transactions: 1315
Failed transactions: 0
Longest transaction: 1.80
Shortest transaction: 0.00
Symfony with APC
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 5666 hits
Availability: 100.00 %
Elapsed time: 29.98 secs
Data transferred: 1.43 MB
Response time: 0.08 secs
Transaction rate: 188.99 trans/sec
Throughput: 0.05 MB/sec
Concurrency: 14.86
Successful transactions: 5681
Failed transactions: 0
Longest transaction: 0.84
Shortest transaction: 0.00
Zoop Lunar r824
Zoop Lunar
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 2386 hits
Availability: 100.00 %
Elapsed time: 29.88 secs
Data transferred: 0.50 MB
Response time: 0.19 secs
Transaction rate: 79.85 trans/sec
Throughput: 0.02 MB/sec
Concurrency: 14.80
Successful transactions: 2401
Failed transactions: 0
Longest transaction: 0.88
Shortest transaction: 0.00
Zoop Lunar with APC
** SIEGE 2.66
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 11240 hits
Availability: 100.00 %
Elapsed time: 30.11 secs
Data transferred: 8.82 MB
Response time: 0.04 secs
Transaction rate: 373.30 trans/sec
Throughput: 0.29 MB/sec
Concurrency: 14.87
Successful transactions: 11255
Failed transactions: 0
Longest transaction: 0.61
Shortest transaction: 0.00
