benchmarks

Spyc vs. syck: Speedier YAML parsing in PHP

This post is mostly pretty pictures. I just ran some really quick benchmarks on a couple of of the YAML parsing options available for PHP. The blue line is the Spyc YAML library. The green line is the syck PECL extension. The yellow line is a standard PHP include(), and is on the chart as a reference point.

The first two charts were tested with a fixed file size and a variable number of iterations. The final chart tracks the change in parsing time as file size increases. Each test was run with 100 iterations, and the file size was doubled at each stage.

The pretty pictures

Average parse time

Total parse time

Total parse time for larger files

More info than you cared to know

  • This is a very informal benchmark. Feel free to supplement it with testing of your own.
  • All benchmarks were run on the same Ubuntu 9.04 virtual machine running PHP 5.2.x.
  • Tests were run in a Zoop Framework skeleton app (from the upcoming Lunar release), since I had one handy and it's really easy to deploy.
  • The YAML benchmarks on the first two charts consist of loading and parsing a ~160 line file (a YAML dump of the default Zoop skeleton configuration).
  • The PHP include benchmark exists only as a reference point. It consists of loading a var_dump()'d version of the same configuration array used in the YAML tests. This file was included via a standard PHP include().
  • The first two sets of tests were run with 1, 50, 100, 250, and 500 iterations, respectively. The final test was run with 100 iterations and variable file size.

A PHP Framework Showdown

I've enjoyed working on the Zoop Framework's Lunar branch, and it's gotten pretty rad in the last several months. I wanted to compare it with some of the other offerings.

For some background, watch Simple is Hard, Rasmus Lerdorf's really long talk at Drupalcon Szeged last year. Make sure to check out the presentation slides for all the gory details. If you just want to see some numbers and pretty charts, see below.

I wrote a quick Hello World in the latest versions of CakePHP, CodeIgniter and Symfony, and compared them with the a skeleton app from Zoop's Lunar branch. I kept each of the apps as vanilla and basic as I could, so they look a lot like the ones Rasmus wrote.

For comparison, I also included a trivial PHP and HTML page with the exact same output

<html>
<head>
<title>Test</title>
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>

The output looks like just like that for each of the frameworks.

Each framework was responsible for duplicating the following PHP snippet

<html>
<head>
<title><?php echo 'Title'; ?></title>
</head>
<body>
<h1><?php echo 'Hello world!'; ?></h1>
</body>
</html>

Here are the good parts of the Zoop app

class zone_default extends Zone {
    function pageIndex() {
        global $gui;
        $gui->assign('title', 'Test');
        $gui->assign('message', 'Hello world!');
        $gui->display('base/hello_world.tpl');
    }
}
<html>
<head>
<title>{$title}</title>
</head>
<body>
<h1>{$message}</h1>
</body>
</html>

This kept the differences to a minimum. All of my tests were done on a virtual machine with PHP 5.2.4, and tested with 15 concurrent connections, using Siege in benchmark mode. These benchmarks essentially measure the minimum overhead of each framework while completing a trivial task. Note that it doesn't address feature sets, functionality, ease of use, available libraries and support, or any of the other reasons you might want to use a framework... That's another blog post for another day.

Here's the big picture

PHP Framework showdown

The HTML and trivial PHP pages are obviously fast. And APC (the blue sections) is worth every cent. It's a pretty chart, too. Raw PHP is twice as fast as CodeIgniter, which is twice as fast as Lunar. Lunar is twice as fast as Symfony, and CakePHP just crawls, even with APC enabled. Scroll to the bottom of this post to see all the numbers, if you're into that sort of thing.

Tweaking Zoop for even better performance after the jump.