Phoronix: “While we have conducted studies related to the Linux kernel performance in the past such as benchmarking up to twelve kernel releases, going out the door this morning are the results from the largest-ever Linux kernel comparison conducted at Phoronix, and very likely the largest ever of its kind regardless of source. Every major Linux kernel release from Linux 2.6.12, which was released in mid-2005, up through the latest Linux 2.6.37 development code was tested. This represents the past five years of the Linux kernel and shows how the performance has evolved over the past 25 stable kernel releases and the most recent 2.6.37 development kernel.
Benchmarking 26 kernels was no easy feat with running nearly two dozen tests each time and each test being run multiple times (usually three to five times as a minimum). Fortunately, with the Phoronix Test Suite combined with an Intel Core i7 “Gulftown” made this process much faster, easier, and more reliable than what would otherwise have been possible. A huge thank you goes out to Intel for supplying Phoronix with the Intel Core i7 970, which is their 32nm Gulftown processor with six physical cores plus Hyper Threading to provide a total of 12 threads. The Core i7 970 has 12MB of L3 cache and is clocked at 3.20GHz while having a maximum turbo frequency of 3.46GHz. This is one very fast desktop processor as shown in our Intel Core i7 970 Linux review and more recently within our LLVMpipe Scaling On Gulftown article where the performance of this Intel LGA-1366 CPU was looked at when running Gallium3D’s LLVMpipe when enabling 1/2/3/4/5/6/12-threads. While the i7-970 is very fast, it’s also very expensive at approximately $900 USD (NewEgg.com and Amazon.com), but it allowed this major Linux kernel comparison to happen in just under a week of constant testing, which is significantly less time than it would have required if using one of the less powerful Intel or AMD CPUs.
With all of this kernel benchmarking being carried out by the Phoronix Test Suite, particularly when using the latest Phoronix Test Suite 3.0 “Iveland” work, benchmarking all of these kernels was not so tedious and it ensures our kernel test results were automated, easily reproducible, and statistically significant with the tests being carried out multiple times and other measures taken plus other recent advancements. As this is also being done with an Iveland snapshot, the new Phoronix Test Suite graphs are being utilized, which provide error bars on graphs where relevant. The set of tests we ran on every major kernel release from Linux 2.6.12 through Linux 2.6.37 Git included GnuPG, Gcrypt, OpenSSL, NASA NAS Parallel Benchmarks, TTSIOD 3D Renderer, C-Ray, Crafty, MAFFT, Himeno, John The Ripper, LAME MP3, 7-Zip, BYTE, Loopback TCP Network Performance, timed Apache compilation, Apache web-server, PostMark, FS-Mark, IOzone, Threaded I/O Tester, and PBZIP2.”