Date: Thu, 5 Apr 2012 14:03:01 -0400 From: Arnaud Lacombe <lacombar@gmail.com> To: freebsd-performance@freebsd.org Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Scheduler + IPC performance on FreeBSD 7.4, 8.2, 9.0 and -CURRENT Message-ID: <CACqU3MXOM1WOPkinxfs2YJmGbgx8-gAmUbK4L3epKPg6OpQXAw@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi folks, Over the past months, I ran on a couple of unused box the `hackbench'[HACKBENCH] benchmark used by the Linux folks for tracking down various kind of regression/improvement. `hackbench' is a scheduler + IPC test (socket xor pipe). It creates producers/consumers groups and let a variable quantity of small messages flow happily. Producers and consumers are either processes xor threads. Tested platforms were - Atom D510, Intel, (incomplete) - Core 2 Quad Q9560, Intel - Soekris net5501, AMD (incomplete) - Xeon E5645, Intel (incomplete) - Xeon E5620 (dual package), Intel - Xeon E5-1650 (pending completion) - Vortex86, DMP Tested kernel were: - FreeBSD 7.4-RELEASE - FreeBSD 8.2-RELEASE - FreeBSD 9.0-RC3 and FreeBSD 9.0-RELEASE - FreeBSD 10-CURRENT as of r231573 on the following architecture: - amd64 (if supported, incomplete) - i386 1) DISCLAMER Let me start by pointing out something important: [I] "I am _not_ interested in testing released version FOO with feature BAR enabled, if enabling BAR require a kernel rebuild." All tests for release kernels were made for the kernel shipped officially, it is the developers responsability to decide whether or not enable a feature by default, not mine. If option BAR gives a 20% performance improvement, enable it, don't complain the test to be 20% slower. [II] "I am _not_ interested in altering any hints, tunables or sysctl, unless they prevent the execution of the test." The exception added to the above rule is due to limitation introduced by `kern.threads.max_threads_per_proc' and `kern.ipc.maxpipekva'. Those were respectively set to 8192 and between 16M/64M. note: rule [I] is alleviated for -CURRENT kernels, which were built with the same alteration made to GENERIC during the CURRENT->RELEASE transition (ie. WITNESS and a couple of other option disabled). 2) Tests description `hackbench' has the following tunable: - IPC to use for messaging, either `pipe' or `socket'. - Threading model, either `thread' or `process' - Number of iteration to run - Number of group to create The tests covered all of these adjustments more or less heavily depending on the platform capability. 3) Scripts Test scripts are available in the `master' branch of the git repository at: https://github.com/lacombar/hackbench in the `hackbench/' directory. 4) Results Full results are available in the `runs/*' branches of the GitHub repository. 5) Quick results summary * UP case FreeBSD 9.0 behaves better than FreeBSD 8.2 in process mode, especially with sockets. Results are comparable with thread. 9.0-RC3 shows a 10% hits in thread/socket mode on the LX800, this will need confirmation. Linux is stable and scales linearly in all situation. It is only beaten by FreeBSD 8.2-RELEASE with thread/socket. * MP case These is a pretty bad regression with FreeBSD 9.0 in thread/pipe mode, which scale almost in O(N^2), ending up in way worse performance than FreeBSD 7.4 or 8.2 on the Core 2 Quad. Beside that, it is really difficult to draw a general trends, ie. whether FreeBSD 9.0 behaves better than FreeBSD 8.2, or the other way around. Pretty much all situation arises, FreeBSD 9.0 can beat FreeBSD 8.2 on some workload, behave the same, or be beaten on others. None really scales regularly either. Pretty much every runs shows thresholds where scheduling decision change and/or became erratic. 6) Anticipated question and remarks Q1: "You should truly enable kick-ass feature BAZ in the kernel." R1: "I'm lazy. Do your job as a developer to integrate the feature. If it should be the default, make it the default." Q2: "You should set `kern.vm.whatever' to 42, or enable feature BAZ in the kernel, to get full performance from the Warp engine on Constellation-class starship." R2: "Would you ask Lt. Worf to re-aligh plasma injectors or would ask Lt. Commander La Forge to plan an assault, seriously ?" Q3: "You built the Linux kernel, why can't you rebuild FreeBSD's ?" For a couple of reason: - the Linux kernel does not provide binary release per-se. - the Linux kernel was not the focus of the tests, but merely a comparative of what others-can-do. - I did not tweak the Linux kernel configuration. The kernels configuration tested derived from the `defconfig', with very few amendment[0], mostly about hardware support not enabled by default Q3: "Could you post all the graph ?" R3: I could, but there is really tons of them, so posting a subset of them would be subjective, all the materials is available on the git repository. Q4: "So, how can I get all the graph ?" R4: All you need is git, a posix shell, a couple of utility (find, sort, ...), a recent gnuplot, and a ruby interpreter. Comments and suggestions will be greatly appreciated. - Arnaud [HACKBENCH]: http://people.redhat.com/mingo/cfs-scheduler/tools/hackbench.c [0]: the exact list is: # CONFIG_KERNEL_GZIP is not set CONFIG_KERNEL_XZ=y CONFIG_IKCONFIG=y CONFIG_IKCONFIG_PROC=y # CONFIG_MODULES is not set CONFIG_X86_BIGSMP=y CONFIG_NR_CPUS=32 CONFIG_PATA_IT8213=y CONFIG_PATA_IT821X=y CONFIG_IGB=y CONFIG_IGBVF=y CONFIG_IXGB=y CONFIG_IXGBE=y CONFIG_IXGBEVF=y # CONFIG_EXT3_FS is not set CONFIG_EXT4_FS=y
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACqU3MXOM1WOPkinxfs2YJmGbgx8-gAmUbK4L3epKPg6OpQXAw>