Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 06 Apr 2012 14:33:27 +0200
From:      "O. Hartmann" <ohartman@zedat.fu-berlin.de>
To:        Arnaud Lacombe <lacombar@gmail.com>
Cc:        freebsd-performance@freebsd.org, FreeBSD Current <freebsd-current@freebsd.org>
Subject:   Re: Scheduler + IPC performance on FreeBSD 7.4, 8.2, 9.0 and -CURRENT
Message-ID:  <4F7EE297.2000402@zedat.fu-berlin.de>
In-Reply-To: <CACqU3MXOM1WOPkinxfs2YJmGbgx8-gAmUbK4L3epKPg6OpQXAw@mail.gmail.com>
References:  <CACqU3MXOM1WOPkinxfs2YJmGbgx8-gAmUbK4L3epKPg6OpQXAw@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig0453718867B7A11D2983DCD6
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Am 04/05/12 20:03, schrieb Arnaud Lacombe:
> Hi folks,
>=20
> Over the past months, I ran on a couple of unused box the
> `hackbench'[HACKBENCH] benchmark used by the Linux folks for tracking
> down various kind of regression/improvement. `hackbench' is a
> scheduler + IPC test (socket xor pipe). It creates producers/consumers
> groups and let a variable quantity of small messages flow happily.
> Producers and consumers are either processes xor threads.
>=20
> Tested platforms were
>  - Atom D510, Intel, (incomplete)
>  - Core 2 Quad Q9560, Intel
>  - Soekris net5501, AMD (incomplete)
>  - Xeon E5645, Intel (incomplete)
>  - Xeon E5620 (dual package), Intel
>  - Xeon E5-1650 (pending completion)
>  - Vortex86, DMP
>=20
> Tested kernel were:
>  - FreeBSD 7.4-RELEASE
>  - FreeBSD 8.2-RELEASE
>  - FreeBSD 9.0-RC3 and FreeBSD 9.0-RELEASE
>  - FreeBSD 10-CURRENT as of r231573
>=20
> on the following architecture:
>  - amd64 (if supported, incomplete)
>  - i386
>=20
> 1) DISCLAMER
>=20
> Let me start by pointing out something important:
>=20
>  [I] "I am _not_ interested in testing released version FOO with
> feature BAR enabled, if enabling BAR require a kernel rebuild."
>=20
> All tests for release kernels were made for the kernel shipped
> officially, it is the developers responsability to decide whether or
> not enable a feature by default, not mine. If option BAR gives a 20%
> performance improvement, enable it, don't complain the test to be 20%
> slower.
>=20
>  [II] "I am _not_ interested in altering any hints, tunables or
> sysctl, unless they prevent the execution of the test."
>=20
> The exception added to the above rule is due to limitation introduced
> by `kern.threads.max_threads_per_proc' and `kern.ipc.maxpipekva'.
> Those were respectively set to 8192 and between 16M/64M.
>=20
> note: rule [I] is alleviated for -CURRENT kernels, which were built
> with the same alteration made to GENERIC during the CURRENT->RELEASE
> transition (ie. WITNESS and a couple of other option disabled).
>=20
>=20
> 2) Tests description
>=20
> `hackbench' has the following tunable:
>  - IPC to use for messaging, either `pipe' or `socket'.
>  - Threading model, either `thread' or `process'
>  - Number of iteration to run
>  - Number of group to create
>=20
> The tests covered all of these adjustments more or less heavily
> depending on the platform capability.
>=20
>=20
> 3) Scripts
>=20
> Test scripts are available in the `master' branch of the git repository=
 at:
>=20
> https://github.com/lacombar/hackbench
>=20
> in the `hackbench/' directory.
>=20
>=20
> 4) Results
>=20
> Full results are available in the `runs/*' branches of the GitHub repos=
itory.
>=20
>=20
> 5) Quick results summary
>=20
>  * UP case
>=20
> FreeBSD 9.0 behaves better than FreeBSD 8.2 in process mode,
> especially with sockets. Results are comparable with thread. 9.0-RC3
> shows a 10% hits in thread/socket mode on the LX800, this will need
> confirmation.
>=20
> Linux is stable and scales linearly in all situation. It is only
> beaten by FreeBSD 8.2-RELEASE with thread/socket.
>=20
>  * MP case
>=20
> These is a pretty bad regression with FreeBSD 9.0 in thread/pipe mode,
> which scale almost in O(N^2), ending up in way worse performance than
> FreeBSD 7.4 or 8.2 on the Core 2 Quad. Beside that, it is really
> difficult to draw a general trends, ie. whether FreeBSD 9.0 behaves
> better than FreeBSD 8.2, or the other way around. Pretty much all
> situation arises, FreeBSD 9.0 can beat FreeBSD 8.2 on some workload,
> behave the same, or be beaten on others. None really scales regularly
> either. Pretty much every runs shows thresholds where scheduling
> decision change and/or became erratic.
>=20
> 6) Anticipated question and remarks
>=20
> Q1: "You should truly enable kick-ass feature BAZ in the kernel."
> R1: "I'm lazy. Do your job as a developer to integrate the feature. If
> it should be the default, make it the default."
>=20
> Q2: "You should set `kern.vm.whatever' to 42, or enable feature BAZ in
> the kernel, to get full performance from the Warp engine on
> Constellation-class starship."
> R2: "Would you ask Lt. Worf to re-aligh plasma injectors or would ask
> Lt. Commander La Forge to plan an assault, seriously ?"
>=20
> Q3: "You built the Linux kernel, why can't you rebuild FreeBSD's ?"
> For a couple of reason:
>=20
>  - the Linux kernel does not provide binary release per-se.
>=20
>  - the Linux kernel was not the focus of the tests, but merely a
> comparative of what others-can-do.
>=20
>  - I did not tweak the Linux kernel configuration. The kernels
> configuration tested derived from the `defconfig', with very few
> amendment[0], mostly about hardware support not enabled by default
>=20
> Q3: "Could you post all the graph ?"
> R3: I could, but there is really tons of them, so posting a subset of
> them would be subjective, all the materials is available on the git
> repository.
>=20
> Q4: "So, how can I get all the graph ?"
> R4: All you need is git, a posix shell, a couple of utility (find,
> sort, ...), a recent gnuplot, and a ruby interpreter.
>=20
> Comments and suggestions will be greatly appreciated.
>=20
>  - Arnaud
>=20
> [HACKBENCH]: http://people.redhat.com/mingo/cfs-scheduler/tools/hackben=
ch.c
>=20
> [0]: the exact list is:
>=20
> # CONFIG_KERNEL_GZIP is not set
> CONFIG_KERNEL_XZ=3Dy
> CONFIG_IKCONFIG=3Dy
> CONFIG_IKCONFIG_PROC=3Dy
> # CONFIG_MODULES is not set
> CONFIG_X86_BIGSMP=3Dy
> CONFIG_NR_CPUS=3D32
> CONFIG_PATA_IT8213=3Dy
> CONFIG_PATA_IT821X=3Dy
> CONFIG_IGB=3Dy
> CONFIG_IGBVF=3Dy
> CONFIG_IXGB=3Dy
> CONFIG_IXGBE=3Dy
> CONFIG_IXGBEVF=3Dy
> # CONFIG_EXT3_FS is not set
> CONFIG_EXT4_FS=3Dy

Thank you very much for this work!


--------------enig0453718867B7A11D2983DCD6
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (FreeBSD)

iQEcBAEBAgAGBQJPfuKdAAoJEOgBcD7A/5N8qTEIAMduemjFoPHfNvfrEpF5pE0w
sdKXOtKm59QEQ58NdJlR3s5AzY147tmhgm8ESSczc7LnFFSze530dlKxcMorSVgC
zbQT3CbTtdi5ZlDsroX9qRcKT2gVYGgnFMSnN9HLF55YjveUHr1xgfyzEWLRB0LC
oqVQKrnP5md2M+mhVAKI6WdQU8yvs+odNN/Wd5igG/G5IppNneeJDli4+00sm3Np
Pyx/SFasxRhQPhHbnpanHU2KZb2g4VvHeMRFV1daJ1e5z/h9qu3SpZcaNMf0BhsS
L1T9DGjDso88pE+MSkNdccT8he5Mlatj1ML66m6//D6CicVH1svvnfI0rnDPlig=
=z50+
-----END PGP SIGNATURE-----

--------------enig0453718867B7A11D2983DCD6--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F7EE297.2000402>