From owner-freebsd-performance@FreeBSD.ORG Sat Jan 28 22:40:57 2012 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B004106566C; Sat, 28 Jan 2012 22:40:57 +0000 (UTC) (envelope-from flo@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 631718FC0A; Sat, 28 Jan 2012 22:40:57 +0000 (UTC) Received: from nibbler-wlan.fritz.box (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0SMesYw014466; Sat, 28 Jan 2012 22:40:55 GMT (envelope-from flo@FreeBSD.org) Message-ID: <4F247975.9050208@FreeBSD.org> Date: Sat, 28 Jan 2012 23:40:53 +0100 From: Florian Smeets User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0) Gecko/20120124 Thunderbird/10.0 MIME-Version: 1.0 To: performance@FreeBSD.org X-Enigmail-Version: 1.4a1pre Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig68C92A85B2E237E1C5E2AFBB" X-Mailman-Approved-At: Sun, 29 Jan 2012 00:13:53 +0000 Cc: Subject: ULE vs. 4BSD scheduler benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 22:40:57 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig68C92A85B2E237E1C5E2AFBB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable [current@ bcc'ed to get a wider audience, please discuss on performance@]= Hi, in recent times i saw a lot of threads where it was suggested people should switch from the ULE to the 4BSD scheduler. That got me thinking and i decided to run a few benchmarks. I looked through all the stuff Kris and Jeff did a few years ago and tried to follow their example. The main motivation is however that we (Attilio Rao and I) want to set a baseline for future reference, mainly for the work that's going on in the vmcontention branch right now, that is the reason why all tests were run on head@r229659. All debugging was disabled (WITNESS and friends for the kernel and MALLOC_PRODUCTION=3Dyes for libc). For now i ran 3 different things. MySQL/sysbench, PostgreSQL/pgbench and pbzip2. All software was installed from ports with the default system gcc (gcc version 4.2.1 20070831 patched [FreeBSD]), with the exception of PostgreSQL. I created new postgres92-{server,client} ports with a snapshot of PostgreSQL 9.2dev from 16.01.2012, as a lot of scalability work was done in PostgreSQL 9.2. MySQL version 5.5.20 sysbench version 0.4.12 PostgreSQL/pgbench version 9.2dev PBZIP2 version v1.1.6 The machine these test were run on is a 2x4 core Xeon L5310 @ 1.60GHz with 4GB RAM. Here is the complete topology: kern.sched.topology_spec: 0, 1, 2, 3, 4, 5, 6, 7 0, 1, 2, 3 4, 5, 6, 7 The database benchmarks were all run with a work set that fit into the configured database memory, so after the warmup phase no disk io was involved. sysbench was run with 1 million rows, innodb was the engine we used as Kris work already showed that it scales much better than myisam (also innodb is the default in MySQL's 5.5 branch). Pgbench was run using a scaling factor of 100. The connection to the databases was using a unix socket, also only read only tests were run. The input and output files for the pbzip2 test were on tmpfs. The results are available in this Google docs spreadsheet, if you scroll down there are also some nice graphs. https://docs.google.com/spreadsheet/ccc?key=3D0Ai0N1xDe3uNAdDRxcVFiYjNMSn= JWOTZhUWVWWlBlemc Over time i will add more benchmarks to the doc (i.e nginx/php-fpm and so on). I tried to run some nginx benchmarks, but those are limited by netisr, as i did not find a web server benchmark tool which can use unix sockets, any suggestions welcome. The conclusion right now seems to be that ULE is faster for database workload, but for strongly CPU-bound workloads 4BSD can be a better choice. I can provide KTR traces and/or schedgraph output for cases where 4BSD is better than ULE. I want to thank Sean Bruno and Yahoo for setting up / providing the machines to run these test on, and Attilio for suggestions and his general helpfulness. Florian --------------enig68C92A85B2E237E1C5E2AFBB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAk8keXYACgkQapo8P8lCvwkTIACgg1+ajRlZiWwfc5CVdEDIakk6 BfEAoInUvD5/UI1RWQ2Lg9OYsZLma4O6 =dhNX -----END PGP SIGNATURE----- --------------enig68C92A85B2E237E1C5E2AFBB-- From owner-freebsd-performance@FreeBSD.ORG Sun Jan 29 00:56:57 2012 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id EACE9106564A; Sun, 29 Jan 2012 00:56:57 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-150-251.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 983EB150338; Sun, 29 Jan 2012 00:56:56 +0000 (UTC) Message-ID: <4F249957.6010404@FreeBSD.org> Date: Sat, 28 Jan 2012 16:56:55 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120119 Thunderbird/9.0 MIME-Version: 1.0 To: Florian Smeets References: <4F247975.9050208@FreeBSD.org> In-Reply-To: <4F247975.9050208@FreeBSD.org> X-Enigmail-Version: 1.3.5 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 29 Jan 2012 01:08:04 +0000 Cc: performance@FreeBSD.org Subject: Re: ULE vs. 4BSD scheduler benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 00:56:58 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 01/28/2012 14:40, Florian Smeets wrote: > The results are available in this Google docs spreadsheet, if you > scroll down there are also some nice graphs. > > https://docs.google.com/spreadsheet/ccc?key=0Ai0N1xDe3uNAdDRxcVFiYjNMSnJWOTZhUWVWWlBlemc Can > you convert this to PDF and put it on the web? The research sounds interesting, and is badly needed. Doug - -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJPJJlXAAoJEFzGhvEaGryE0goIAMPI6te9JOL3dfILIztoPPlf f2CPxm/L4lzxeejBIJFAHvlSFSAxj/iTcjriezYU7EGasPDXiBQvT+RTYyKKprU6 LuVdSnEGiq2fXpukGgrONUuGEO63pdMlzHE+F7FOGtoYFhdV8n73zLrc+9QBEXsw fBH5bs0EnoMI4nKfMwfkAV/04em6Zk6R3t/kRqbVIHCA2yIBlIy12KzSG+sfBWR1 p7b4aFrQp/NCOIMN6iLOeMyvpoEnTN9g2fSVM51xjwWMhhIY8KiueADyorDNg/vJ PPsIsmp9wuVqTQoYJyDDTNgZW0kd0RdvLv5Hl6orT+2lll/lSI2tbQ7rVqSpxWY= =/mvx -----END PGP SIGNATURE----- From owner-freebsd-performance@FreeBSD.ORG Sun Jan 29 01:15:12 2012 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F005B106566B; Sun, 29 Jan 2012 01:15:12 +0000 (UTC) (envelope-from flo@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D562B8FC0A; Sun, 29 Jan 2012 01:15:12 +0000 (UTC) Received: from nibbler-wlan.fritz.box (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0T1FBqo062621; Sun, 29 Jan 2012 01:15:12 GMT (envelope-from flo@FreeBSD.org) Message-ID: <4F249D9F.2050700@FreeBSD.org> Date: Sun, 29 Jan 2012 02:15:11 +0100 From: Florian Smeets User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0) Gecko/20120124 Thunderbird/10.0 MIME-Version: 1.0 To: Doug Barton References: <4F247975.9050208@FreeBSD.org> <4F249957.6010404@FreeBSD.org> In-Reply-To: <4F249957.6010404@FreeBSD.org> X-Enigmail-Version: 1.4a1pre Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig21C2DFE6570FC12748426464" X-Mailman-Approved-At: Sun, 29 Jan 2012 02:00:33 +0000 Cc: performance@FreeBSD.org Subject: Re: ULE vs. 4BSD scheduler benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 01:15:13 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig21C2DFE6570FC12748426464 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 29.01.12 01:56, Doug Barton wrote: > On 01/28/2012 14:40, Florian Smeets wrote: >> The results are available in this Google docs spreadsheet, if you >> scroll down there are also some nice graphs. >=20 >> https://docs.google.com/spreadsheet/ccc?key=3D0Ai0N1xDe3uNAdDRxcVFiYjN= MSnJWOTZhUWVWWlBlemc >=20 > Can >=20 > you convert this to PDF and put it on the web? The research sounds > interesting, and is badly needed. >=20 Good idea. Here it is. http://people.freebsd.org/~flo/ule-vs-4bsd-2012.pdf Florian --------------enig21C2DFE6570FC12748426464 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAk8knZ8ACgkQapo8P8lCvwmprwCgn3NEfng/Io0Ybg5wLGGVQFER 4GoAoK1MWV2z+V3rQP6KyzRmO6xsk7ZS =Djko -----END PGP SIGNATURE----- --------------enig21C2DFE6570FC12748426464-- From owner-freebsd-performance@FreeBSD.ORG Sun Jan 29 01:29:19 2012 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E78F1106566B; Sun, 29 Jan 2012 01:29:19 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (pancho.soaustin.net [76.74.250.40]) by mx1.freebsd.org (Postfix) with ESMTP id C60C48FC13; Sun, 29 Jan 2012 01:29:19 +0000 (UTC) Received: by mail.soaustin.net (Postfix, from userid 502) id 6BD635619E; Sat, 28 Jan 2012 19:11:28 -0600 (CST) Date: Sat, 28 Jan 2012 19:11:28 -0600 From: Mark Linimon To: Florian Smeets Message-ID: <20120129011128.GA15206@lonesome.com> References: <4F247975.9050208@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F247975.9050208@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Sun, 29 Jan 2012 02:06:23 +0000 Cc: performance@FreeBSD.org Subject: Re: ULE vs. 4BSD scheduler benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 01:29:20 -0000 Thank you for doing this work. mcl From owner-freebsd-performance@FreeBSD.ORG Sun Jan 29 03:01:52 2012 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FC91106564A; Sun, 29 Jan 2012 03:01:52 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 28CE88FC08; Sun, 29 Jan 2012 03:01:51 +0000 (UTC) Received: by obcwo16 with SMTP id wo16so4483302obc.13 for ; Sat, 28 Jan 2012 19:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LW10aNSV6wddCctvmyNBbXJ8y1d4jJBO9YzFYHC/6SE=; b=SGJgPMqbtHXhEZhAChLi1TThtVpQQM4mlxILnFlj2i9N9klZqPjytwCAqt87BTtGwK D65pmqSZYaJwOt59UGzcQrGuVnwAyW00S68e9iFcBssdNR6RKp+ZyKfQFoqbK6bxFHti MROUNGyHSR13TzWCXE112dn+CqJEzP1vshGnQ= MIME-Version: 1.0 Received: by 10.182.74.66 with SMTP id r2mr15581811obv.67.1327804685511; Sat, 28 Jan 2012 18:38:05 -0800 (PST) Received: by 10.182.214.70 with HTTP; Sat, 28 Jan 2012 18:38:05 -0800 (PST) In-Reply-To: <4F247975.9050208@FreeBSD.org> References: <4F247975.9050208@FreeBSD.org> Date: Sat, 28 Jan 2012 21:38:05 -0500 Message-ID: From: Mehmet Erol Sanliturk To: Florian Smeets Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: performance@freebsd.org Subject: Re: ULE vs. 4BSD scheduler benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 03:01:52 -0000 On Sat, Jan 28, 2012 at 5:40 PM, Florian Smeets wrote: > [current@ bcc'ed to get a wider audience, please discuss on performance@] > > Hi, > > in recent times i saw a lot of threads where it was suggested people > should switch from the ULE to the 4BSD scheduler. That got me thinking > and i decided to run a few benchmarks. I looked through all the stuff > Kris and Jeff did a few years ago and tried to follow their example. The > main motivation is however that we (Attilio Rao and I) want to set a > baseline for future reference, mainly for the work that's going on in > the vmcontention branch right now, that is the reason why all tests were > run on head@r229659. All debugging was disabled (WITNESS and friends for > the kernel and MALLOC_PRODUCTION=yes for libc). > > For now i ran 3 different things. MySQL/sysbench, PostgreSQL/pgbench and > pbzip2. > > All software was installed from ports with the default system gcc (gcc > version 4.2.1 20070831 patched [FreeBSD]), with the exception of > PostgreSQL. I created new postgres92-{server,client} ports with a > snapshot of PostgreSQL 9.2dev from 16.01.2012, as a lot of scalability > work was done in PostgreSQL 9.2. > > MySQL version 5.5.20 > sysbench version 0.4.12 > PostgreSQL/pgbench version 9.2dev > PBZIP2 version v1.1.6 > > The machine these test were run on is a 2x4 core Xeon L5310 @ 1.60GHz > with 4GB RAM. Here is the complete topology: > > kern.sched.topology_spec: > > 0, 1, 2, 3, 4, 5, 6, 7 > > > 0, 1, 2, 3 > > > 4, 5, 6, 7 > > > > > > The database benchmarks were all run with a work set that fit into the > configured database memory, so after the warmup phase no disk io was > involved. sysbench was run with 1 million rows, innodb was the engine we > used as Kris work already showed that it scales much better than myisam > (also innodb is the default in MySQL's 5.5 branch). Pgbench was run > using a scaling factor of 100. The connection to the databases was using > a unix socket, also only read only tests were run. > > The input and output files for the pbzip2 test were on tmpfs. > > The results are available in this Google docs spreadsheet, if you scroll > down there are also some nice graphs. > > > https://docs.google.com/spreadsheet/ccc?key=0Ai0N1xDe3uNAdDRxcVFiYjNMSnJWOTZhUWVWWlBlemc > > Over time i will add more benchmarks to the doc (i.e nginx/php-fpm and > so on). I tried to run some nginx benchmarks, but those are limited by > netisr, as i did not find a web server benchmark tool which can use unix > sockets, any suggestions welcome. > > The conclusion right now seems to be that ULE is faster for database > workload, but for strongly CPU-bound workloads 4BSD can be a better > choice. I can provide KTR traces and/or schedgraph output for cases > where 4BSD is better than ULE. > > I want to thank Sean Bruno and Yahoo for setting up / providing the > machines to run these test on, and Attilio for suggestions and his > general helpfulness. > > Florian > > > Please forgive me for the following message : The work reported is so useful that I wish it should be converted to a testing facility applicable by the users with their own work loads and generate data to be analyzed statistically . To explain what I want to say , the following part is written ... In some messages , the people are complaining about quality of FreeBSD . Sometimes , I also expressed a wish about a testing facility usable by the users/installers of the FreeBSD : (1) There is no any hardware database maintained by the installed FreeBSD : In some distributions , at the end of a successful installation , a profile of the hardfware is sent to their hardware compatibility data base . (2) There is NO any Handbook about testing the FreeBSD as a whole or its parts : When it is installed we do not know which parts will work and will not work . As an example , Microsoft is continuously supplying test programs to check whether its new version can be installed on the user computer or not without installing it . Such a facility does not exist for FreeBSD ( to my knowledge ) . My wish is to have programs / scripts to be applied in such a way that if they find a trouble point , they will automatically generate a structured message ( for example , in XML ) to a central FreeBSD data base . The existing parts for testing in SVN are the following : http://svnweb.freebsd.org/base/head/tools/test/ http://svnweb.freebsd.org/base/head/tools/regression/ There is no any guide / scripts to be applied after installation to check the installed system by the users ( to my knowledge ). In Linux world , there are some sites about testing : http://www.linuxtesting.org/ http://sourceforge.net/projects/posixtest/ http://sourceforge.net/projects/ltp/ http://sourceforge.net/projects/nptltracetool/ http://sourceforge.net/projects/tslogparser/ http://lttng.org/ As an example , the following pages may be useful : http://www.mageia.org/en/ http://www.mageia.org/en/contribute/ https://wiki.mageia.org/en/QA_Team https://wiki.mageia.org/en/QA_Team_portal https://wiki.mageia.org/en/Mageia_2_Alpha_2_testing_wanted https://wiki.mageia.org/en/Working_on_pre-release_ISOs https://wiki.mageia.org/en/QA_process_for_testing_upgrades https://wiki.mageia.org/en/QA_process_for_testing_installations https://wiki.mageia.org/en/QA_process_for_validating_updates https://wiki.mageia.org/en/QA_testing_procedures https://wiki.mageia.org/en/Category:QA_procedures https://wiki.mageia.org/en/Bugzilla https://wiki.mageia.org/en/How_to_report_a_bug_properly A similar site / pages may be generated for the FreeBSD . There was a site for OpenSolaris as http://test.opensolaris.org/ but now it is shut down . The following pages are still available : http://hub.opensolaris.org/bin/view/Community+Group+testing/downloads http://dlc.sun.com/osol/test/downloads/current/ For the FreeBSD , there is chicken-egg problem : Due to difficulties of use of FreeBSD , its default settings for desk top users , difficulties about PC-BSD as an alternative desk top , in my opinion , is preventing wide adoption of FreeBSD . Over the years , I am waiting that one day , I will be able to use FreeBSD very fluently without living a night mare of setting parameters for each installation one by one . For example , in FreeBSD 9.0 amd64 Release : GNOME is working with a very slow start in root login , completely unusable in another user login . KDE4 , physically unusable due to waiting to start . Fluxbox is working very well , but it is not for starters . These problems can not be solved by the probable users . In GNOME or KDE pages testing is requested . I do NOT know how to test them : There is no explicit guides / scripts / algorithms to be applied , and reports to be generated . I appreciate works performed by the FreeBSD developers and maintainers with my heartiest thanks . Now , my wish is that , in some way , the user interface of FreeBSD should be improved in a way that , the people will be able to install it and start immediately to work with it . If I can do it myself , I could do it immediately . I am not an expert about operating systems . If I were , I could do the following as a starting point : (1) Monitors and video cards are cheap . Attach to a computer extranous three monitors . In Loader.conf , define these monitors for (i) stdin (ii) stdout (iii) stderr to eliminate necessity of serial console for desk top systems . I studied SVN sources through http://fxr.watson.org/ but , I could not find places to modify to divert the outputs to distinct monitors . (2) When X is starting , it is blanking the screen , and is continuing to load the required window manager and its parts . During that time , the parts are generating a large number of messages at the back ground which they are not visible . There is a necessaity to open a messages window and display messages in that window to track what is going on . (3) When X is starting , the other parts should divert their messages to log files instead of displaying them on the invisible back ground of X . (4) For X , use two monitors : One is primary , and other is for messages . If the above modifications are done , it will be possible to understand the problems and report them correctly . In summary : (1) Prepare a FreeBSD Testing Handbook . (2) By making the above modifications , eliminate necessity of serial console as much as possible for desk top users . (3) Supply scripts / algorithms , and suitable programs for testing and automatic reporting . (4) Apply outcomes of testing to FreeBSD . Improve its easiness of usability . Thnk you very much . Mehmet Erol Sanliturk From owner-freebsd-performance@FreeBSD.ORG Sun Jan 29 10:12:40 2012 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B7D6106566C for ; Sun, 29 Jan 2012 10:12:40 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id E76BF8FC0C for ; Sun, 29 Jan 2012 10:12:39 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) for performance@FreeBSD.org with esmtp (envelope-from ) id <1RrROz-0000Ey-Nt>; Sun, 29 Jan 2012 10:50:25 +0100 Received: from e178029223.adsl.alicedsl.de ([85.178.29.223] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) for performance@FreeBSD.org with esmtpsa (envelope-from ) id <1RrROz-0007JT-Hj>; Sun, 29 Jan 2012 10:50:25 +0100 Message-ID: <4F25165B.6080805@zedat.fu-berlin.de> Date: Sun, 29 Jan 2012 10:50:19 +0100 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120119 Thunderbird/9.0 MIME-Version: 1.0 To: performance@FreeBSD.org References: <4F247975.9050208@FreeBSD.org> In-Reply-To: <4F247975.9050208@FreeBSD.org> X-Enigmail-Version: 1.3.5 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB5B931BAC01B4C3DE16448CC" X-Originating-IP: 85.178.29.223 Cc: Subject: Re: ULE vs. 4BSD scheduler benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 10:12:40 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB5B931BAC01B4C3DE16448CC Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Am 01/28/12 23:40, schrieb Florian Smeets: > [current@ bcc'ed to get a wider audience, please discuss on performance= @] >=20 > Hi, >=20 > in recent times i saw a lot of threads where it was suggested people > should switch from the ULE to the 4BSD scheduler. That got me thinking > and i decided to run a few benchmarks. I looked through all the stuff > Kris and Jeff did a few years ago and tried to follow their example. Th= e > main motivation is however that we (Attilio Rao and I) want to set a > baseline for future reference, mainly for the work that's going on in > the vmcontention branch right now, that is the reason why all tests wer= e > run on head@r229659. All debugging was disabled (WITNESS and friends fo= r > the kernel and MALLOC_PRODUCTION=3Dyes for libc). >=20 > For now i ran 3 different things. MySQL/sysbench, PostgreSQL/pgbench an= d > pbzip2. >=20 > All software was installed from ports with the default system gcc (gcc > version 4.2.1 20070831 patched [FreeBSD]), with the exception of > PostgreSQL. I created new postgres92-{server,client} ports with a > snapshot of PostgreSQL 9.2dev from 16.01.2012, as a lot of scalability > work was done in PostgreSQL 9.2. >=20 > MySQL version 5.5.20 > sysbench version 0.4.12 > PostgreSQL/pgbench version 9.2dev > PBZIP2 version v1.1.6 >=20 > The machine these test were run on is a 2x4 core Xeon L5310 @ 1.60GHz > with 4GB RAM. Here is the complete topology: >=20 > kern.sched.topology_spec: > > 0, 1, 2, 3, 4, 5, 6, 7 > > > 0, 1, 2, 3 > > > 4, 5, 6, 7 > > > > >=20 > The database benchmarks were all run with a work set that fit into the > configured database memory, so after the warmup phase no disk io was > involved. sysbench was run with 1 million rows, innodb was the engine w= e > used as Kris work already showed that it scales much better than myisam= > (also innodb is the default in MySQL's 5.5 branch). Pgbench was run > using a scaling factor of 100. The connection to the databases was usin= g > a unix socket, also only read only tests were run. >=20 > The input and output files for the pbzip2 test were on tmpfs. >=20 > The results are available in this Google docs spreadsheet, if you scrol= l > down there are also some nice graphs. >=20 > https://docs.google.com/spreadsheet/ccc?key=3D0Ai0N1xDe3uNAdDRxcVFiYjNM= SnJWOTZhUWVWWlBlemc >=20 > Over time i will add more benchmarks to the doc (i.e nginx/php-fpm and > so on). I tried to run some nginx benchmarks, but those are limited by > netisr, as i did not find a web server benchmark tool which can use uni= x > sockets, any suggestions welcome. >=20 > The conclusion right now seems to be that ULE is faster for database > workload, but for strongly CPU-bound workloads 4BSD can be a better > choice. I can provide KTR traces and/or schedgraph output for cases > where 4BSD is better than ULE. >=20 > I want to thank Sean Bruno and Yahoo for setting up / providing the > machines to run these test on, and Attilio for suggestions and his > general helpfulness. >=20 > Florian >=20 Florian, thanks a lot for your efforts. Nice work. We got a new workstation, two socket 6-core westmere XEON's, I forgot the specifications, but they're driven with 2,66 GHz each and have access to 96GB RAM. Maybe I can also setup some benchmarks, but I need advice since I'm not a kernel GURU. The box is prmarily running Linux due to the TESLA/GPGPU stuff we run on it. A colleague of mine developend a software for huge satellite imagery correction needed in planetary science, the software is highly scalable (OpenMP) and massively using OpenCL, but using OpenCL could be switched off. We are not interested in database performance, but more in HPC stuff and scientific calculations. I guess we could provode also some benchmark results after a proper setup for the workload. Since this box in question is also running a Linux Ubuntu 11.04 server, I would be interesting having a comparison to that. I intended to use FreeBSD 9.0-STABLE for benchmarking, but as I can see, it is more applicable to use FreeBSD 10 since development goes on on that branch. Oliver --------------enigB5B931BAC01B4C3DE16448CC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBAgAGBQJPJRZgAAoJEOgBcD7A/5N8eqMH/iruFUnkx6Q92t3SJrh4G3YR kuacxQtVa5f1hOQUUSTS7vfmV60kMY+1lmOcUlo/SALZOqVldd0yuEbR6bhsTmeT ZaXWweOlviG+X7j2FDfUBJeuYqRmM8/oPJBIQfsZuW59bhZHGTBjdwGEqfhuuNWK XvKZupPM3azy+obQJuaQ2vmcIwF5pCfXB1Rl6NvGSDSUqNuDE62LkwCIZcPxjBG1 jd3d0i5yU1u3m7J47/T2WYcrjQY43+JX7OYIikuJYPrk5p1jBSXyUimqnvQ30qaD 4cemOQsWbbMyxU8wSzrEpq7bnoi4sqs0skk+KSNOG6524JzFS5zthA/A5RbAS90= =7oAA -----END PGP SIGNATURE----- --------------enigB5B931BAC01B4C3DE16448CC-- From owner-freebsd-performance@FreeBSD.ORG Mon Jan 30 14:01:36 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4212B1065672 for ; Mon, 30 Jan 2012 14:01:36 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by mx1.freebsd.org (Postfix) with ESMTP id F250D8FC15 for ; Mon, 30 Jan 2012 14:01:35 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Rrrnb-0006Cd-5R for freebsd-performance@freebsd.org; Mon, 30 Jan 2012 15:01:35 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 30 Jan 2012 15:01:35 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 30 Jan 2012 15:01:35 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Mon, 30 Jan 2012 15:01:25 +0100 Lines: 18 Message-ID: References: <4F247975.9050208__14496.2912811481$1327796183$gmane$org@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120110 Thunderbird/9.0 In-Reply-To: <4F247975.9050208__14496.2912811481$1327796183$gmane$org@FreeBSD.org> Subject: Re: ULE vs. 4BSD scheduler benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 14:01:36 -0000 On 28/01/2012 23:40, Florian Smeets wrote: > The conclusion right now seems to be that ULE is faster for database > workload, I've done the same benchmarks with Bullet Cache last year and 4BSD is *ridiculously* inefficient and slow for this specific workload which involves a lot of inter-thread and inter-process communications. The results were somewhere in the ratio of 1:10 in favor of ULE. >but for strongly CPU-bound workloads 4BSD can be a better > choice. I can provide KTR traces and/or schedgraph output for cases > where 4BSD is better than ULE. Can you try manually bind processes to CPUs with the CPU-heavy benchmark? This could be a bit hard if you use the regular pbzip2 because it spawns threads, but if you manually spawn 8 CPU-bound processes (with cpuset(1)) in parallel and measure that, it would be useful.