From owner-freebsd-performance@FreeBSD.ORG Sun Jan 29 03:01:52 2012 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FC91106564A; Sun, 29 Jan 2012 03:01:52 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 28CE88FC08; Sun, 29 Jan 2012 03:01:51 +0000 (UTC) Received: by obcwo16 with SMTP id wo16so4483302obc.13 for ; Sat, 28 Jan 2012 19:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LW10aNSV6wddCctvmyNBbXJ8y1d4jJBO9YzFYHC/6SE=; b=SGJgPMqbtHXhEZhAChLi1TThtVpQQM4mlxILnFlj2i9N9klZqPjytwCAqt87BTtGwK D65pmqSZYaJwOt59UGzcQrGuVnwAyW00S68e9iFcBssdNR6RKp+ZyKfQFoqbK6bxFHti MROUNGyHSR13TzWCXE112dn+CqJEzP1vshGnQ= MIME-Version: 1.0 Received: by 10.182.74.66 with SMTP id r2mr15581811obv.67.1327804685511; Sat, 28 Jan 2012 18:38:05 -0800 (PST) Received: by 10.182.214.70 with HTTP; Sat, 28 Jan 2012 18:38:05 -0800 (PST) In-Reply-To: <4F247975.9050208@FreeBSD.org> References: <4F247975.9050208@FreeBSD.org> Date: Sat, 28 Jan 2012 21:38:05 -0500 Message-ID: From: Mehmet Erol Sanliturk To: Florian Smeets Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: performance@freebsd.org Subject: Re: ULE vs. 4BSD scheduler benchmarks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 03:01:52 -0000 On Sat, Jan 28, 2012 at 5:40 PM, Florian Smeets wrote: > [current@ bcc'ed to get a wider audience, please discuss on performance@] > > Hi, > > in recent times i saw a lot of threads where it was suggested people > should switch from the ULE to the 4BSD scheduler. That got me thinking > and i decided to run a few benchmarks. I looked through all the stuff > Kris and Jeff did a few years ago and tried to follow their example. The > main motivation is however that we (Attilio Rao and I) want to set a > baseline for future reference, mainly for the work that's going on in > the vmcontention branch right now, that is the reason why all tests were > run on head@r229659. All debugging was disabled (WITNESS and friends for > the kernel and MALLOC_PRODUCTION=yes for libc). > > For now i ran 3 different things. MySQL/sysbench, PostgreSQL/pgbench and > pbzip2. > > All software was installed from ports with the default system gcc (gcc > version 4.2.1 20070831 patched [FreeBSD]), with the exception of > PostgreSQL. I created new postgres92-{server,client} ports with a > snapshot of PostgreSQL 9.2dev from 16.01.2012, as a lot of scalability > work was done in PostgreSQL 9.2. > > MySQL version 5.5.20 > sysbench version 0.4.12 > PostgreSQL/pgbench version 9.2dev > PBZIP2 version v1.1.6 > > The machine these test were run on is a 2x4 core Xeon L5310 @ 1.60GHz > with 4GB RAM. Here is the complete topology: > > kern.sched.topology_spec: > > 0, 1, 2, 3, 4, 5, 6, 7 > > > 0, 1, 2, 3 > > > 4, 5, 6, 7 > > > > > > The database benchmarks were all run with a work set that fit into the > configured database memory, so after the warmup phase no disk io was > involved. sysbench was run with 1 million rows, innodb was the engine we > used as Kris work already showed that it scales much better than myisam > (also innodb is the default in MySQL's 5.5 branch). Pgbench was run > using a scaling factor of 100. The connection to the databases was using > a unix socket, also only read only tests were run. > > The input and output files for the pbzip2 test were on tmpfs. > > The results are available in this Google docs spreadsheet, if you scroll > down there are also some nice graphs. > > > https://docs.google.com/spreadsheet/ccc?key=0Ai0N1xDe3uNAdDRxcVFiYjNMSnJWOTZhUWVWWlBlemc > > Over time i will add more benchmarks to the doc (i.e nginx/php-fpm and > so on). I tried to run some nginx benchmarks, but those are limited by > netisr, as i did not find a web server benchmark tool which can use unix > sockets, any suggestions welcome. > > The conclusion right now seems to be that ULE is faster for database > workload, but for strongly CPU-bound workloads 4BSD can be a better > choice. I can provide KTR traces and/or schedgraph output for cases > where 4BSD is better than ULE. > > I want to thank Sean Bruno and Yahoo for setting up / providing the > machines to run these test on, and Attilio for suggestions and his > general helpfulness. > > Florian > > > Please forgive me for the following message : The work reported is so useful that I wish it should be converted to a testing facility applicable by the users with their own work loads and generate data to be analyzed statistically . To explain what I want to say , the following part is written ... In some messages , the people are complaining about quality of FreeBSD . Sometimes , I also expressed a wish about a testing facility usable by the users/installers of the FreeBSD : (1) There is no any hardware database maintained by the installed FreeBSD : In some distributions , at the end of a successful installation , a profile of the hardfware is sent to their hardware compatibility data base . (2) There is NO any Handbook about testing the FreeBSD as a whole or its parts : When it is installed we do not know which parts will work and will not work . As an example , Microsoft is continuously supplying test programs to check whether its new version can be installed on the user computer or not without installing it . Such a facility does not exist for FreeBSD ( to my knowledge ) . My wish is to have programs / scripts to be applied in such a way that if they find a trouble point , they will automatically generate a structured message ( for example , in XML ) to a central FreeBSD data base . The existing parts for testing in SVN are the following : http://svnweb.freebsd.org/base/head/tools/test/ http://svnweb.freebsd.org/base/head/tools/regression/ There is no any guide / scripts to be applied after installation to check the installed system by the users ( to my knowledge ). In Linux world , there are some sites about testing : http://www.linuxtesting.org/ http://sourceforge.net/projects/posixtest/ http://sourceforge.net/projects/ltp/ http://sourceforge.net/projects/nptltracetool/ http://sourceforge.net/projects/tslogparser/ http://lttng.org/ As an example , the following pages may be useful : http://www.mageia.org/en/ http://www.mageia.org/en/contribute/ https://wiki.mageia.org/en/QA_Team https://wiki.mageia.org/en/QA_Team_portal https://wiki.mageia.org/en/Mageia_2_Alpha_2_testing_wanted https://wiki.mageia.org/en/Working_on_pre-release_ISOs https://wiki.mageia.org/en/QA_process_for_testing_upgrades https://wiki.mageia.org/en/QA_process_for_testing_installations https://wiki.mageia.org/en/QA_process_for_validating_updates https://wiki.mageia.org/en/QA_testing_procedures https://wiki.mageia.org/en/Category:QA_procedures https://wiki.mageia.org/en/Bugzilla https://wiki.mageia.org/en/How_to_report_a_bug_properly A similar site / pages may be generated for the FreeBSD . There was a site for OpenSolaris as http://test.opensolaris.org/ but now it is shut down . The following pages are still available : http://hub.opensolaris.org/bin/view/Community+Group+testing/downloads http://dlc.sun.com/osol/test/downloads/current/ For the FreeBSD , there is chicken-egg problem : Due to difficulties of use of FreeBSD , its default settings for desk top users , difficulties about PC-BSD as an alternative desk top , in my opinion , is preventing wide adoption of FreeBSD . Over the years , I am waiting that one day , I will be able to use FreeBSD very fluently without living a night mare of setting parameters for each installation one by one . For example , in FreeBSD 9.0 amd64 Release : GNOME is working with a very slow start in root login , completely unusable in another user login . KDE4 , physically unusable due to waiting to start . Fluxbox is working very well , but it is not for starters . These problems can not be solved by the probable users . In GNOME or KDE pages testing is requested . I do NOT know how to test them : There is no explicit guides / scripts / algorithms to be applied , and reports to be generated . I appreciate works performed by the FreeBSD developers and maintainers with my heartiest thanks . Now , my wish is that , in some way , the user interface of FreeBSD should be improved in a way that , the people will be able to install it and start immediately to work with it . If I can do it myself , I could do it immediately . I am not an expert about operating systems . If I were , I could do the following as a starting point : (1) Monitors and video cards are cheap . Attach to a computer extranous three monitors . In Loader.conf , define these monitors for (i) stdin (ii) stdout (iii) stderr to eliminate necessity of serial console for desk top systems . I studied SVN sources through http://fxr.watson.org/ but , I could not find places to modify to divert the outputs to distinct monitors . (2) When X is starting , it is blanking the screen , and is continuing to load the required window manager and its parts . During that time , the parts are generating a large number of messages at the back ground which they are not visible . There is a necessaity to open a messages window and display messages in that window to track what is going on . (3) When X is starting , the other parts should divert their messages to log files instead of displaying them on the invisible back ground of X . (4) For X , use two monitors : One is primary , and other is for messages . If the above modifications are done , it will be possible to understand the problems and report them correctly . In summary : (1) Prepare a FreeBSD Testing Handbook . (2) By making the above modifications , eliminate necessity of serial console as much as possible for desk top users . (3) Supply scripts / algorithms , and suitable programs for testing and automatic reporting . (4) Apply outcomes of testing to FreeBSD . Improve its easiness of usability . Thnk you very much . Mehmet Erol Sanliturk