Date: Sat, 28 Jan 2012 21:38:05 -0500 From: Mehmet Erol Sanliturk <m.e.sanliturk@gmail.com> To: Florian Smeets <flo@freebsd.org> Cc: performance@freebsd.org Subject: Re: ULE vs. 4BSD scheduler benchmarks Message-ID: <CAOgwaMtzu7KfNSTqeUuRheX7PzfK7W3ix2seqbiqszzUpt=RYQ@mail.gmail.com> In-Reply-To: <4F247975.9050208@FreeBSD.org> References: <4F247975.9050208@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Jan 28, 2012 at 5:40 PM, Florian Smeets <flo@freebsd.org> wrote:
> [current@ bcc'ed to get a wider audience, please discuss on performance@]
>
> Hi,
>
> in recent times i saw a lot of threads where it was suggested people
> should switch from the ULE to the 4BSD scheduler. That got me thinking
> and i decided to run a few benchmarks. I looked through all the stuff
> Kris and Jeff did a few years ago and tried to follow their example. The
> main motivation is however that we (Attilio Rao and I) want to set a
> baseline for future reference, mainly for the work that's going on in
> the vmcontention branch right now, that is the reason why all tests were
> run on head@r229659. All debugging was disabled (WITNESS and friends for
> the kernel and MALLOC_PRODUCTION=yes for libc).
>
> For now i ran 3 different things. MySQL/sysbench, PostgreSQL/pgbench and
> pbzip2.
>
> All software was installed from ports with the default system gcc (gcc
> version 4.2.1 20070831 patched [FreeBSD]), with the exception of
> PostgreSQL. I created new postgres92-{server,client} ports with a
> snapshot of PostgreSQL 9.2dev from 16.01.2012, as a lot of scalability
> work was done in PostgreSQL 9.2.
>
> MySQL version 5.5.20
> sysbench version 0.4.12
> PostgreSQL/pgbench version 9.2dev
> PBZIP2 version v1.1.6
>
> The machine these test were run on is a 2x4 core Xeon L5310 @ 1.60GHz
> with 4GB RAM. Here is the complete topology:
>
> kern.sched.topology_spec: <groups>
> <group level="1" cache-level="0">
> <cpu count="8" mask="ff">0, 1, 2, 3, 4, 5, 6, 7</cpu>
> <children>
> <group level="2" cache-level="2">
> <cpu count="4" mask="f">0, 1, 2, 3</cpu>
> </group>
> <group level="2" cache-level="2">
> <cpu count="4" mask="f0">4, 5, 6, 7</cpu>
> </group>
> </children>
> </group>
> </groups>
>
> The database benchmarks were all run with a work set that fit into the
> configured database memory, so after the warmup phase no disk io was
> involved. sysbench was run with 1 million rows, innodb was the engine we
> used as Kris work already showed that it scales much better than myisam
> (also innodb is the default in MySQL's 5.5 branch). Pgbench was run
> using a scaling factor of 100. The connection to the databases was using
> a unix socket, also only read only tests were run.
>
> The input and output files for the pbzip2 test were on tmpfs.
>
> The results are available in this Google docs spreadsheet, if you scroll
> down there are also some nice graphs.
>
>
> https://docs.google.com/spreadsheet/ccc?key=0Ai0N1xDe3uNAdDRxcVFiYjNMSnJWOTZhUWVWWlBlemc
>
> Over time i will add more benchmarks to the doc (i.e nginx/php-fpm and
> so on). I tried to run some nginx benchmarks, but those are limited by
> netisr, as i did not find a web server benchmark tool which can use unix
> sockets, any suggestions welcome.
>
> The conclusion right now seems to be that ULE is faster for database
> workload, but for strongly CPU-bound workloads 4BSD can be a better
> choice. I can provide KTR traces and/or schedgraph output for cases
> where 4BSD is better than ULE.
>
> I want to thank Sean Bruno and Yahoo for setting up / providing the
> machines to run these test on, and Attilio for suggestions and his
> general helpfulness.
>
> Florian
>
>
>
Please forgive me for the following message :
The work reported is so useful that I wish it should be converted to a
testing facility
applicable by the users with their own work loads and generate data to be
analyzed
statistically .
To explain what I want to say , the following part is written ...
In some messages , the people are complaining about quality of FreeBSD .
Sometimes , I also expressed a wish about a testing facility usable by the
users/installers
of the FreeBSD :
(1) There is no any hardware database maintained by the installed FreeBSD :
In some distributions , at the end of a successful installation ,
a profile of the hardfware is sent to their hardware compatibility
data base .
(2) There is NO any Handbook about testing the FreeBSD as a whole or its
parts :
When it is installed we do not know which parts will work and will not
work .
As an example , Microsoft is continuously supplying test programs to
check whether
its new version can be installed on the user computer or not without
installing it .
Such a facility does not exist for FreeBSD ( to my knowledge ) .
My wish is to have programs / scripts to be applied in such a way
that if they find
a trouble point , they will automatically generate a structured
message
( for example , in XML ) to a central FreeBSD data base .
The existing parts for testing in SVN are the following :
http://svnweb.freebsd.org/base/head/tools/test/
http://svnweb.freebsd.org/base/head/tools/regression/
There is no any guide / scripts to be applied after installation to check
the
installed system by the users ( to my knowledge ).
In Linux world , there are some sites about testing :
http://www.linuxtesting.org/
http://sourceforge.net/projects/posixtest/
http://sourceforge.net/projects/ltp/
http://sourceforge.net/projects/nptltracetool/
http://sourceforge.net/projects/tslogparser/
http://lttng.org/
As an example , the following pages may be useful :
http://www.mageia.org/en/
http://www.mageia.org/en/contribute/
https://wiki.mageia.org/en/QA_Team
https://wiki.mageia.org/en/QA_Team_portal
https://wiki.mageia.org/en/Mageia_2_Alpha_2_testing_wanted
https://wiki.mageia.org/en/Working_on_pre-release_ISOs
https://wiki.mageia.org/en/QA_process_for_testing_upgrades
https://wiki.mageia.org/en/QA_process_for_testing_installations
https://wiki.mageia.org/en/QA_process_for_validating_updates
https://wiki.mageia.org/en/QA_testing_procedures
https://wiki.mageia.org/en/Category:QA_procedures
https://wiki.mageia.org/en/Bugzilla
https://wiki.mageia.org/en/How_to_report_a_bug_properly
A similar site / pages may be generated for the FreeBSD .
There was a site for OpenSolaris as
http://test.opensolaris.org/
but now it is shut down .
The following pages are still available :
http://hub.opensolaris.org/bin/view/Community+Group+testing/downloads
http://dlc.sun.com/osol/test/downloads/current/
For the FreeBSD , there is chicken-egg problem :
Due to difficulties of use of FreeBSD , its default settings for desk top
users , difficulties about PC-BSD as an alternative desk top , in my
opinion , is
preventing wide adoption of FreeBSD .
Over the years , I am waiting that one day , I will be able to use FreeBSD
very fluently without living a night mare of setting parameters for each
installation one by one .
For example , in FreeBSD 9.0 amd64 Release :
GNOME is working with a very slow start in root login ,
completely unusable in another user login .
KDE4 , physically unusable due to waiting to start .
Fluxbox is working very well , but it is not for starters .
These problems can not be solved by the probable users .
In GNOME or KDE pages testing is requested .
I do NOT know how to test them : There is no explicit guides / scripts /
algorithms
to be applied , and reports to be generated .
I appreciate works performed by the FreeBSD developers and maintainers with
my
heartiest thanks . Now , my wish is that , in some way , the user interface
of FreeBSD
should be improved in a way that , the people will be able to install it
and
start immediately to work with it .
If I can do it myself , I could do it immediately . I am not an expert
about
operating systems . If I were , I could do the following as a starting
point :
(1) Monitors and video cards are cheap .
Attach to a computer extranous three monitors .
In Loader.conf , define these monitors for
(i) stdin
(ii) stdout
(iii) stderr
to eliminate necessity of serial console for desk top systems .
I studied SVN sources through
http://fxr.watson.org/
but , I could not find places to modify to divert the outputs to
distinct monitors .
(2) When X is starting , it is blanking the screen , and is continuing to
load the required
window manager and its parts .
During that time , the parts are generating a large number of messages
at the back ground which they are not visible .
There is a necessaity to open a messages window and display messages in
that window
to track what is going on .
(3) When X is starting , the other parts should divert their messages to
log files instead
of displaying them on the invisible back ground of X .
(4) For X , use two monitors : One is primary , and other is for messages .
If the above modifications are done , it will be possible to understand the
problems and
report them correctly .
In summary :
(1) Prepare a FreeBSD Testing Handbook .
(2) By making the above modifications , eliminate necessity of serial
console
as much as possible for desk top users .
(3) Supply scripts / algorithms , and suitable programs for testing and
automatic reporting .
(4) Apply outcomes of testing to FreeBSD .
Improve its easiness of usability .
Thnk you very much .
Mehmet Erol Sanliturk
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOgwaMtzu7KfNSTqeUuRheX7PzfK7W3ix2seqbiqszzUpt=RYQ>