Date: Fri, 23 Dec 2011 02:05:38 +0100 From: "O. Hartmann" <ohartman@zedat.fu-berlin.de> To: Johan Hendriks <joh.hendriks@gmail.com> Cc: freebsd-performance@freebsd.org, FreeBSD <freebsd-current@freebsd.org> Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server Message-ID: <4EF3D3E2.9000403@zedat.fu-berlin.de> In-Reply-To: <4EF2F210.6080009@gmail.com> References: <xf5fxrkpagw2qf65tk9y2njf.1324492907631@email.android.com> <4EF25468.9040204@gmail.com> <4EF2D814.1090805@freebsd.org> <4EF2F210.6080009@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigA2AB74508AC85FFC0BBAD257 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 12/22/11 10:02, Johan Hendriks wrote: > Stefan Esser schreef: >> Am 21.12.2011 22:49, schrieb Johan Hendriks: >>> Nice page, but one thing i do not get is the following. >>> >>> [quote] >>> If you compare FreeBSD / GCC 4.2.1 against, for example, Ubuntu / GCC= >>> 4.7 then the results are unlikely to tell you anything meaningful abo= ut >>> FreeBSD vs Ubuntu. >>> [/quote] >>> >>> That is a little strange in my opinion. >>> It tells me that FreeBSD falls more and more behind on Linux. >>> The reason is or could be that FreeBSD cannot or will not include GCC= >>> 4.7 and that FreeBSD will not be on par with Linux anymore. >>> To compare it with Formula1 cars. >>> If Mercedes decide to use the engine from 2 seasons back (the engine >>> version 4.2.1) in there 2012 car, and Ferrari uses there new Engine >>> (version 4.7). >>> Can we not compare them anymore because of the decission from Mercede= s >>> to use the old engine? >>> No we just say, if you want to win a race, get the Ferrari. >>> >>> It is the reallity, FreeBSD uses 4.2.1 as there compiler!!! >> As has been pointed out by others, FreeBSD ships with gcc-4.2.1 (with >> some local modifications and fixes) as the system compiler. >=20 >>> If you tune up FreeBSD to use the GCC 4.7 compiler, or downgrade linu= x >>> to 4.2.1, then that will tell me nothing about FreeBSD vs Linux. >> The gcc version distributed with FreeBSD was chosen for license reason= s, >> not for technical reasons. If you are OK with installing a GPLv3 >> licensed compiler on your systems, then just do it and take advantage = of >> the improved code generated by it. > It does not matter what the decission is to use the old compiler, it is= > a fact that the base comes with 4.2.x > Does that mean we can not compare/benchmark against other distributions= > because they use GPLv3 stuff. > No, i want to know where standard released FreeBSD stands against > standard released Linux distributions. > If you compare benchmark userland applictions, then it is fair to use > the latest compiler for the userland software also on FreeBSD. > But what if the ports tree defaults to LLVM, then again we want to know= > what FreeBSD does against a Linux distribution. > Why because that is what most of us will be using...!! Who ever tried to use gcc 4.6 to compile the base system knows that it is no eays task and it isn't so easy to simply change the compiler! This is also true for a lot of ports. If it is so easy to use a more modern compiler as some of the statements made here would suggest, then I would expect a dedicated chapter in the handbook! In such a case, every systems administrator trying to make a long-term decission what operating system might be the base for the future, does not need to be an enthusiastic of either BSD or Linux to understand how to tweak - fanboys, developers or enthusiasts have already choosen, apart from any rational or reason. What matters are advantages which can be approved. Downlad the DVD, install the OS, do some adjustments regarding to some pages of the manual, choose a propper set of applicable software to benchmark, compile, benchmark the system. Phoronix did so. Well, it's hard for me to find the chapter in the handbook which describes the performance tuning of SCHED_ULE and its sysctl tweaks, someone may call me stupid and point me to the page, please ... >=20 > If we start to compile all the ports with gcc 4.7 to be on par in > comparising and benchmarking, why spend all the time getting LLVM as th= e > default compiler for ports also? > Why not take that effort into making the WHOLE ports tree to compile > with GCC4.7? >=20 > Reason, because FreeBSD goes the LLVM route. That is a decission FreeBS= D > is making! Yes, and it is legitime to question that and bring pro and contra for that decission. But since "FreeBSD" is obviously a small club of people sitting like a duck on eggs (and, by the way, not their own genuine invented eggs, more or less reingeneered eggs), those decissions get more obscure than they seem to be anyway. > And that choise will be the FreeBSD that is used in comparising and > benchmarks on the net , not the utterly overcopiled and tuned FreeBSD > against stock Ubuntu or whatever Linux distribution. >=20 > If it is a good or bad choice! That we will see in the > comparising/benchmarks we will be seeing when that time comes. >=20 > Same goes for the scheduler! and all the other subsystems FreeBSD has > choosen, that makes FreeBSD. >=20 =2E.. sometimes the underdog has to pick up what's left ... >> >>> I my opinion, you benchmark the latest release of Linux, FreeBSD, >>> Solaris, Windows and whatever OS you want to compare! >> As you probably know, Linux is just the kernel and the distributions a= dd >> user space programs, including a compiler. You can easily create a >> "FreeBSD distribution" with more advanced compiler and use or even sel= l >> it. But the FreeBSD project was cautious to not heavily depend on a >> GPLv3 compiler (for reasons openly discussed at the time this decision= >> was made). > I know Linux is a kernel, re read Linux as Linux Distribution! > Yes you can use a more advanced compiler on FreeBSD, BUT you can do tha= t > on Linux also ,so where do you stop? > Are you going to spend a month to compare a fullly tuned up FreeBSD > system against a Linux distribution? > No because the users will not spend months tuning and recompile there > servers. > They use the FreeBSD version that comes with the CD! > And that we want to compare/benchmark against a Linux distribution. When it comes to the question what to benchmark, what would you suggest? Only the Linux kernel, so called LINUX and a couple of self selected, self compiled, probably seld optimized GNU userland tools? At the end I pick up a distribution as it comes from the "vendor", pick up some informations about tweaks for several target workloads and start benchmarking. >=20 >>> You want to benchmark the release and not a tuned version against a >>> standard version. >>> And that in general are the versions most of us users will use. >> If you compare operating systems from a technical point of view, then >> you'll be interested in relative performance of algorithms and methods= >> chosen. This is best achieved by using the same compiler for each of t= he >> candidates. >> >> If you compare performance from a user point of view, you are correct >> that performance delivered out of the box (without complicated tuning)= >> may be, what counts for most users. But those users that depend on bes= t >> performance e.g. for a FreeBSD based embedded product or a data center= , >> may tune the system, including compilation with a newer compiler than >> the system default. >> >>> And what if in the future LLVM gets on par with Linux, is it stil fai= r >>> to compare FreeBSD with Linux? >> You can always compare anything with whatever you like (even apples wi= th >> oranges), but you need to be aware of what you compare and what your >> goals are, to be able to draw reasonable conclusions. >> >> If you want to test out of the box performance, then test with system >> compilers (or just those binaries delivered with the system). >> >> If you want to test for code efficiency or scalability, then use the >> same compilers for each system under test to remove differences >> introduced by the compilers (which are an external component not >> developed by the FreeBSD people). >> >>> Or do we say, well we are on par, but it is not fair, yes we used the= >>> latest releases, but you can not blame Linux because they are still >>> using GCC. >> Depends on what you want or need to measure ... > I am mainly talking about the out of the box comparison, because that i= s > what most sites will do when doing a benchmark. > they want to know if the the all new version of FreeBSD can compete wit= h > the latest release of Ubuntu. > That are the benchmarks you will see on the net. And this is the right way to do. The only way. I start my project now. I got funding now. I have a time contraint to keep to fullfill my scientific work, say, a PhD or a project. I need to look into the future, I need to make decissions for a long haul development. So, I pick up the OSs of my choice and benchmark them as they are. All right, if there is a chapter in the handbook explaining how to tune towards a specific expected workload, much better, so do so =2E.. If this is not the way it is, please correct me! >=20 >=20 >>> No what we will see then are haleluja blogs that FreeBSD is on par wi= th >>> Linux. >> Such blog messages are not common in the FreeBSD community. FreeBSD us= ed >> to have big technical and performance advantages when Linux was young,= >> but even then, there was technical discussion between camps (and many >> concepts were implemented in Linux based on BSD examples; I have taken= >> part in such discussions myself, some 15 to 20 years back). > Well not quite, i remember the mysql on SCHED_ULE reports, and they > where quit haleluja about the whole thing. > Not wrong at all, we are all human and like to compete and even more > like to be on top in what we do. >=20 >=20 >>> For me peformance is not a show stopper, and for the most of us i thi= nk >>> it is not. >>> FreeBSD for me is a clean system that does the job perfect and has a >>> very helpful community. >> Well, this are valid aspects, too, and very hard to with benchmarks ;-= ) >> >> Regards, STefan > I am not saying FreeBSD is bad and is not performing, i just do not > understand why FreeBSD is doing all these things to look good. > Again, most comparisings/benchmarks will be stock *BSD vs a stock Linux= > distribution. > If we want to look good, FreeBSD must make sure there released version > gets on par! > We all know FreeBSD is quite conservative with its default settings. > Maybe we need to set some default settings less conservative. > But i do not know if that is good just for the numbers! >=20 > Regards, > Johan oh --------------enigA2AB74508AC85FFC0BBAD257 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iQEcBAEBAgAGBQJO89PiAAoJEOgBcD7A/5N8hNcIAJatBurrv4Hjl/pyaGSP+wlV v2OjnU5nAL9ahjoC3xqZZ3BAvSGxN2XpFozE5LS4TEQQaPNs7Uy4DBRk3h4Zgc7Z YiVt7wJQouX9Lps7H+5d3sPV7XXqbYJjA304bwWeRoHMjE/gqjmM55eLi3h1Cs0I toHrQf6EV0IOeOqObguBcDdANX4tA1VVZPO0Y48bCC1wSsJvUlt8qw9OOPbUUKGR BFsW6Aqc587vVG+sW1GkKXKShiBfjkVl0sogmxqCSvTesQYkmlRaeN/h2swEzIhK DQm3M4+unIxypCmX9VcgMbtnlywr0JE+7+P8wzl/ftE9M02W3adPftwbDdbsih4= =Q9m2 -----END PGP SIGNATURE----- --------------enigA2AB74508AC85FFC0BBAD257--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4EF3D3E2.9000403>