From owner-freebsd-hackers@FreeBSD.ORG Mon May 20 17:49:21 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id CBD31B62 for ; Mon, 20 May 2013 17:49:21 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qe0-f53.google.com (mail-qe0-f53.google.com [209.85.128.53]) by mx1.freebsd.org (Postfix) with ESMTP id 927D391F for ; Mon, 20 May 2013 17:49:21 +0000 (UTC) Received: by mail-qe0-f53.google.com with SMTP id s14so224795qeb.12 for ; Mon, 20 May 2013 10:49:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=FqpC7TfBi+zvMEzxbgH6n2CrdO2ForDV8X9y1ORXUsE=; b=eISAFlUhUfgOmSvBEX3sjh04+xnHbW/X4vAxCBGckmVeT37TgHT7Hg5ZWnmY/PZSKA U+LDq03/6RyJqS1TQvDYv13qQxrHLonD00BKCzd+xgGio1RedVPfGrKPSPT2P6E6Ef1R 9PjShlK/kTMKwZsT0GxTuPsb87yEcclywObmWPDDz1l0/up8Er9fmRwGDGEyeuDkYh45 5ji7DOhQ6Df31aW2XmkyCdGrhi5GlQDkJMmngIemWiO8fpcIfrhi6aWkaW1/LzhsGhT4 K7xZqgiN46N8WbfUWCh1TvDOpcE/gHtnDAvgih3EFil04tf5YdCqQ7Mx8ZIdzhQruJG1 Fq1Q== MIME-Version: 1.0 X-Received: by 10.224.4.74 with SMTP id 10mr226137qaq.38.1369072155484; Mon, 20 May 2013 10:49:15 -0700 (PDT) Received: by 10.49.133.137 with HTTP; Mon, 20 May 2013 10:49:13 -0700 (PDT) In-Reply-To: <51976D13.4010604@peterschmitt.fr> References: <51976D13.4010604@peterschmitt.fr> Date: Mon, 20 May 2013 11:49:13 -0600 Message-ID: Subject: Re: blogbench and write-open serialization From: asomers@gmail.com To: florent+FreeBSD-hackers@peterschmitt.fr Content-Type: multipart/mixed; boundary=001a11c2523e7fe4c604dd29f365 X-Mailman-Approved-At: Mon, 20 May 2013 18:19:06 +0000 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 17:49:21 -0000 --001a11c2523e7fe4c604dd29f365 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Sat, May 18, 2013 at 5:59 AM, Florent Peterschmitt wrote: > Le 18/05/2013 06:04, Ivan Voras a =E9crit : > >> During the BSDCan & DevSummit I got interested in finding out why >> blogbench is so slow on FreeBSD. After talking to jhb, it looked like >> one of the reasons might be that opening files with O_RDWR or O_WRONLY >> (anything opening the file for writing) is serialized. >> >> To check this, I've written a small test program, which I've run on >> CentOS 6.3 and FreeBSD 10-HEAD on the same hardware. Here are the result= s: >> >> https://wiki.freebsd.org/Benchmarking/OpenCloseBenchmark >> >> Conclusions: >> >> * Linux opens and closes files much faster than FreeBSD >> * Linux does not serialize write-open operations, while FreeBSD does >> * Even with O_RDONLY, FreeBSD is much slower in opening (and closing) >> files. >> >> I'd welcome a review of these results and comments. > > > Hi, > > I'm no able to say anything about that (because I've no idea of how does > Linux or FreeBSD works ), but could it be a problem from filesystem ? > > Everytime I had UFS I found the entire system very slow when doing some I= /O > (many little freezes), and with ZFS it's globally much better. > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " I largely reproduced your results, but saw even worse scaling. * Linux was much faster, for all thread counts and on all file systems * Linux's performance degraded by about 25% at high thread counts. FreeBSD's penalty was file-system dependent, but usually higher * The filesystem and hard disk mattered little on linux (tested tmpfs and ext4). The hard disk mattered little on FreeBSD, but the file system was very significant. * On FreeBSD, tmpfs was the fasted for a single thread. For nine threads, tmpfs was fastest for writes (about 33% faster than UFS or ZFS), but substantially slower than ZFS for reads. * For O_RDONLY, ZFS scales much better than UFS as the number of threads increases, but at low thread counts UFS was faster. I was testing on a dual-socket, 6 core/socket Xeon E5645 @ 2.4GHz machine. For Linux, I used Debian 6.0.6 with the 2.6.32-5 kernel. For FreeBSD, I was using a custom kernel based on stable/9. --001a11c2523e7fe4c604dd29f365--