From owner-freebsd-performance@FreeBSD.ORG Mon Jan 2 22:19:37 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0B40106567C for ; Mon, 2 Jan 2012 22:19:37 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 8D8458FC21 for ; Mon, 2 Jan 2012 22:19:37 +0000 (UTC) Received: (qmail 15218 invoked by uid 0); 2 Jan 2012 22:19:31 -0000 Received: from 67.206.187.152 by rms-us010.v300.gmx.net with HTTP Content-Type: text/plain; charset="utf-8" Date: Mon, 02 Jan 2012 17:19:27 -0500 From: "Dieter BSD" Message-ID: <20120102221929.218250@gmx.com> MIME-Version: 1.0 To: freebsd-performance@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: JkhwbyU03zOlNR3dAHAhHg9+IGRvb0CT Subject: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 22:19:37 -0000 Task: cp(1) a several-GB file from one drive to another, then run cmp(1) to verify.  Cp runs as expected, but cmp runs slower than expected.  Neither the disks nor the cpu is maxed out.  Local drives, no network involved.  Machine is otherwise idle. FreeBSD 8.2 amd64 4 GiB main memory FFS soft updates SATA 2TB drives Doesn't matter which drives/controllers, the example below uses 3132 (siis) with 3726 PM, and JMB363 (ahci), both on PCIe x1 cards Since this ML seems obscessed with the scheduler: kern.sched.preemption: 1 kern.sched.idlespinthresh: 4 kern.sched.idlespins: 10000 kern.sched.static_boost: 160 kern.sched.preempt_thresh: 64 kern.sched.interact: 30 kern.sched.slice: 13 kern.sched.name: ULE cp: extended device statistics                                             device    r/s    w/s     kr/s     kw/s wait  svc_t  %b  controller      ada3  610.9    0.0  77968.6      0.0    1    1.0  64  siisch0_pm3 NCQ ada12    0.5  607.4      8.0  77736.7    6    4.4  60  ahcich1     NCQ    6 users    Load  0.54  0.46  0.33                  Jan  2 13:16 Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP PAGER        Tot   Share      Tot    Share    Free           in   out     in   out Act  276728   42264  3615536   100420  117572  count All  335744   43452 1077460k   135596          pages Proc:                                                            Interrupts  r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt        cow    3921 total  1         153      8076  191 1277 1920 1341             zfod        atkbd0 1                                                          ozfod       uart0 irq4 32.7%Sys   3.4%Intr  0.7%User  0.0%Nice 63.1%Idle        %ozfod    66 psm0 irq12 |    |    |    |    |    |    |    |    |    |    |       daefr       cx23880mpe ================++                                        prcfr   619 siis0 puc0                                       120 dtbuf          totfr  1235 cx23880mpe Namei     Name-cache   Dir-cache    135408 desvn          react     1 fwohci1 bg   Calls    hits   %    hits   %     51442 numvn        2 pdwak       ohci0+ 21      25      25 100                 33843 frevn    36283 pdpgs       ehci0+ 22                                                          intrn       pcm0 nfe0 Disks  ada3 ada12                                  322804 wire   2000 cpu0: time KB/t    128   127                                 1441764 act tps     619   620                                 1670132 inact MB/s  77.07 77.01                                  107684 cache %busy    65    63                                    9888 free                                                   380800 buf cmp: extended device statistics                                             device    r/s    w/s     kr/s     kw/s wait  svc_t  %b  controller      ada3  705.2    0.0  45042.3      0.0    0    0.6  40  siisch0_pm3 NCQ ada12  706.2    0.0  45042.3      0.0    0    0.5  33  ahcich1     NCQ    6 users    Load  0.00  0.07  0.17                  Jan  2 13:32 Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP PAGER        Tot   Share      Tot    Share    Free           in   out     in   out Act 1970756   45312 37243128   100420  145224  count  1399             All 2029948   46696 1111086k   135596          pages 22348 Proc:                                                            Interrupts  r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt        cow    4187 total  2   1     153       10k 7161 1312 2187 1524 7024        zfod        atkbd0 1                                                          ozfod       uart0 irq4 10.6%Sys   2.5%Intr 19.3%User  0.0%Nice 67.5%Idle        %ozfod    88 psm0 irq12 |    |    |    |    |    |    |    |    |    |    |       daefr       cx23880mpe =====++>>>>>>>>>                                          prcfr   700 siis0 puc0                                        11 dtbuf     1321 totfr  1399 cx23880mpe Namei     Name-cache   Dir-cache    135408 desvn          react       fwohci1 bg   Calls    hits   %    hits   %     51442 numvn          pdwak       ohci0+ 21                                     33639 frevn          pdpgs       ehci0+ 22                                                          intrn       pcm0 nfe0 Disks  ada3 ada12                                  312052 wire   2000 cpu0: time KB/t  63.88 63.88                                 1764068 act tps     700   700                                 1330864 inact MB/s  43.65 43.65                                  134912 cache %busy    38    33                                   10312 free                                                   380800 buf The disks can read much faster: cat file > /dev/null: extended device statistics                                             device    r/s    w/s     kr/s     kw/s wait  svc_t  %b  controller      ada3  781.2    0.0  99663.3      0.0    0    1.1  83  siisch0_pm3 NCQ ada12  712.3    0.0  90642.3      0.0    1    1.1  78  ahcich1     NCQ    6 users    Load  0.16  0.14  0.20                  Jan  2 13:49 Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP PAGER        Tot   Share      Tot    Share    Free           in   out     in   out Act  276604   41864  3630548   100420  191588  count All  335608   43064 1077474k   135596          pages Proc:                                                            Interrupts  r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt        cow    4270 total  3         153       12k 1599  95k 2271 1582             zfod        atkbd0 1                                                          ozfod       uart0 irq4 37.2%Sys   3.7%Intr  0.9%User  0.0%Nice 58.2%Idle        %ozfod    82 psm0 irq12 |    |    |    |    |    |    |    |    |    |    |       daefr       cx23880mpe ===================+>                                     prcfr   757 siis0 puc0                                         4 dtbuf          totfr  1431 cx23880mpe Namei     Name-cache   Dir-cache    135408 desvn          react       fwohci1 bg   Calls    hits   %    hits   %     51441 numvn        2 pdwak       ohci0+ 21      25      25 100                 33826 frevn    49879 pdpgs       ehci0+ 22                                                          intrn       pcm0 nfe0 Disks  ada3 ada12                                  305216 wire   2000 cpu0: time KB/t    128   128                                 1442164 act tps     757   716                                 1613304 inact MB/s  94.31 89.20                                  181700 cache %busy    80    79                                    9888 free                                                   380800 buf So reading from the disks isn't the bottleneck, and systat reports that the cpu is 67% idle so the cpu isn't the bottleneck, I'm wondering what *is* cmp's bottleneck?  What else is there that wouldn't show up as one or the other? From owner-freebsd-performance@FreeBSD.ORG Mon Jan 2 22:31:39 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B43081065673 for ; Mon, 2 Jan 2012 22:31:39 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7BA9A8FC18 for ; Mon, 2 Jan 2012 22:31:39 +0000 (UTC) Received: by obbwd18 with SMTP id wd18so18022580obb.13 for ; Mon, 02 Jan 2012 14:31:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=tkWiTKWlGxirMWVr1uIhumVbJwjwI88qNNKOJvdrWaQ=; b=YkFsY3lDOf9kTEHXiWtK+KLtBksVn612qknEik6TgSAqi6z3ercvkAJvxfccBQb0h6 GYcbkiaN1K6qIU+pojPnyNpAA71X14RgTFYf2Lcjujzzvc4cjy7XVLiSilWb96iIlw1u KSeRubQ4uvtLaLaHxnEv9vb9rQVSFNzfoMgMY= MIME-Version: 1.0 Received: by 10.182.159.70 with SMTP id xa6mr43063092obb.1.1325543498836; Mon, 02 Jan 2012 14:31:38 -0800 (PST) Received: by 10.182.152.6 with HTTP; Mon, 2 Jan 2012 14:31:38 -0800 (PST) In-Reply-To: <20120102221929.218250@gmx.com> References: <20120102221929.218250@gmx.com> Date: Mon, 2 Jan 2012 14:31:38 -0800 Message-ID: From: Garrett Cooper To: Dieter BSD Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 22:31:39 -0000 On Mon, Jan 2, 2012 at 2:19 PM, Dieter BSD wrote: > Task: cp(1) a several-GB file from one drive to another, > then run cmp(1) to verify. =A0Cp runs as expected, but > cmp runs slower than expected. =A0Neither the disks > nor the cpu is maxed out. =A0Local drives, no network > involved. =A0Machine is otherwise idle. 1. How are you running cmp? 2. Why do you claim cmp is the bottleneck? Is it spinning the CPU? Thanks, -Garrett From owner-freebsd-performance@FreeBSD.ORG Tue Jan 3 01:29:30 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4987106566C for ; Tue, 3 Jan 2012 01:29:30 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.mail.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 582938FC17 for ; Tue, 3 Jan 2012 01:29:30 +0000 (UTC) Received: (qmail 29808 invoked by uid 0); 3 Jan 2012 01:29:29 -0000 Received: from 67.206.162.250 by rms-us001.v300.gmx.net with HTTP Content-Type: text/plain; charset="utf-8" Date: Mon, 02 Jan 2012 20:29:27 -0500 From: "Dieter BSD" Message-ID: <20120103012929.218270@gmx.com> MIME-Version: 1.0 To: freebsd-performance@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: rjxwbyc03zOlNR3dAHAh/Np+IGRvb0CY Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2012 01:29:30 -0000 >> Task: cp(1) a several-GB file from one drive to another, >> then run cmp(1) to verify.  Cp runs as expected, but >> cmp runs slower than expected.  Neither the disks >> nor the cpu is maxed out.  Local drives, no network >> involved.  Machine is otherwise idle. > >    1. How are you running cmp? >    2. Why do you claim cmp is the bottleneck? Is it spinning the CPU? cmp big_file /other_disk/big_file Cmp is running slower than it should.  It isn't cpu bound ( 67.5%Idle ) but it isn't disk bound either.  Seems like it should be one or the other. From owner-freebsd-performance@FreeBSD.ORG Tue Jan 3 01:38:03 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 089AC106564A for ; Tue, 3 Jan 2012 01:38:03 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id C4B8A8FC14 for ; Tue, 3 Jan 2012 01:38:02 +0000 (UTC) Received: by obbwd18 with SMTP id wd18so18105534obb.13 for ; Mon, 02 Jan 2012 17:38:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=f3u2DiTv9Grphx3i0Ojfcq9hD4TBPP+LoVY/7nDpJQk=; b=kMt36B5QdQcYrbud44CWT7TSTL+q13rVY8a81Xrsc8/Ri6n+BkuqzyloME+gL6KRTj oWIcvgfD1mXM7H8/AXj+j8jRnKIYnfThgNXGhWZokA9L8qvl7dh5zI6l6QTxYvw6Jg2q 2/od0qW8Rs4KiMStjLVLFnpSMhH9iFMHAq1jg= MIME-Version: 1.0 Received: by 10.182.1.8 with SMTP id 8mr43058267obi.11.1325554682257; Mon, 02 Jan 2012 17:38:02 -0800 (PST) Received: by 10.182.152.6 with HTTP; Mon, 2 Jan 2012 17:38:02 -0800 (PST) In-Reply-To: <20120103012929.218270@gmx.com> References: <20120103012929.218270@gmx.com> Date: Mon, 2 Jan 2012 17:38:02 -0800 Message-ID: From: Garrett Cooper To: Dieter BSD Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2012 01:38:03 -0000 On Mon, Jan 2, 2012 at 5:29 PM, Dieter BSD wrote: >>> Task: cp(1) a several-GB file from one drive to another, >>> then run cmp(1) to verify. =A0Cp runs as expected, but >>> cmp runs slower than expected. =A0Neither the disks >>> nor the cpu is maxed out. =A0Local drives, no network >>> involved. =A0Machine is otherwise idle. >> >> =A0 =A01. How are you running cmp? >> =A0 =A02. Why do you claim cmp is the bottleneck? Is it spinning the CPU= ? > > cmp big_file /other_disk/big_file > > Cmp is running slower than it should. =A0It isn't cpu bound ( 67.5%Idle ) > but it isn't disk bound either. =A0Seems like it should be one or the > other. What gets output on the console when you do CTRL-T? Thanks, -Garrett From owner-freebsd-performance@FreeBSD.ORG Tue Jan 3 07:37:40 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F10C1065670 for ; Tue, 3 Jan 2012 07:37:40 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id EC1B78FC08 for ; Tue, 3 Jan 2012 07:37:39 +0000 (UTC) Received: (qmail 25271 invoked by uid 0); 3 Jan 2012 07:37:38 -0000 Received: from 67.206.183.46 by rms-us008.v300.gmx.net with HTTP Content-Type: text/plain; charset="utf-8" Date: Tue, 03 Jan 2012 02:37:34 -0500 From: "Dieter BSD" Message-ID: <20120103073736.218240@gmx.com> MIME-Version: 1.0 To: freebsd-performance@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: F9VwbyQ03zOlNR3dAHAhDZ5+IGRvb8Cg Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2012 07:37:40 -0000 >>>> Task: cp(1) a several-GB file from one drive to another, >>>> then run cmp(1) to verify. Cp runs as expected, but >>>> cmp runs slower than expected. Neither the disks >>>> nor the cpu is maxed out. Local drives, no network >>>> involved. Machine is otherwise idle. >>> >>> 1. How are you running cmp? >>> 2. Why do you claim cmp is the bottleneck? Is it spinning the CPU? >> >> cmp big_file /other_disk/big_file >> >> Cmp is running slower than it should. It isn't cpu bound ( 67.5%Idle ) >> but it isn't disk bound either. Seems like it should be one or the >> other. > >    What gets output on the console when you do CTRL-T? load: 0.59  cmd: cmp 93304 [vnread] 56.99r 8.50u 3.80s 23% 720k load: 0.59  cmd: cmp 93304 [vnread] 57.68r 8.60u 3.85s 22% 720k load: 0.54  cmd: cmp 93304 [vnread] 60.69r 9.03u 4.12s 22% 780k load: 0.54  cmd: cmp 93304 [runnable] 63.79r 9.58u 4.26s 22% 720k load: 0.58  cmd: cmp 93304 [runnable] 68.33r 10.28u 4.62s 21% 788k load: 0.53  cmd: cmp 93304 [runnable] 71.92r 10.78u 4.94s 23% 720k load: 0.53  cmd: cmp 93304 [vnread] 72.31r 10.84u 4.96s 21% 780k load: 0.44  cmd: cmp 93304 [vnread] 198.84r 30.64u 14.36s 23% 720k From owner-freebsd-performance@FreeBSD.ORG Tue Jan 3 08:21:12 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 171351065677 for ; Tue, 3 Jan 2012 08:21:12 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id D6E828FC0C for ; Tue, 3 Jan 2012 08:21:11 +0000 (UTC) Received: by obbwd18 with SMTP id wd18so18297520obb.13 for ; Tue, 03 Jan 2012 00:21:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=4FlWYbJc/szY//NcNQfD9tJRgdtR1ONn1G3ZUJDLoJk=; b=efeOZZp7DiSzsuxTzcSjuPQ0wGtN3t6cCDy8UyP9/qwDXNtskI6/rwdRQ8WkY2dBKY p9mlVxgxMVcmVb5y/BHd6YNKZJDtjFqZrJ6C213EG44M6czCsU1UWXWzSk4/DQTwYZWW s5pexBDqTshANp41CQ6V8EPovJ7bs7poeRtco= MIME-Version: 1.0 Received: by 10.182.1.67 with SMTP id 3mr44120134obk.31.1325578870547; Tue, 03 Jan 2012 00:21:10 -0800 (PST) Received: by 10.182.152.6 with HTTP; Tue, 3 Jan 2012 00:21:10 -0800 (PST) In-Reply-To: <20120103073736.218240@gmx.com> References: <20120103073736.218240@gmx.com> Date: Tue, 3 Jan 2012 00:21:10 -0800 Message-ID: From: Garrett Cooper To: Dieter BSD Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2012 08:21:12 -0000 On Mon, Jan 2, 2012 at 11:37 PM, Dieter BSD wrote: >>>>> Task: cp(1) a several-GB file from one drive to another, >>>>> then run cmp(1) to verify. Cp runs as expected, but >>>>> cmp runs slower than expected. Neither the disks >>>>> nor the cpu is maxed out. Local drives, no network >>>>> involved. Machine is otherwise idle. >>>> >>>> 1. How are you running cmp? >>>> 2. Why do you claim cmp is the bottleneck? Is it spinning the CPU? >>> >>> cmp big_file /other_disk/big_file >>> >>> Cmp is running slower than it should. It isn't cpu bound ( 67.5%Idle ) >>> but it isn't disk bound either. Seems like it should be one or the >>> other. >> >> =A0 =A0What gets output on the console when you do CTRL-T? > > load: 0.59 =A0cmd: cmp 93304 [vnread] 56.99r 8.50u 3.80s 23% 720k > load: 0.59 =A0cmd: cmp 93304 [vnread] 57.68r 8.60u 3.85s 22% 720k > load: 0.54 =A0cmd: cmp 93304 [vnread] 60.69r 9.03u 4.12s 22% 780k > load: 0.54 =A0cmd: cmp 93304 [runnable] 63.79r 9.58u 4.26s 22% 720k > load: 0.58 =A0cmd: cmp 93304 [runnable] 68.33r 10.28u 4.62s 21% 788k > load: 0.53 =A0cmd: cmp 93304 [runnable] 71.92r 10.78u 4.94s 23% 720k > load: 0.53 =A0cmd: cmp 93304 [vnread] 72.31r 10.84u 4.96s 21% 780k > load: 0.44 =A0cmd: cmp 93304 [vnread] 198.84r 30.64u 14.36s 23% 720k Here's a pastebin to the gprof output for cmp of two almost identical files (I added a byte at the end of the file): http://pastebin.com/Rw355d8G . Here's the time output of the process: $ /usr/bin/time -l /usr/obj/usr/src/usr.bin/cmp/cmp /scratch/foo.iso* 99.48 real 27.35 user 33.32 sys 5820 maximum resident set size 251 average shared memory size 2083 average unshared data size 127 average unshared stack size 1310 page reclaims 1569083 page faults 0 swaps 49325 block input operations 88 block output operations 0 messages sent 0 messages received 0 signals received 396 voluntary context switches 13514 involuntary context switches $ uname -a FreeBSD streetfighter.ixsystems.com 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r227801: Mon Nov 21 14:04:39 PST 2011 root@streetfighter.ixsystems.com:/usr/obj/usr/src/sys/STREETFIGHTER amd64 The file is 3.0GB in size. Look at all those page faults though! Thanks! -Garrett From owner-freebsd-performance@FreeBSD.ORG Tue Jan 3 08:46:59 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 661CA1065670 for ; Tue, 3 Jan 2012 08:46:59 +0000 (UTC) (envelope-from marcolz@zlo.nu) Received: from mzh.zlo.nu (ns0.zlo.nu [85.17.141.90]) by mx1.freebsd.org (Postfix) with ESMTP id 1F8AC8FC1B for ; Tue, 3 Jan 2012 08:46:58 +0000 (UTC) Received: by mzh.zlo.nu (Postfix, from userid 1000) id 89BBF140C5; Tue, 3 Jan 2012 09:34:54 +0100 (CET) Date: Tue, 3 Jan 2012 09:34:54 +0100 From: Marc Olzheim To: Garrett Cooper Message-ID: <20120103083454.GA22673@zlo.nu> References: <20120103073736.218240@gmx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/04w6evG8XlLl3ft" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-performance@freebsd.org, Dieter BSD Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2012 08:46:59 -0000 --/04w6evG8XlLl3ft Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 03, 2012 at 12:21:10AM -0800, Garrett Cooper wrote: > The file is 3.0GB in size. Look at all those page faults though! > Thanks! > -Garrett =46rom usr.bin/cmp/c_regular.c: #define MMAP_CHUNK (8*1024*1024) =2E.. for (..) { mmap() chunk of size MMAP_CHUNK. compare munmap()k } That 8 MB chunk size sounds like a bad plan to me. I can imagine something needed to be done to compare files larger than X GB on a 32bit system, but 8MB is pretty small... Marc --/04w6evG8XlLl3ft Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAk8Cva4ACgkQezjnobFOgrGrbwCfdhbKDodxrvUN2TZhpPbbUnhE 6vEAoMVxRX6Fn8rtoqDGwZ9Iaz8Sf/kV =rOJW -----END PGP SIGNATURE----- --/04w6evG8XlLl3ft-- From owner-freebsd-performance@FreeBSD.ORG Tue Jan 3 11:00:43 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 561A11065673 for ; Tue, 3 Jan 2012 11:00:43 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1196F8FC18 for ; Tue, 3 Jan 2012 11:00:42 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so21484877vcb.13 for ; Tue, 03 Jan 2012 03:00:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=H0aUwlf+fs8k/p1KLnOE+41IYgJTPooWoCqW99VROT4=; b=F+VRLXryrPb10YztQRdJzVIdAsu3h4ulD6LJf0y7D0i0YftyIC6DglX4ocA3do9Czz 6Enzfq2uP9ASJS9SUOr7uTVhbqGLrASfBDTkUr3rTvQSyIAMCYS990weu2cc0IDIUcoP wHkXcROb+rP7O8/QSTzIeatHfclsbihLEoODw= MIME-Version: 1.0 Received: by 10.220.213.200 with SMTP id gx8mr29554548vcb.13.1325588442376; Tue, 03 Jan 2012 03:00:42 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.36.5 with HTTP; Tue, 3 Jan 2012 03:00:42 -0800 (PST) In-Reply-To: <20120103083454.GA22673@zlo.nu> References: <20120103073736.218240@gmx.com> <20120103083454.GA22673@zlo.nu> Date: Tue, 3 Jan 2012 03:00:42 -0800 X-Google-Sender-Auth: Ki6uiJGpOPA0MIy7zAGJmaxvEyg Message-ID: From: Adrian Chadd To: Marc Olzheim Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Garrett Cooper , freebsd-performance@freebsd.org, Dieter BSD Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2012 11:00:43 -0000 On 3 January 2012 00:34, Marc Olzheim wrote: > On Tue, Jan 03, 2012 at 12:21:10AM -0800, Garrett Cooper wrote: >> =A0 =A0 The file is 3.0GB in size. Look at all those page faults though! >> Thanks! >> -Garrett > > From usr.bin/cmp/c_regular.c: > > #define MMAP_CHUNK (8*1024*1024) > ... > for (..) { > =A0 =A0 =A0 =A0mmap() chunk of size MMAP_CHUNK. > =A0 =A0 =A0 =A0compare > =A0 =A0 =A0 =A0munmap()k > } > > That 8 MB chunk size sounds like a bad plan to me. I can imagine > something needed to be done to compare files larger than X GB on a 32bit > system, but 8MB is pretty small... Er, hint: look at the average IO size in the cmp versus cp stats above? Something/somehow it's issuing smaller IOs when using mmap? Adrian From owner-freebsd-performance@FreeBSD.ORG Tue Jan 3 14:19:47 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CB8C106566B for ; Tue, 3 Jan 2012 14:19:47 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail02.syd.optusnet.com.au (mail02.syd.optusnet.com.au [211.29.132.183]) by mx1.freebsd.org (Postfix) with ESMTP id BDBE58FC15 for ; Tue, 3 Jan 2012 14:19:46 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail02.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q03EJfrh030971 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 4 Jan 2012 01:19:42 +1100 Date: Wed, 4 Jan 2012 01:19:41 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Marc Olzheim In-Reply-To: <20120103083454.GA22673@zlo.nu> Message-ID: <20120104000111.K6684@besplex.bde.org> References: <20120103073736.218240@gmx.com> <20120103083454.GA22673@zlo.nu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Garrett Cooper , freebsd-performance@freebsd.org, Dieter BSD Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2012 14:19:47 -0000 On Tue, 3 Jan 2012, Marc Olzheim wrote: > On Tue, Jan 03, 2012 at 12:21:10AM -0800, Garrett Cooper wrote: >> The file is 3.0GB in size. Look at all those page faults though! >> Thanks! >> -Garrett > > From usr.bin/cmp/c_regular.c: > > #define MMAP_CHUNK (8*1024*1024) > ... > for (..) { > mmap() chunk of size MMAP_CHUNK. > compare > munmap()k > } > > That 8 MB chunk size sounds like a bad plan to me. I can imagine > something needed to be done to compare files larger than X GB on a 32bit > system, but 8MB is pretty small... 8MB is more than large enough. It works at disk speed in my tests. cp still uses this value. Old versions of cmp used the bogus value of SIZE_T_MAX and aborted on large regular files when mmap() failed. SIZE_T_MAX is bogus because it is larger than can possibly be mmapped on 32-bit machines (except certain unsupported segmented ones), yet it is not large enough for all files on 32-bit machines. On 64-bite machines, it is still more than can be mmapped (except...), but effectively infinity since it is larger than all files. cmp was changed to be more like cp. Both are still remarkably defective. cp is also remarkably ugly, especially in its fallback for when mmap() fails. The fallback for cmp is missing the ugliness, but it uses getc() so it is very slow. This might be the problem here. The fallback is to use c_special(), and c_special() is also used unconditionally for "special" files, and special files are detected badly: - there is no way to force a file to be special (or not special). This would be useful for testing the mmap() method and the non-mmap() method on the same file - if one of the files is named "-", then this is an alias for stdin and the file is considered special. I see no good reason to force specialness here. It can be used to avoid the mmap() method. - otherwise, one of the files is special if it is not regular according to fstat() on it. For some reason, the fstat()s are not done if specialness was forced by one of the file names being "-". In my tests, using "-" for one of the files mainly takes lots more user time. It only reduces the real time by 25%. This is on a core2. On a system with a slow CPU, it is easy for getc() to be much slower than the disk. Bruce From owner-freebsd-performance@FreeBSD.ORG Tue Jan 3 22:16:13 2012 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A1071065675 for ; Tue, 3 Jan 2012 22:16:13 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (m209-73.dsl.rawbw.com [198.144.209.73]) by mx1.freebsd.org (Postfix) with ESMTP id 14F2F8FC1B for ; Tue, 3 Jan 2012 22:16:12 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.5/8.14.5) with ESMTP id q03LdSw7020558 for ; Tue, 3 Jan 2012 13:39:28 -0800 (PST) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.5/8.14.5/Submit) id q03LdS4J020557 for performance@freebsd.org; Tue, 3 Jan 2012 13:39:28 -0800 (PST) (envelope-from david) Date: Tue, 3 Jan 2012 13:39:28 -0800 From: David Wolfskill To: performance@freebsd.org Message-ID: <20120103213928.GP1733@albert.catwhisker.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4SRTEifjNkXp0Rce" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: Subject: vm.exec_map_entries vs. hw.ncpu vs. ...? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2012 22:16:13 -0000 --4SRTEifjNkXp0Rce Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable A colleague at work has been researching the nature of some issues we're seeing in a FreeBSD/amd64 release/8.2.0 environment in which amd (which is providing services to a jail, where the "useful work" actually gets done) is apparently failing (on occasion) to perform the work it is supposed to do because vm.exec_map_entries is set too low. vm.exec_map_entries default value is 16 -- see src/sys/vm/vm_init.c:91-94: static int exec_map_entries =3D 16; TUNABLE_INT("vm.exec_map_entries", &exec_map_entries); SYSCTL_INT(_vm, OID_AUTO, exec_map_entries, CTLFLAG_RD, &exec_map_entries, = 0, "Maximum number of simultaneous execs"); We had not previously modified the setting. The machines in question are dual-package, 6-core machines with Intel's SMT (Simultaneous Multi-Threading) enabled, so hw.ncpu is 24. Is it reasonable for exec_map_entries to be thus initialized, independently of number of logical cores, kern.maxusers, amount of memory, or anything else? Are there guidelines or suggestions for setting it? Apparently each entry consumes a bit over 200KB, so making it arbitrarily large is probably not a good idea. :-} I don't see it mentioned in tuning(7). Thanks for any inisight. Peace, david --=20 David H. Wolfskill david@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --4SRTEifjNkXp0Rce Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk8DdY8ACgkQmprOCmdXAD0uRgCfZKODGaZcA/ZsB73aqYHRTveq XbEAoIMqTRc8vMGP8dOMg8yab2m7BG5w =7beb -----END PGP SIGNATURE----- --4SRTEifjNkXp0Rce-- From owner-freebsd-performance@FreeBSD.ORG Wed Jan 4 21:58:30 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E75EE106566C; Wed, 4 Jan 2012 21:58:30 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id BCC328FC0C; Wed, 4 Jan 2012 21:58:29 +0000 (UTC) Received: by werb13 with SMTP id b13so15664963wer.13 for ; Wed, 04 Jan 2012 13:58:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=6XEMeBte3msHnx3roQDC91QmbOtPNieE6mOxFDdWWoY=; b=u7K0sXTxjsX9ufdSpSwmxwnfC3tIap1qhVMswDIwcOrNjxdij2yGdZlOuHFKYdCXYO m5ETSJcKwAd/jkg20Dcg8dk/gi/5Ro/ve57Lkl43lmnnEph+MZYljPgqTMvFIe87yIj8 lGUEw5j2CzDBgeKgQBmR7V3PBJabBBYw8LsSE= MIME-Version: 1.0 Received: by 10.216.131.90 with SMTP id l68mr37657293wei.36.1325714308629; Wed, 04 Jan 2012 13:58:28 -0800 (PST) Received: by 10.216.178.204 with HTTP; Wed, 4 Jan 2012 13:58:28 -0800 (PST) In-Reply-To: <4eeb7cf9.43bfec0a.3e38.ffff9d19SMTPIN_ADDED@mx.google.com> References: <4eeb7cf9.43bfec0a.3e38.ffff9d19SMTPIN_ADDED@mx.google.com> Date: Wed, 4 Jan 2012 16:58:28 -0500 Message-ID: From: Arnaud Lacombe To: matthew@phoronix.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Wed, 04 Jan 2012 22:35:17 +0000 Cc: Adrian Chadd , FreeBSD Stable Mailing List , Current FreeBSD , Joe Holden , Michael Larabel , Stefan Esser , freebsd-performance@freebsd.org, "O. Hartmann" , Jeremy Chadwick Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2012 21:58:31 -0000 Hi, On Fri, Dec 16, 2011 at 12:16 PM, wrote: > Thanks. > > My request for the person documenting the tunings also runs the benchmark= to > ensure expected behaviour. > Why should you have to tune anything ? Did you tune the Oracle Server install ? If not, you should not have to tune the FreeBSD install, that wouldn't be fair. If you tune FreeBSD, you should tune the Oracle Server install too. It is pretty easy to win at least 30% in performance for certain workload by choosing the right kernel configuration. - Arnaud > The installation, execution and comparison against the benchmarks in the > article is fairly simple. > > Note that some tuning may not be relevant or recommended (ie: some of the= fs > benchmarks are sensitive to barriers and other synchronous operations). = =A0I'd > recommend bowing out of a benchmark with a 'we're going to be slower sinc= e > the default configuration is this way for the following reason' if this i= s > the case. > > Thanks 'someone'. > > Matthew > > > =A0Dec 16, 2011 8:46 AM, Adrian Chadd wrote: > > Can someone please write up a nice, concise blog post somewhere > outlining all of this? > > Extra bonus points if it's a blog that is picked up by > blogs.freebsdish.org and/or some of the other BSD sites. > > Guys/girls/fuzzy things - this is 2011; people look at shiny blog > sites with graphs rather than mailing lists. Sorry, we lost that > battle. :) > > > > Adrian > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Wed Jan 4 22:31:58 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 911B11065678; Wed, 4 Jan 2012 22:31:58 +0000 (UTC) (envelope-from matthew@phoronix.com) Received: from phx1.phoronix.com (173.192.77.202-static.reverse.softlayer.com [173.192.77.202]) by mx1.freebsd.org (Postfix) with ESMTP id 5CF0C8FC14; Wed, 4 Jan 2012 22:31:58 +0000 (UTC) Received: from mobile-166-205-136-164.mycingular.net ([166.205.136.164] helo=www.palm.com) by phx1.phoronix.com with esmtpa (Exim 4.69) (envelope-from ) id 1RiZNC-0007LN-Oy; Wed, 04 Jan 2012 16:31:56 -0600 Date: Wed, 04 Jan 2012 14:31:55 -0800 From: To: "Arnaud Lacombe" In-Reply-To: X-Mailer: Palm webOS X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - phx1.phoronix.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - phoronix.com Message-Id: <20120104223158.911B11065678@hub.freebsd.org> X-Mailman-Approved-At: Wed, 04 Jan 2012 23:13:51 +0000 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Adrian Chadd , FreeBSD Stable Mailing List , Current FreeBSD , Joe Holden , Michael Larabel , Stefan Esser , freebsd-performance@freebsd.org, "O. Hartmann" , Jeremy Chadwick Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2012 22:31:58 -0000 Thanks for the comment Arnaud. For comparative benchmarking on = [1]Phoronix.com, Michael inva= riable leaves it in the default configuration 'in the way the developers or= vendor wanted it for production'. This is by rule. However, i= nvariable the community or vendor for platforms that post poor scores on be= nchmark cry foul about using the default config. 'it should be tuned,= no-one deploys an untuned system' or 'the system is configured for a diffe= rent workload'. The response from us to this comes in two forms. &nb= sp; 1) If it is the wrong workload for the platform, do a public pos= t explaining and analysing the results. Highlighting the rationale fo= r the concious reduction in performance (ie: journaling filesystems with ba= rriers suffer in some write benchmarks for the sake of filesystem integrity= =2E 2) If tuning can have a material impact on the results, post a t= uning guide with step by step and rationale. Ie: educate the communit= y and users. Michael and I have had many discussions with vendors an= d communities on this. In almost all cases, the vendor has either cha= nged the default configuration or accepted the results as valid. As = a service to the community or vendor that publishes the tuning guide, Micha= el is more than willing to redo a tuned vs untuned comparison. To dat= e, the communities have never taken us up on that offer. In part, thi= s affects [2]Phoronix.com's perception in the public, but that is more of a result of a one sided d= iscussion by a party external to a particular community (with a healthy tou= ch of journalisticly pumped compare & contrast). For the FreeBSD = community, who else outside of the FreeBSD community actually runs public c= omparisons of FreeBSD against anything? Matthew -- Sent from my HP Pre3 _________________________________________________________________ On Jan 4, 2012 1:58 PM, Arnaud Lacombe wrote:=0D > Thanks.=0D >=0D &= gt; My request for the person documenting the tunings also runs the benchma= rk to=0D > ensure expected behaviour.=0D >=0D Why should you= have to tune anything ? Did you tune the Oracle Server=0D install ? If = not, you should not have to tune the FreeBSD install,=0D that wouldn't b= e fair. If you tune FreeBSD, you should tune the Oracle=0D Server instal= l too. It is pretty easy to win at least 30% in=0D performance for certa= in workload by choosing the right kernel=0D configuration.=0D =0D = - Arnaud=0D =0D > The installation, execution and comparison agai= nst the benchmarks in the=0D > article is fairly simple.=0D >= =0D > Note that some tuning may not be relevant or recommended (ie: s= ome of the fs=0D > benchmarks are sensitive to barriers and other syn= chronous operations). =C2=A0I'd=0D > recommend bowing out of a benchm= ark with a 'we're going to be slower since=0D > the default configura= tion is this way for the following reason' if this is=0D > the case.= =0D >=0D > Thanks 'someone'.=0D >=0D > Matthew=0D>=0D >=0D > =C2=A0Dec 16, 2011 8:46 AM, Adrian Chadd wrote:=0D >=0D > Can someone please write= up a nice, concise blog post somewhere=0D > outlining all of this?= =0D >=0D > Extra bonus points if it's a blog that is picked up = by=0D > blogs.freebsdish.org and/or some of the other BSD sites.=0D>=0D > Guys/girls/fuzzy things - this is 2011; people look at sh= iny blog=0D > sites with graphs rather than mailing lists. Sorry, we = lost that=0D > battle. :)=0D >=0D >=0D >=0D >= Adrian=0D > _______________________________________________=0D &g= t; freebsd-performance@freebsd.org mailing list=0D > http://lists.fre= ebsd.org/mailman/listinfo/freebsd-performance=0D > To unsubscribe, se= nd any mail to=0D > "freebsd-performance-unsubscribe@freebsd.org"=0D<= br> References 1. 3D"http://Phoronix.com"/ 2. 3D"http://Phoronix.com"/ From owner-freebsd-performance@FreeBSD.ORG Thu Jan 5 00:09:01 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86A87106566C; Thu, 5 Jan 2012 00:09:01 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1EB448FC1C; Thu, 5 Jan 2012 00:09:01 +0000 (UTC) Received: by obbwd18 with SMTP id wd18so21226709obb.13 for ; Wed, 04 Jan 2012 16:09:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=XbdItQKYiYomCveqZYxYJ+LUwLQyfgERPDjPndldPFk=; b=cELNsjMMEMFMmHw6np5h5sHVkIfqFFkE20+dUbkkCnnX/oJzZ901KT3nE+phbwargC 5my6IDplur3bPgiqW8xs5gy8zxHHYeTXte31U4LcXSGTeQIFTVKWfGBJHAbifNSS1/hW yZ+th9THjZv11CzWZwitcMJuHLWsrGoRYBPDc= MIME-Version: 1.0 Received: by 10.182.159.70 with SMTP id xa6mr49957557obb.1.1325722140669; Wed, 04 Jan 2012 16:09:00 -0800 (PST) Received: by 10.182.152.6 with HTTP; Wed, 4 Jan 2012 16:08:59 -0800 (PST) In-Reply-To: References: <4eeb7cf9.43bfec0a.3e38.ffff9d19SMTPIN_ADDED@mx.google.com> Date: Wed, 4 Jan 2012 16:08:59 -0800 Message-ID: From: Garrett Cooper To: Arnaud Lacombe Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Thu, 05 Jan 2012 00:16:21 +0000 Cc: Adrian Chadd , FreeBSD Stable Mailing List , "O. Hartmann" , Joe Holden , Michael Larabel , Current FreeBSD , freebsd-performance@freebsd.org, matthew@phoronix.com, Jeremy Chadwick Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 00:09:01 -0000 On Wed, Jan 4, 2012 at 1:58 PM, Arnaud Lacombe wrote: > Hi, > > On Fri, Dec 16, 2011 at 12:16 PM, =A0 wrote: >> Thanks. >> >> My request for the person documenting the tunings also runs the benchmar= k to >> ensure expected behaviour. >> > Why should you have to tune anything ? Did you tune the Oracle Server > install ? If not, you should not have to tune the FreeBSD install, > that wouldn't be fair. If you tune FreeBSD, you should tune the Oracle > Server install too. It is pretty easy to win at least 30% in > performance for certain workload by choosing the right kernel > configuration. This assumes that Oracle doesn't do secret sauce tuning... the Vanilla CentOS/RHEL base is probably a better comparison than the Oracle custom distro. Thanks! -Garrett From owner-freebsd-performance@FreeBSD.ORG Thu Jan 5 17:46:28 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AB6D106564A for ; Thu, 5 Jan 2012 17:46:28 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.mail.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 434698FC15 for ; Thu, 5 Jan 2012 17:46:28 +0000 (UTC) Received: (qmail 32135 invoked by uid 0); 5 Jan 2012 17:46:27 -0000 Received: from 67.206.161.160 by rms-us011.v300.gmx.net with HTTP Content-Type: text/plain; charset="utf-8" Date: Thu, 05 Jan 2012 12:46:24 -0500 From: "Dieter BSD" Message-ID: <20120105174626.218240@gmx.com> MIME-Version: 1.0 To: freebsd-performance@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: pYR3byQ03zOlNR3dAHAhr4x+IGRvbwCX Subject: Re: cmp(1) has a bottleneck, but where? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 17:46:28 -0000 > Something/somehow it's issuing smaller IOs when using mmap? On my box, 64K reads.  Using the '-' to avoid mmap it uses 128K. The big difference I found was that the default mmap case isn't using read-ahead. So it has to wait on the disk every time.  :-( Using the '-' to avoid mmap it benefits from read-ahead, but the default of 8 isn't large enough.  Crank up vfs.read_max and it becomes cpu bound.  (assuming using 2 disks and not limited by both disks being on the same wimpy controller) A) Should the default vfs.read_max be increased? B) Can the mmap case be fixed?  What is the aledged benefit of using mmap anyway?  All I've even seen are problems. From owner-freebsd-performance@FreeBSD.ORG Thu Jan 5 19:30:03 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E75310656B0; Thu, 5 Jan 2012 19:30:03 +0000 (UTC) (envelope-from owner-freebsd-stable@freebsd.org) Received: from web.npulse.net (web.npulse.net [79.172.194.2]) by mx1.freebsd.org (Postfix) with SMTP id D42EC8FC3D; Thu, 5 Jan 2012 19:30:01 +0000 (UTC) Received: by web (Postfix, from userid 143) id AE12EDC213; Wed, 4 Jan 2012 23:19:08 +0000 (UTC) Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by web (Postfix) with ESMTP id 1EBFFDC1F7 for ; Wed, 4 Jan 2012 23:19:06 +0000 (UTC) Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id BA290157997; Wed, 4 Jan 2012 23:14:14 +0000 (UTC) Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 588D21065740; Wed, 4 Jan 2012 23:14:12 +0000 (UTC) (envelope-from owner-freebsd-stable@freebsd.org) Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 911B11065678; Wed, 4 Jan 2012 22:31:58 +0000 (UTC) (envelope-from matthew@phoronix.com) Received: from phx1.phoronix.com (173.192.77.202-static.reverse.softlayer.com [173.192.77.202]) by mx1.freebsd.org (Postfix) with ESMTP id 5CF0C8FC14; Wed, 4 Jan 2012 22:31:58 +0000 (UTC) Received: from mobile-166-205-136-164.mycingular.net ([166.205.136.164] helo=www.palm.com) by phx1.phoronix.com with esmtpa (Exim 4.69) (envelope-from ) id 1RiZNC-0007LN-Oy; Wed, 04 Jan 2012 16:31:56 -0600 Date: Wed, 04 Jan 2012 14:31:55 -0800 From: To: "Arnaud Lacombe" In-Reply-To: X-Mailer: Palm webOS X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - phx1.phoronix.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - phoronix.com Message-Id: <20120104223158.911B11065678@hub.freebsd.org> X-Mailman-Approved-At: Wed, 04 Jan 2012 23:14:09 +0000 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Sender: owner-freebsd-stable@freebsd.org Errors-To: owner-freebsd-stable@freebsd.org Cc: Adrian Chadd , FreeBSD Stable Mailing List , Joe Holden , Michael Larabel , Current FreeBSD , freebsd-performance@freebsd.org, "O. Hartmann" , Jeremy Chadwick Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server X-BeenThere: freebsd-performance@freebsd.org List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jan 2012 19:30:04 -0000 Thanks for the comment Arnaud. For comparative benchmarking on = [1]Phoronix.com, Michael inva= riable leaves it in the default configuration 'in the way the developers or= vendor wanted it for production'. This is by rule. However, i= nvariable the community or vendor for platforms that post poor scores on be= nchmark cry foul about using the default config. 'it should be tuned,= no-one deploys an untuned system' or 'the system is configured for a diffe= rent workload'. The response from us to this comes in two forms. &nb= sp; 1) If it is the wrong workload for the platform, do a public pos= t explaining and analysing the results. Highlighting the rationale fo= r the concious reduction in performance (ie: journaling filesystems with ba= rriers suffer in some write benchmarks for the sake of filesystem integrity= =2E 2) If tuning can have a material impact on the results, post a t= uning guide with step by step and rationale. Ie: educate the communit= y and users. Michael and I have had many discussions with vendors an= d communities on this. In almost all cases, the vendor has either cha= nged the default configuration or accepted the results as valid. As = a service to the community or vendor that publishes the tuning guide, Micha= el is more than willing to redo a tuned vs untuned comparison. To dat= e, the communities have never taken us up on that offer. In part, thi= s affects [2]Phoronix.com's perception in the public, but that is more of a result of a one sided d= iscussion by a party external to a particular community (with a healthy tou= ch of journalisticly pumped compare & contrast). For the FreeBSD = community, who else outside of the FreeBSD community actually runs public c= omparisons of FreeBSD against anything? Matthew -- Sent from my HP Pre3 _________________________________________________________________ On Jan 4, 2012 1:58 PM, Arnaud Lacombe wrote:=0D > Thanks.=0D >=0D &= gt; My request for the person documenting the tunings also runs the benchma= rk to=0D > ensure expected behaviour.=0D >=0D Why should you= have to tune anything ? Did you tune the Oracle Server=0D install ? If = not, you should not have to tune the FreeBSD install,=0D that wouldn't b= e fair. If you tune FreeBSD, you should tune the Oracle=0D Server instal= l too. It is pretty easy to win at least 30% in=0D performance for certa= in workload by choosing the right kernel=0D configuration.=0D =0D = - Arnaud=0D =0D > The installation, execution and comparison agai= nst the benchmarks in the=0D > article is fairly simple.=0D >= =0D > Note that some tuning may not be relevant or recommended (ie: s= ome of the fs=0D > benchmarks are sensitive to barriers and other syn= chronous operations). =C2=A0I'd=0D > recommend bowing out of a benchm= ark with a 'we're going to be slower since=0D > the default configura= tion is this way for the following reason' if this is=0D > the case.= =0D >=0D > Thanks 'someone'.=0D >=0D > Matthew=0D>=0D >=0D > =C2=A0Dec 16, 2011 8:46 AM, Adrian Chadd wrote:=0D >=0D > Can someone please write= up a nice, concise blog post somewhere=0D > outlining all of this?= =0D >=0D > Extra bonus points if it's a blog that is picked up = by=0D > blogs.freebsdish.org and/or some of the other BSD sites.=0D>=0D > Guys/girls/fuzzy things - this is 2011; people look at sh= iny blog=0D > sites with graphs rather than mailing lists. Sorry, we = lost that=0D > battle. :)=0D >=0D >=0D >=0D >= Adrian=0D > _______________________________________________=0D &g= t; freebsd-performance@freebsd.org mailing list=0D > http://lists.fre= ebsd.org/mailman/listinfo/freebsd-performance=0D > To unsubscribe, se= nd any mail to=0D > "freebsd-performance-unsubscribe@freebsd.org"=0D<= br> References 1. 3D"http://Phoronix.com"/ 2. 3D"http://Phoronix.com"/ _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"