From owner-freebsd-performance@FreeBSD.ORG Mon Mar 17 10:09:35 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2263B106564A for ; Mon, 17 Mar 2008 10:09:35 +0000 (UTC) (envelope-from valerio.daelli@gmail.com) Received: from el-out-1112.google.com (el-out-1112.google.com [209.85.162.177]) by mx1.freebsd.org (Postfix) with ESMTP id 8E9BA8FC18 for ; Mon, 17 Mar 2008 10:09:34 +0000 (UTC) (envelope-from valerio.daelli@gmail.com) Received: by el-out-1112.google.com with SMTP id v27so2491516ele.12 for ; Mon, 17 Mar 2008 03:09:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=kreJ2QqXT0TG/xBw8tM1kz569IarKj0vFAXkq++Np6I=; b=FiGwq9SvMary+jz+ZAeOsQMDjuHh+QA2qQJ0+OPCZZlrl1R3ton8/6F5TYJEFbGROyKr9zqKZP06Xp1BhLLJZUvs8Wmftbtqgpm0zVP5H5r4ySL7c/h80tHlNLpnxWha/NgT21mIwdL078KuoArBquGTe6GvL0rtBxQogGTvLy4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=IZymg2ySayRX/7dwk7UZ/ILmEqO4VsIb/0LUYHeANRwYVb/mkN3vRuVGQQ4GIRUQ6uVIWXX3duZEJ+KIf9Aa/vDR6nHyYMxzaUDSXbXmZZRwO0wSyHmS12VHSLJti5v0xW4i8+lXhJJSICWnc0AK/90qYeDsWepOonLsFWhZ7QM= Received: by 10.114.36.1 with SMTP id j1mr17311073waj.119.1205748572530; Mon, 17 Mar 2008 03:09:32 -0700 (PDT) Received: by 10.114.113.14 with HTTP; Mon, 17 Mar 2008 03:09:32 -0700 (PDT) Message-ID: <27dbfc8c0803170309p372e5904vef49b20eff2f4899@mail.gmail.com> Date: Mon, 17 Mar 2008 11:09:32 +0100 From: "Valerio Daelli" To: "Eric Anderson" , "freebsd-performance@freebsd.org" In-Reply-To: <47BEBBCF.7040907@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <27dbfc8c0802190243y113d3059yd0c602850a4dbd6b@mail.gmail.com> <47BB33AD.1050005@FreeBSD.org> <27dbfc8c0802200323r13f69905l4940d0d5accd1eb1@mail.gmail.com> <47BC25C5.1000300@freebsd.org> <27dbfc8c0802200705k482152d4h1bf6e63de24edf59@mail.gmail.com> <47BC5325.8070504@freebsd.org> <27dbfc8c0802210031q3590cafbnbe31698ebdc2d1f2@mail.gmail.com> <47BEBBCF.7040907@freebsd.org> Cc: Subject: Re: Bad performance of 7.0 nfs client with Solaris nfs server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2008 10:09:35 -0000 > Just now got a chance to look at the trace. It looks like FILE_SYNC is > enabled on the write, which will cause the filer to fully commit the > block (8k in this case) to disk before replying. This will usually hurt > performance. I'm not certain where it is getting set, but you might try > some mount options, like 'async' mode. This might also be a bug in > FreeBSD that is forcing it to be enabled all the time. I'll look > through some source code and see what I can find. > > Eric > > Hi I have yes solved this issue and I have another test. Now the mount is sync (no async) and the iozone includes the -D flag. Now the write performance boosts from 3MB/s to 30MB/s. --- root@bsd7:~ iozone -D -+q 1 -i 0 -i 1 -r 2048 -n 2048 -g 2G -Raceb iozone.xls -f /mnt/nest.ifom-ieo-campus.it/iozone/file.tmp Iozone: Performance Test of File I/O Version $Revision: 3.283 $ Compiled for 32 bit mode. Build: freebsd Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Erik Habbinga, Kris Strecker, Walter Wong. Run began: Mon Mar 17 11:06:28 2008 Using msync(MS_ASYNC) on mmap files Delay 1 seconds between tests enabled. Record Size 2048 KB Using minimum file size of 2048 kilobytes. Using maximum file size of 2097152 kilobytes. Excel chart generation enabled Auto Mode Include close in write timing Include fsync in write timing Command line used: iozone -D -+q 1 -i 0 -i 1 -r 2048 -n 2048 -g 2G -Raceb iozone.xls -f /mnt/nest.ifom-ieo-campus.it/iozone/file.tmp Output is in Kbytes/sec Time Resolution = 0.000004 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride KB reclen write rewrite read reread read write read rewrite read 2048 2048 49419 49755 629565 632905 4096 2048 7713 47431 625536 616224 8192 2048 28479 49564 630012 620276 16384 2048 26492 49515 631681 621500 32768 2048 13030 49572 631771 617552 65536 2048 24907 37586^C --- Notice that now we have using msync(MS_ASYNC) on mmap files (not a kernel expert so not sure if it is related to our problem). Without the -D flag we get 3MB/s with iozone. Thanks for you help! Valerio Daelli