From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 23 05:19:17 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A49FF87B; Tue, 23 Oct 2012 05:19:17 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id 003A98FC0C; Tue, 23 Oct 2012 05:19:16 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id hq12so2669691wib.13 for ; Mon, 22 Oct 2012 22:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=ASsoJ+QDqMb6kymBGRj72FbckqXp+vIxl8DhWtR2tPQ=; b=dZPRTc4qL+sH/Po+ke4AOz7hCAqnSNtGrJhScJLS+KmJIqsnugkYmrQ7eRFYqItLTu rRrOhLNnB8yARBMMuc8GSJArCHAfF+ZhxdpDZPzr9XpND2P/8DEtP81Yt4UlYrQomL3A HT82ydGRLBe9hRqUmeQ1E+RVAUkR7kKkIsjb5e8IQsrxfiZYw619gd9J0TU4cggOQlz3 uMTGaA+cNi2d+78tfN9siFjHnWEjv8BIORDhqTOSQiWMMZMWPa3APu6nAyv2nXtoYf56 V7n2L0FtyMhKPHEDdrfcI/FDyMnc1xGUhIpJLfr7qGpSzOUV3iOjGO53fYpa9b+GO8zW cAwg== Received: by 10.180.108.45 with SMTP id hh13mr41867104wib.15.1350969555536; Mon, 22 Oct 2012 22:19:15 -0700 (PDT) Received: from [10.0.0.86] ([93.152.184.10]) by mx.google.com with ESMTPS id di7sm3197331wib.11.2012.10.22.22.19.12 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 22 Oct 2012 22:19:13 -0700 (PDT) Subject: Re: NFS server bottlenecks Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: <656944923.2668891.1350949001204.JavaMail.root@erie.cs.uoguelph.ca> Date: Tue, 23 Oct 2012 08:19:11 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: <656944923.2668891.1350949001204.JavaMail.root@erie.cs.uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1499) Cc: "freebsd-hackers@freebsd.org Hackers" , Ivan Voras X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 05:19:17 -0000 On Oct 23, 2012, at 2:36 AM, Rick Macklem wrote: > Ivan Voras wrote: >> On 20 October 2012 13:42, Nikolay Denev wrote: >>=20 >>> Here are the results from testing both patches : >>> http://home.totalterror.net/freebsd/nfstest/results.html >>> Both tests ran for about 14 hours ( a bit too much, but I wanted to >>> compare different zfs recordsize settings ), >>> and were done first after a fresh reboot. >>> The only noticeable difference seems to be much more context >>> switches with Ivan's patch. >>=20 >> Thank you very much for your extensive testing! >>=20 >> I don't know how to interpret the rise in context switches; as this = is >> kernel code, I'd expect no context switches. I hope someone else can >> explain. >>=20 >> But, you have also shown that my patch doesn't do any better than >> Rick's even on a fairly large configuration, so I don't think there's >> value in adding the extra complexity, and Rick knows NFS much better >> than I do. >>=20 >> But there are a few things other than that I'm interested in: like = why >> does your load average spike almost to 20-ties, and how come that = with >> 24 drives in RAID-10 you only push through 600 MBit/s through the 10 >> GBit/s Ethernet. Have you tested your drive setup locally (AESNI >> shouldn't be a bottleneck, you should be able to encrypt well into >> Gbyte/s range) and the network? >>=20 >> If you have the time, could you repeat the tests but with a recent >> Samba server and a CIFS mount on the client side? This is probably = not >> important, but I'm just curious of how would it perform on your >> machine. >=20 > Oh, I realized that, if you are testing 9/stable (and not head), that > you won't have r227809. Without that, all reads on a given file will > be serialized, because the server will acquire an exclusive lock on > the vnode. >=20 > The patch for r227809 in head is at: > http://people.freebsd.org/~rmacklem/lkshared.patch > This should apply fine to a 9 system (but not 8.n), I think. >=20 > Good luck with it and have fun, rick >=20 >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to >> "freebsd-hackers-unsubscribe@freebsd.org" Thanks, I've applied the patch by hand because of some differences and = I'm now rebuilding. In case they are still needed here are the "dd" tests with loopback UDP = mount : http://home.totalterror.net/freebsd/nfstest/udp-dd.html Over udp writing degrades much worse...=