From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 20 12:46:42 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E8803C31; Sat, 20 Oct 2012 12:46:42 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 820768FC16; Sat, 20 Oct 2012 12:46:42 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEANebglCDaFvO/2dsb2JhbAA8CIYUvACCIAEBAQMBAQEBICsfAQsFFg4KAgINGQIjBgEJJgYIBwQBHASHUQMJBguoaohRDYlUgSCJUmgWBIVDgRIDk0FYgVWBF4oRhRCDC4FHNQ X-IronPort-AV: E=Sophos;i="4.80,621,1344225600"; d="scan'208";a="184523521" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 20 Oct 2012 08:45:32 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 6AFCAB405E; Sat, 20 Oct 2012 08:45:32 -0400 (EDT) Date: Sat, 20 Oct 2012 08:45:32 -0400 (EDT) From: Rick Macklem To: Ivan Voras Message-ID: <191784842.2570110.1350737132305.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: Subject: Re: NFS server bottlenecks MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE7 (Win)/6.0.10_GA_2692) Cc: "freebsd-hackers@freebsd.org Hackers" , Nikolay Denev X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Oct 2012 12:46:43 -0000 Ivan Voras wrote: > On 20 October 2012 13:42, Nikolay Denev wrote: > > > Here are the results from testing both patches : > > http://home.totalterror.net/freebsd/nfstest/results.html > > Both tests ran for about 14 hours ( a bit too much, but I wanted to > > compare different zfs recordsize settings ), > > and were done first after a fresh reboot. > > The only noticeable difference seems to be much more context > > switches with Ivan's patch. > > Thank you very much for your extensive testing! > > I don't know how to interpret the rise in context switches; as this is > kernel code, I'd expect no context switches. I hope someone else can > explain. > Don't the mtx_lock() calls spin for a little while and then context switch if another thread still has it locked? > But, you have also shown that my patch doesn't do any better than > Rick's even on a fairly large configuration, so I don't think there's > value in adding the extra complexity, and Rick knows NFS much better > than I do. > Hmm, I didn't look, but were there any tests using UDP mounts? (I would have thought that your patch would mainly affect UDP mounts, since that is when my version still has the single LRU queue/mutex. As I think you know, my concern with your patch would be correctness for UDP, not performance.) Anyhow, sounds like you guys are having fun with it and learning some useful things. Keep up the good work, rick > But there are a few things other than that I'm interested in: like why > does your load average spike almost to 20-ties, and how come that with > 24 drives in RAID-10 you only push through 600 MBit/s through the 10 > GBit/s Ethernet. Have you tested your drive setup locally (AESNI > shouldn't be a bottleneck, you should be able to encrypt well into > Gbyte/s range) and the network? > > If you have the time, could you repeat the tests but with a recent > Samba server and a CIFS mount on the client side? This is probably not > important, but I'm just curious of how would it perform on your > machine. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org"