From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 20 21:03:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D8D84E2D; Sat, 20 Oct 2012 21:03:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 79C038FC0A; Sat, 20 Oct 2012 21:03:10 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap8EAPUQg1CDaFvO/2dsb2JhbAA7CIYUvAKCIAEBAQMBAQEBICsgCxsYAgINGQIpAQkmBggHBAEcBIddBguoYJITgSCKPxYEhUOBEgOTRIItgRePIoMLgUc1 X-IronPort-AV: E=Sophos;i="4.80,622,1344225600"; d="scan'208";a="184554476" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 20 Oct 2012 17:03:09 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id BBB63B402D; Sat, 20 Oct 2012 17:03:09 -0400 (EDT) Date: Sat, 20 Oct 2012 17:03:09 -0400 (EDT) From: Rick Macklem To: Outback Dingo Message-ID: <1800695432.2577499.1350766989710.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: Subject: Re: NFS server bottlenecks MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE7 (Win)/6.0.10_GA_2692) Cc: "freebsd-hackers@freebsd.org Hackers" , Ivan Voras , Nikolay Denev X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Oct 2012 21:03:12 -0000 Outback Dingo wrote: > On Sat, Oct 20, 2012 at 3:28 PM, Ivan Voras > wrote: > > On 20 October 2012 14:45, Rick Macklem wrote: > >> Ivan Voras wrote: > > > >>> I don't know how to interpret the rise in context switches; as > >>> this is > >>> kernel code, I'd expect no context switches. I hope someone else > >>> can > >>> explain. > >>> > >> Don't the mtx_lock() calls spin for a little while and then context > >> switch if another thread still has it locked? > > > > Yes, but are in-kernel context switches also counted? I was assuming > > they are light-weight enough not to count. > > > >> Hmm, I didn't look, but were there any tests using UDP mounts? > >> (I would have thought that your patch would mainly affect UDP > >> mounts, > >> since that is when my version still has the single LRU > >> queue/mutex. > > > > Another assumption - I thought UDP was the default. > > TCP has been the default for a FreeBSD client for a long time. It was changed for the old NFS client before I became a committer. (You can explicitly set one or the other as mount options or check via wireshark/tcpdump) > >> As I think you know, my concern with your patch would be > >> correctness > >> for UDP, not performance.) > > > > Yes. > > Ive got a similar box config here, with 2x 10GB intel nics, and 24 2TB > drives on an LSI controller. > Im watching the thread patiently, im kinda looking for results, and > answers, Though Im also tempted to > run benchmarks on my system also see if i get similar results I also > considered that netmap might be one > but not quite sure if it would help NFS, since its to hard to tell if > its a network bottle neck, though it appears > to be network related. > NFS network traffic looks very different that a TCP stream (ala bit torrent or ...). I've seen this cause issues before. You can look at a packet trace in wireshark and see if TCP is retransmitting segments. rick > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to > > "freebsd-hackers-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org"