From owner-freebsd-stable@FreeBSD.ORG Fri Sep 16 15:31:06 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 802D91065673 for ; Fri, 16 Sep 2011 15:31:06 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 3CB6E8FC12 for ; Fri, 16 Sep 2011 15:31:05 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap8EAGZqc06DaFvO/2dsb2JhbABBhFWjfYFTAQEBAQIBAQEBICsgCwUWDgoCAg0ZAikBCSYGCAcEARwEh1YEomGRa4EshDuBEQSRNIIWkUg X-IronPort-AV: E=Sophos;i="4.68,394,1312171200"; d="scan'208";a="134744921" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 16 Sep 2011 11:31:04 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 82E1CB4017; Fri, 16 Sep 2011 11:31:04 -0400 (EDT) Date: Fri, 16 Sep 2011 11:31:04 -0400 (EDT) From: Rick Macklem To: Robert Schulze Message-ID: <376339967.1495550.1316187064510.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4E734803.2050203@bytecamp.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-stable@freebsd.org Subject: Re: increase in dropped udp datagrams X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Sep 2011 15:31:06 -0000 Robert Schulze wrote: > Hi, > > we are running an NFS server (8.2-STABLE/amd64 from 2011-09-08) with > several 8.1-RELEASE-p1/amd64 clients (webservers). After updating the > server, we now realize a slight increase in udp datagrams "dropped due > to full socket buffers", which wasn't the case in the previously > installed 8.2-STABLE from 2011-05-08. This also results in higher > averages in HTTP request time on the clients. > Well, there isn't much that changed in the krpc or NFS server's handling of UDP during that period. The only change in the krpc is r225384 and I can't think how that would have this affect. (All it does is avoid a loop in the kernel when msleep() returns EINTR or ERESTART.) > Nothing was changed on the configuration regarding sysctls or loader > tuneables. > > kern.ipc.maxsockbuf is at 1024000 on the server. > > Was there a change which could cause the drops? Should this tuneable > be > increased further? > You can certainly try this. (It defaults to 2*1024*1024 in -current now.) > Regarding maxsockbuf: is this number per-socket? So does one have to > keep in mind that there could be maxsockbuf*maxsockets allocated for > UDP? > It's per-socket, but in the case of the NFS server, it only uses one UDP socket for all requests. (I have no idea what else you are running that uses UDP?) > I'm willing to provide further information upon question. > Sounds to me like you've bumped into some second-order effect. One thing I'd try (you didn't mention where you currently have it set) is bump the # of nfsd threads up. For example, you can set it to 64 by putting this line in your /etc/rc.conf. nfs_server_flags="-u -t -n 64" If you have too many of them, the extra ones just sit waiting for a request and don't take up many resources. On the other hand, if you don't have enough of them, requests will get backed up in the socket's receive queue. I'll also mention considering using TCP mounts from the clients. (I can't remember if we've already had this discussion:-) Good luck with it, rick > with kind regards, > Robert Schulze > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to > "freebsd-stable-unsubscribe@freebsd.org"