From owner-freebsd-fs@FreeBSD.ORG Tue Jun 8 23:39:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1A801065670 for ; Tue, 8 Jun 2010 23:39:28 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 5749A8FC1F for ; Tue, 8 Jun 2010 23:39:27 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEADdxDkyDaFvK/2dsb2JhbACeRnG/ZoUWBA X-IronPort-AV: E=Sophos;i="4.53,387,1272859200"; d="scan'208";a="80004278" Received: from fraser.cs.uoguelph.ca ([131.104.91.202]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 08 Jun 2010 19:39:25 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by fraser.cs.uoguelph.ca (Postfix) with ESMTP id 526E8109C2C3; Tue, 8 Jun 2010 19:39:27 -0400 (EDT) X-Virus-Scanned: amavisd-new at fraser.cs.uoguelph.ca Received: from fraser.cs.uoguelph.ca ([127.0.0.1]) by localhost (fraser.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s1hizAzPQcVt; Tue, 8 Jun 2010 19:39:26 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by fraser.cs.uoguelph.ca (Postfix) with ESMTP id D485B109C24A; Tue, 8 Jun 2010 19:39:26 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o58NtWP09898; Tue, 8 Jun 2010 19:55:32 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Tue, 8 Jun 2010 19:55:32 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Anders Nordby In-Reply-To: <20100608083649.GA77452@fupp.net> Message-ID: References: <20100608083649.GA77452@fupp.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@FreeBSD.org Subject: Re: Odd network issues on ZFS based NFS server X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2010 23:39:28 -0000 On Tue, 8 Jun 2010, Anders Nordby wrote: > Hi! > > I have a file server running 8.1-PRERELEASE amd64, where I share some > filesystems using NFS and Samba. After running for a day or two, the > server starts to get around 25% packet loss, browsing directories across > NFS gets really slow etc. Rebooting solves it until it happens again. > Has anyone experienced anything similar? I had this issue in FreeBSD 7 > as well, upgrading did not help. > Well, here's a few things you might try. (I know nothing about ZFS, except what I see discussed on the mailing lists.) - "netstat -m" will show you mbuf allocations. Might give you a hint w.r.t. mbuf/mbuf cluster exhaustion. - I'd try setting zio_use_uma = 0, since there have been reports of issues related to ZFS using the uma allocator and mbuf allocation uses the uma allocator now, too. (I think this is fairly recent, so might not be relevant to FreeBSD7.) - You can try the experimental NFS server to see if that affects the behaviour. ("-e" option on both mountd and nfsd) - If you have some different network hardware, you could try a different net interface. This would isolate the problem, if it happens to be related to the network device driver for the hardware you have. There are lots of email messages in the archive related to tuning the arc for zfs. I know nothing about it, but I'd look for a message that describes what the current recommendations are for amd64 w.r.t. this. Hopefully others can suggest other things to check. It smells like some sort of resource exhaustion problem, but who knows??? Good luck with it, rick