From owner-freebsd-performance@FreeBSD.ORG Tue Jun 10 01:26:40 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BFBEE37B404 for ; Tue, 10 Jun 2003 01:26:40 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33B3343F3F for ; Tue, 10 Jun 2003 01:26:40 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from user-38lc1ah.dialup.mindspring.com ([209.86.5.81] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19PeSj-0000vk-00; Tue, 10 Jun 2003 01:26:38 -0700 Message-ID: <3EE595D2.B223CA19@mindspring.com> Date: Tue, 10 Jun 2003 01:24:50 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Eric Anderson References: <20030609211526.58641.qmail@web14908.mail.yahoo.com> <3EE4FAED.6090603@centtech.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4658d19a7afa5b5fe8205ece409146b5d350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: freebsd-performance@freebsd.org Subject: Re: Slow disk write speeds over network X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2003 08:26:41 -0000 Eric Anderson wrote: > >>No, considering I would have to update about 600 hosts. Plus, those > >>same hosts are running just fine with my other file servers at the > >>moment. > > > > If your server's network buffers are ok, I would guess that the > > "bottleneck" is on your clients machines. > > > > just my 2 cents... > > Paulo, thanks for the thoughts, however, that doesn't make a whole lot > of sense (although I see why you are saying that), because in this case > I have several fileservers on the same gigabit switches as this > fileserver, and clients can transfer data quickly between all, in both > directions, all the time, except data being sent TO this one host. I've > even tried machines that are idle, on the same switch as this server, > and I still receive slow transfers. Swap cables with another box. BTW: 4 Gigabit cards in one box, with you going to local disk... you've got about 8 cards worth of traffic over your PCI bus. Unless this is a PCI-X based box, you are most likely livelocked; even if it's a PCI-X based box, you could still be livelocked. You haven't said if you were using UDP or TCP for the mounts; you should definitely use TCP with FreeBSD NFS servers; it's also just generally a good idea, since UDP frags act as a fixed non-sliding window: NFS over UDP sucks. Also, you haven't said whether you are using aliases on your network cards; aliases and NFS tend to interact badly. Finally, you probably want to tweak some sysctl's, e.g. net.inet.ip.check_interface=0 net.inet.tcp.inflight_enable=1 net.inet.tcp.inflight_debug=0 net.inet.tcp.msl=3000 net.inet.tcp.inflight_min=6100 net.isr.enable=1 Given your overloading of your bus, that last one is probably the most important one: it enables direct dispatch. You'll also want to enable DEVICE_POLLING in your kernel config file (assuming you have a good ethernet card whose driver supports it): options DEVICE_POLLING options HZ=2000 ...and yet more sysctl's for this: kern.polling.enable=1 kern.polling.user_frac=50 # 0..100; whatever works best If you've got a really terrible Gigabit Ethernet card, then you may be copying all your packets over again (e.g. m_pullup()), and that could be eating your bus, too. -- Terry