From owner-freebsd-questions@FreeBSD.ORG Fri Jan 16 16:45:22 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D7C9F16A4CE for ; Fri, 16 Jan 2004 16:45:22 -0800 (PST) Received: from gw.opentable.com (gw.opentable.com [63.89.164.120]) by mx1.FreeBSD.org (Postfix) with SMTP id CF9DD43D64 for ; Fri, 16 Jan 2004 16:45:20 -0800 (PST) (envelope-from efreis@opentable.com) Received: (qmail 69206 invoked from network); 17 Jan 2004 00:45:22 -0000 Received: from unknown (HELO mail.opentable.com) (10.0.0.6) by ns1.opentable.com.ot with SMTP; 17 Jan 2004 00:45:22 -0000 Received: by mail.opentable.com with Internet Mail Service (5.5.2657.72) id ; Fri, 16 Jan 2004 16:43:50 -0800 Message-ID: <34BD9C97B714D511B01C00D0B73EC07303EEE654@mail.opentable.com> From: Elliott Freis To: "'freebsd-questions@FreeBSD.ORG'" Date: Fri, 16 Jan 2004 16:43:43 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain; charset="iso-8859-1" Subject: Odd lag/hanging issue with production ftp server - Please help AS AP! X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2004 00:45:23 -0000 > I have been struggling with this one for over a month now. Here is a > quick layout of my setup: > > Primary FTP server: > Compaq DL380 1gb ram > FreeBSD 4.5 > 3x36gb RAID 5 drives as local boot/storage > ProFTPd > > Array server for FTP: > AMD Athlon 2200 512mb ram > FreeBSD 4.8 > 7x36gb Fiber channel drives, RAID 5 via Vinum. > > Both machines are connected via a cross-over cable, that has been tested > good and swapped just in case. The primary storage for FTP is done on the > Fiber drives via nfs from FTP to Array server. NFS options are -U -3. > > My problem is this. As more users connect and store files, the primary > FTP machine becomes increasingly unresponsive. Currently, I max at about > 350 concurrent FTP connections. The most basic test I have been doing is > just holding down enter on an SSH session. As you hold enter down, you > see it visually just hang for a second or more (up to about 5 seconds > depending on the load). It is even worse if I spam "df -k" for example. > For part of the time, it responds fine, though its randomly a second to > multiple seconds. In other words, it is randomly responsive and not > responsive every 5 seconds or so. During the "hanging" time, ftp sessions > are also hung. So you see very bursty data transfers. Now thankfully, no > ftp sessions drop, so we do get the data we need. But this is a terrible > thing to be happening to a production server. > > One other thing of note, this happened to me about 4-5 months ago, but a > reboot eventually fixed it for some reason. So I concluded it was just a > hiccup. But it has returned after a different reboot, and won't go away. > Also, if I move the ftp logins to the local drives (which I can't do > permanently due to lack of space) there is no problem at all, no lag. > > Any help in troubleshooting this is very appreciated! Happy new year, > > -Elliott > > Example of "enter" latency (this is from a LAN connection): > At shell prompt ">" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >