From owner-freebsd-stable@FreeBSD.ORG Fri Jan 28 20:58:04 2005 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E37A516A4CE for ; Fri, 28 Jan 2005 20:58:04 +0000 (GMT) Received: from rift.creativeimage.ca (rift.creativeimage.ca [66.207.220.235]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2033B43D49 for ; Fri, 28 Jan 2005 20:58:04 +0000 (GMT) (envelope-from oren@creativeimage.ca) Received: (from www@localhost) by rift.creativeimage.ca (8.11.6/8.11.6) id j0SKwex00997 for freebsd-stable@freebsd.org; Fri, 28 Jan 2005 15:58:40 -0500 (EST) (envelope-from oren@creativeimage.ca) To: freebsd-stable@freebsd.org Message-ID: <1106945920.41faa7806f30b@secure.creativeimage.ca> Date: Fri, 28 Jan 2005 15:58:40 -0500 (EST) From: Oren Baum MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: IMP/PHP IMAP webmail program 2.2.7 X-Originating-IP: 24.85.75.45 Subject: Seemingly odd disc i/o behaviour, need help to diagnose X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 20:58:05 -0000 We're having a hard to track down a recurring problem that we think has some problem to do with disc i/o access. This is slightly beyond our level of expertise and were hoping someone could shed some light on the following problem. If this has been dealt with previously, please let me know under what topic so I can do relevant searches. Description of Server configuration and problems. Configuration: FreeBSD 4.4-RELEASE apache+mod_ssl-1.3.22+2.8.5_1 mysql-server-3.23.42 Hard Drive (dual ATA disks, no RAID) on a Dell P4 PowerEDGE server We had many speed and timeout issues so we recompiled the kernel with maxusers=128 instead of the previous 32 and moved the hard drives into a new P4 2.8Ghz PowerEdge Case. atapci0: port 0xffa0-0xffaf,0x374-0x377,0x170- 0x177,0x3f4-0x3f7,0x1f0-0x1f7 irq 11 at device 31.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 atapci1: port 0xfea0-0xfeaf,0xfe30-0xfe33,0xfe20- 0xfe27,0xfe10-0xfe13,0xfe00-0xfe07 mem 0xdff3fc00-0xdff3ffff irq 5 at device 31.2 on pci0 ata2: at 0xfe00 on atapci1 ata3: at 0xfe20 on atapci1 This cleared up all errors found in fstat and the "file table is full" errors. Server is configured to hold 2 medium sized MySQL DB's accessed through various perl and php scripts via websites on the server. Uptime: 78463 Threads: 18 Questions: 351685 Slow queries: 328 Opens: 2793 Flush tables: 1 Open tables: 64 Queries per second avg: 4.482 Each httpd process is of size 15MB to 20MB * about 15 processes Mysqld process is of size 32MB, resident 16MB Server load is now usually between .24 and .44 With the new kernel, we observed the server for about a week and are trying to pinpoint this issue: If we run pine on a large mailbox or any other disk i/o intensive task, all other processes in motion seem to stall until the disk i/o is complete. This manifests itself in timeouts on webpages that require DB data, IMAP timeouts for other mail accounts, and even odd console/shell behaviour. For instance, at a console/ssh prompt we would run "uptime" or "pwd" and there would be up to a 10 second delay before results are returned. The results also show no heavy load (<.40) on the CPU. Rebooting the server eliminates speed issues for up to a couple of hours. Any thoughts on how to diagnose if this really is a disc I/O issue and how to resolve it would be most helpful! Thank you, __________________________________ Oren Baum Creative Image Communications Inc. oren@creativeimage.ca