From owner-freebsd-stable@FreeBSD.ORG Thu Nov 22 08:51:14 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7349116A419; Thu, 22 Nov 2007 08:51:14 +0000 (UTC) (envelope-from root@kash.tomsk.ru) Received: from mx.kash.tomsk.ru (ns2.kash.tomsk.ru [88.204.35.2]) by mx1.freebsd.org (Postfix) with ESMTP id AA97C13C469; Thu, 22 Nov 2007 08:51:13 +0000 (UTC) (envelope-from root@kash.tomsk.ru) Received: by mx.kash.tomsk.ru (Postfix, from userid 0) id 01B97DAE37; Thu, 22 Nov 2007 02:29:57 +0600 (NOVT) Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by mx.kash.tomsk.ru (Postfix) with ESMTP id 1AE2ADACCD for ; Thu, 22 Nov 2007 02:29:56 +0600 (NOVT) Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id A911D184FD; Wed, 21 Nov 2007 20:29:24 +0000 (UTC) (envelope-from owner-freebsd-hackers@freebsd.org) Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 6F6D516A475; Wed, 21 Nov 2007 20:29:23 +0000 (UTC) (envelope-from owner-freebsd-hackers@freebsd.org) Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD1DE16A421; Wed, 21 Nov 2007 20:18:51 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (pointyhat.freebsd.org [IPv6:2001:4f8:fff6::2b]) by mx1.freebsd.org (Postfix) with ESMTP id 316E913C455; Wed, 21 Nov 2007 20:18:50 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <474492B0.1010108@FreeBSD.org> Date: Wed, 21 Nov 2007 21:18:56 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Kris Kennaway References: <47137D36.1020305@chistydom.ru> <47149E6E.9000500@chistydom.ru> <4715035D.2090802@FreeBSD.org> <4715C297.1020905@chistydom.ru> <4715C5D7.7060806@FreeBSD.org> <471EE4D9.5080307@chistydom.ru> <4723BF87.20302@FreeBSD.org> <47344E47.9050908@chistydom.ru> <47349A17.3080806@FreeBSD.org> <47373B43.9060406@chistydom.ru> <4739557A.6090209@chistydom.ru> <4741EE9E.9050406@FreeBSD.org> In-Reply-To: <4741EE9E.9050406@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Sender: owner-freebsd-hackers@freebsd.org Errors-To: owner-freebsd-hackers@freebsd.org X-DSPAM-Result: Innocent X-DSPAM-Processed: Thu Nov 22 02:29:57 2007 X-DSPAM-Confidence: 0.9990 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 47449545191321860720487 X-DSPAM-Factors: 27, >+Alexey, 0.00028, Alexey, 0.00043, Sender*owner+freebsd, 0.00053, Sender*freebsd, 0.00053, List-Post*freebsd, 0.00053, List-Post*, Alexey Popov , freebsd-stable@freebsd.org Subject: Re: amrd disk performance drop after running under high load X-BeenThere: freebsd-stable@freebsd.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Nov 2007 08:51:14 -0000 Kris Kennaway wrote: > Alexey Popov wrote: >> Hi. >> >> Panagiotis Christias wrote: >>>>>>> In the "good" case you are getting a much higher interrupt rate but >>>>>>> with the data you provided I can't tell where from. You need to run >>>>>>> vmstat -i at regular intervals (e.g. every 10 seconds for a minute) >>>>>>> during the "good" and "bad" times, since it only provides counters >>>>>>> and an average rate over the uptime of the system. >>>>>> Now I'm running 10-process lighttpd and the problem became no so big. >>>>>> >>>>>> I collected interrupt stats and it shows no relation beetween >>>>>> ionterrupts and slowdowns. Here is it: >>>>>> http://83.167.98.162/gprof/intr-graph/ >>>>>> >>>>>> Also I have similiar statistics on mutex profiling and it shows >>>>>> there's no problem in mutexes. >>>>>> http://83.167.98.162/gprof/mtx-graph/mtxgifnew/ >>>>>> >>>>>> I have no idea what else to check. >>>>> I don't know what this graph is showing me :) When precisely is the >>>>> system behaving poorly? >>> what is your RAID controller configuration (read ahead/cache/write >>> policy)? I have seen weird/bogus numbers (~100% busy) reported by >>> systat -v when read ahead was enabled on LSI/amr controllers. >> >> >> ********************************************************************** >> Existing Logical Drive Information >> By LSI Logic Corp.,USA >> >> ********************************************************************** >> [Note: For SATA-2, 4 and 6 channel controllers, please specify >> Ch=0 Id=0..15 for specifying physical drive(Ch=channel, >> Id=Target)] >> >> >> Logical Drive : 0( Adapter: 0 ): Status: OPTIMAL >> --------------------------------------------------- >> SpanDepth :01 RaidLevel: 5 RdAhead : Adaptive Cache: >> DirectIo >> StripSz :064KB Stripes : 6 WrPolicy: WriteBack >> >> Logical Drive 0 : SpanLevel_0 Disks >> Chnl Target StartBlock Blocks Physical Target Status >> ---- ------ ---------- ------ ---------------------- >> 0 00 0x00000000 0x22ec0000 ONLINE >> 0 01 0x00000000 0x22ec0000 ONLINE >> 0 02 0x00000000 0x22ec0000 ONLINE >> 0 03 0x00000000 0x22ec0000 ONLINE >> 0 04 0x00000000 0x22ec0000 ONLINE >> 0 05 0x00000000 0x22ec0000 ONLINE >> >> I tried to run with disabled Read-ahead, but it didn't help. > > I just ran into this myself, and apparently it can be caused by "Patrol > Reads" where the adapter periodically scans the disks to look for media > errors. You can turn this off using -stopPR with the megarc port. > > Kris > Oops, -disPR is the correct command to disable, -stopPR just halts a PR event in progress. Kris _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"