From owner-freebsd-stable@FreeBSD.ORG Tue Jul 15 19:27:05 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95CAC1065670 for ; Tue, 15 Jul 2008 19:27:05 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 3D9E38FC14 for ; Tue, 15 Jul 2008 19:27:05 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: (qmail 63459 invoked by uid 89); 15 Jul 2008 19:30:22 -0000 Received: from unknown (HELO ?IPv6:2607:f118::5?) (steve@ibctech.ca@2607:f118::5) by 2607:f118::b6 with ESMTPA; 15 Jul 2008 19:30:22 -0000 Message-ID: <487CFA08.5000308@ibctech.ca> Date: Tue, 15 Jul 2008 15:27:04 -0400 From: Steve Bertrand User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Matthew Dillon References: <487CCD46.8080506@ibctech.ca> <200807151711.m6FHBgVO007481@apollo.backplane.com> <487CF077.2040201@ibctech.ca> In-Reply-To: <487CF077.2040201@ibctech.ca> X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: taskqueue timeout X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2008 19:27:05 -0000 Steve Bertrand wrote: > Matthew Dillon wrote: > >> If you are getting DMA timeouts, go to this URL: >> >> http://wiki.freebsd.org/JeremyChadwick/ATA_issues_and_troubleshooting >> >> Then I would suggest going into /usr/src/sys/dev/ata (I think, on >> FreeBSD), locate all instances where request->timeout is set to 5, >> and change them all to 10. >> >> cd /usr/src/sys/dev/ata >> fgrep 'request->timeout' *.c >> ... change all assignments of 5 to 10 ... > > Changing 5 to 10 in all cases and rebuilding the kernel does not fix the > problem. Went from 10->15, and it took quite a bit longer into the backup before the problem cropped back up. Here is what I was seeing at the time it failed. Where netstat and zpool iostat drop off is where I start seeing the errors occur: # top last pid: 1069; load averages: 0.09, 0.17, 0.10 up 0+00:08:31 19:22:39 53 processes: 1 running, 52 sleeping CPU states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 28M Active, 3644K Inact, 301M Wired, 76K Cache, 1634M Free Swap: # netstat -w 1 -h 4.8K 0 11M 3.5K 0 5.4M 0 4.5K 0 10M 3.3K 0 5.1M 0 4.9K 0 11M 3.6K 0 5.5M 0 4.8K 0 11M 3.5K 0 5.4M 0 4.3K 0 9.5M 3.1K 0 4.8M 0 5.1K 0 11M 3.7K 0 5.7M 0 5.0K 0 11M 3.6K 0 5.6M 0 5.3K 0 12M 3.9K 0 6.0M 0 4.8K 0 11M 3.5K 0 5.4M 0 4.7K 0 10M 3.4K 0 5.2M 0 4.8K 0 11M 3.5K 0 5.4M 0 4.6K 0 10M 3.4K 0 5.2M 0 4.1K 0 9.1M 3.0K 0 4.6M 0 5.3K 0 12M 3.9K 0 6.0M 0 5.2K 0 12M 3.8K 0 5.8M 0 4.3K 0 9.5M 3.1K 0 4.8M 0 4.3K 0 9.6M 3.2K 0 4.9M 0 5.4K 0 12M 4.0K 0 6.1M 0 4.8K 0 11M 3.5K 0 5.4M 0 2.4K 0 5.1M 1.7K 0 2.5M 0 input (Total) output packets errs bytes packets errs bytes colls 2 0 120 2 0 316 0 3 0 180 4 0 1.0K 0 3 0 180 2 0 316 0 3 0 180 3 0 658 0 5 0 1.6K 5 0 942 0 3 0 254 4 0 840 0 3 0 180 2 0 316 0 # zpool iostat 1 storage 6.40G 1.81T 0 296 0 37.0M storage 6.43G 1.81T 0 188 0 14.5M storage 6.43G 1.81T 0 0 0 0 storage 6.43G 1.81T 0 0 0 0 storage 6.43G 1.81T 0 0 0 0 storage 6.43G 1.81T 0 47 0 5.99M storage 6.46G 1.81T 0 218 0 18.0M storage 6.46G 1.81T 0 0 0 0 storage 6.46G 1.81T 0 0 0 0 storage 6.46G 1.81T 9 0 192K 0 storage 6.46G 1.81T 0 59 0 7.39M storage 6.49G 1.81T 1 250 3.42K 14.9M storage 6.49G 1.81T 0 0 0 0 storage 6.49G 1.81T 0 0 0 0 storage 6.49G 1.81T 0 0 0 0 storage 6.49G 1.81T 0 141 0 17.5M storage 6.52G 1.81T 0 74 0 232K storage 6.52G 1.81T 0 0 0 0 storage 6.52G 1.81T 0 0 0 0 storage 6.52G 1.81T 0 0 0 0 storage 6.52G 1.81T 0 151 0 18.8M storage 6.52G 1.81T 0 114 0 8.07M storage 6.52G 1.81T 0 0 0 0 storage 6.52G 1.81T 0 0 0 0 storage 6.52G 1.81T 0 0 0 0 storage 6.52G 1.81T 0 0 0 0 > Don't know if this will help anyone or not. Steve