From owner-freebsd-current@FreeBSD.ORG Tue Sep 15 15:54:45 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35F1910656AA; Tue, 15 Sep 2009 15:54:45 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id 8B0508FC1D; Tue, 15 Sep 2009 15:54:44 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 9F29945C8A; Tue, 15 Sep 2009 17:54:42 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 54801456B1; Tue, 15 Sep 2009 17:54:37 +0200 (CEST) Date: Tue, 15 Sep 2009 17:54:36 +0200 From: Pawel Jakub Dawidek To: Alexander Motin Message-ID: <20090915155436.GB2199@garage.freebsd.pl> References: <4AAD4E51.5060908@FreeBSD.org> <4AAD5365.5000902@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="A6N2fC+uXW/VQSAv" Content-Disposition: inline In-Reply-To: <4AAD5365.5000902@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: Kris Kennaway , FreeBSD Current Subject: Re: ata timeouts under load X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Sep 2009 15:54:45 -0000 --A6N2fC+uXW/VQSAv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 13, 2009 at 11:17:41PM +0300, Alexander Motin wrote: > Kris Kennaway wrote: > > I am getting timeouts on 8.0b4/HEAD when I do a lot of ZFS I/O to a pool > > on ad4: > >=20 > > atapci0: port > > 0xc800-0xc807,0xc400-0xc403,0xc000-0xc007,0xb800-0xb803,0xb400-0xb40f,0= xb000-0xb0ff > > irq 20 at device 15.0 on pci0 > > ata2: on atapci0 > > ata3: on atapci0 > > ata0: on atapci1 > > ata1: on atapci1 > >=20 > > ad4: 476940MB at ata2-master SATA150 > > ad4: WARNING - SETFEATURES SET TRANSFER MODE taskqueue timeout - > > completing request directly > > ad4: WARNING - SETFEATURES SET TRANSFER MODE taskqueue timeout - > > completing request directly > > ad4: WARNING - SETFEATURES ENABLE RCACHE taskqueue timeout - completing > > request directly > > ad4: WARNING - SETFEATURES ENABLE WCACHE taskqueue timeout - completing > > request directly > > ad4: WARNING - SET_MULTI taskqueue timeout - completing request directly > > ad4: TIMEOUT - WRITE_DMA48 retrying (1 retry left) LBA=3D344052040 > > ad4: WARNING - SETFEATURES SET TRANSFER MODE taskqueue timeout - > > completing request directly > > ad4: WARNING - SETFEATURES SET TRANSFER MODE taskqueue timeout - > > completing request directly > >=20 > > It becomes stuck in a loop displaying the above and is unable to > > complete further I/O operations. I wonder if it is just batching up a > > lot of I/O and then timing out because it is busy, and then not > > recovering from this state? > >=20 > > Any ideas what could be wrong? >=20 > There are two different kinds of timeouts we can see: > - first one, "ad4: WARNING - ..." is just a queue waiting timeout. It > is not the reason, but consequence of the problem. And I have doubts > that it is reasonable to do it. > - second one, "TIMEOUT - WRITE_DMA48 ..." is a real command execution > timeout. I don't know whether this is result of some improper error > recovery, or you drive indeed lost required servo information near > LBA=3D344052040 and tries to find it too long. You can try to read that > sector and nearby ones with dd. Could this be related to BIO_FLUSH requests? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --A6N2fC+uXW/VQSAv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKr7i8ForvXbEpPzQRAjDaAKDKTb9Xl6KXNgVYQH7JrhaUBbIauwCgiZa6 kDiftv2qOLh6T0GYwbPV7Ag= =7NCB -----END PGP SIGNATURE----- --A6N2fC+uXW/VQSAv--