From owner-freebsd-fs@FreeBSD.ORG Sat May 22 23:40:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FCFF106564A; Sat, 22 May 2010 23:40:42 +0000 (UTC) (envelope-from alex.bakhtin@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3FB668FC0A; Sat, 22 May 2010 23:40:40 +0000 (UTC) Received: by fxm4 with SMTP id 4so2277654fxm.13 for ; Sat, 22 May 2010 16:40:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references :in-reply-to:subject:date:message-id:mime-version:content-type :content-transfer-encoding:x-mailer:thread-index:content-language; bh=xVpHHIJ34RbvSdQ0J8gyVZdPf5hoeQN511tZ50u2nBg=; b=awtKjlyibNbrtbYms/jk33HfHL2bkxPaAjS/smOuYgnLPKvMRR1Qq4Ndx2rZooW9P6 t7FVXNoW3sELSrFq4lJSg6FHyHoJNodw3NxRBwt8FryqSmltmYiWtkYQrrVcwyRN1UJ4 gv+/6CU+Mpai4LwgP3W+UXPnck0KI/1kgptOk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-type:content-transfer-encoding:x-mailer :thread-index:content-language; b=eTyzbwtsDInLZZNC0FSGZq/3ju2kYVgLhJCuOHX9FmTSSnlYaiib2+oDDlbtwz3ZWt Fd+6xuJoJVs2lZLRzgWlPA30VOYxulKEuYsQBcBjW8r/Rq8aT/l3vbBZ1EspUTJo2uVL aZp+Bsj9Q9+XgiktPOv9uJOjr9hy5b8ZD/Gcw= Received: by 10.223.144.77 with SMTP id y13mr3226494fau.86.1274571640085; Sat, 22 May 2010 16:40:40 -0700 (PDT) Received: from BAKHTINNB (nateneshi.static.corbina.ru [85.21.246.62]) by mx.google.com with ESMTPS id z12sm11696663fah.21.2010.05.22.16.40.38 (version=SSLv3 cipher=RC4-MD5); Sat, 22 May 2010 16:40:39 -0700 (PDT) From: Alex Bakhtin To: References: <201005130934.o4D9YiJL039462@freefall.freebsd.org> In-Reply-To: Date: Sun, 23 May 2010 03:40:38 +0400 Message-ID: <4bf86b77.0c3fdf0a.45ff.4f6b@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Acr11r3SaNvSr8JYQISoBtL7GpzldgELaWQQ Content-Language: ru Cc: freebsd-fs@freebsd.org, bug-followup@freebsd.org Subject: RE: kern/145339: [zfs] deadlock after detaching block device from raidz pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 May 2010 23:40:42 -0000 Pawel, I made some additional testing. Now I'm 95 percent sure that this deadlock was introduced by this patch. I tried patched and non-patched GENERIC kernel. It seems that it is harder to reproduce this deadlock on raidz1 than raidz2. With raidz2 I tried to detach and reattach back two = disk at the same time, and deadlock is 100% reproducible on patched kernel = but I can't reproduce it on non-patched kernel. How to reproduce: 1. Create raiz2 pool 2. Detach two devices while the pool is idle. 3. Start writing to the pool (dd if=3D/dev/zero of=3D/storage/test = bs=3D1m) 4. atacontrol detach/attach to have disks back. 5. Online two disks at the same time (zpool online storage adX adY). 6. Wait some time (in my testing - several seconds, less than one = minute) - all disk activity would be stopped. After that it's impossible to abort zpool online or dd command. Also it's impossible to reboot without hard-reset. If you need core from deadlocked kernel please let me know. Alex Bakhtin -----Original Message----- From: Alex Bakhtin [mailto:alex.bakhtin@gmail.com]=20 Sent: Monday, May 17, 2010 7:37 PM To: pjd@freebsd.org Cc: freebsd-fs@freebsd.org; bug-followup@freebsd.org Subject: Re: kern/145339: [zfs] deadlock after detaching block device = from raidz pool Pawel, I tested your patch in the following zfs configuration (all on 5x2TB WD20EARS drivers): 1. raidz1 on top of physical disks. 2. raidz1 on top of geli 3. raidz2 on top of physical disks. In all three cases it seems that the problem was fixed - I can't crash zfs in vdev_geom when unplugging the disk. Unfortunately, 3 times I got a deadlock in zfs after plugging vdevs back under load. It happens several seconds after zpool online command. I'm not 100 percent sure that deadlocks are related to this patch, but... I'm going to make some additional testing with patched and not patched kernels. 2010/5/13 : > Synopsis: [zfs] deadlock after detaching block device from raidz pool > > State-Changed-From-To: open->feedback > State-Changed-By: pjd > State-Changed-When: czw 13 maj 2010 09:33:20 UTC > State-Changed-Why: > Could you try this patch: > > =9A =9A =9A = =9Ahttp://people.freebsd.org/~pjd/patches/vdev_geom.c.3.patch > > It is against most recent HEAD. If it is rejected on 8-STABLE, just = grab > entire vdev_geom.c from HEAD and patch this. > > > Responsible-Changed-From-To: freebsd-fs->pjd > Responsible-Changed-By: pjd > Responsible-Changed-When: czw 13 maj 2010 09:33:20 UTC > Responsible-Changed-Why: > I'll take this one. > > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D145339 >