From owner-freebsd-fs@FreeBSD.ORG Thu Aug 2 18:48:32 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7927106564A for ; Thu, 2 Aug 2012 18:48:32 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 7C8D78FC0C for ; Thu, 2 Aug 2012 18:48:32 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id ED70DDE2D56; Thu, 2 Aug 2012 20:41:29 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 14.9538] X-CRM114-CacheID: sfid-20120802_20412_00FD67CC X-CRM114-Status: Good ( pR: 14.9538 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Thu Aug 2 20:41:29 2012 X-DSPAM-Confidence: 0.7009 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 501ac9d9723861388317973 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00010, FreeBSD, 0.00044, FreeBSD, 0.00044, bytes, 0.00322, FreeBSD+8, 0.00497, bug, 0.00513, bug, 0.00513, this?, 0.00547, which+may, 0.00683, command, 0.00779, null, 0.00779, the+machine, 0.00779, machines, 0.00779, metadata, 0.00779, User-Agent*i686, 0.00861, User-Agent*Linux+i686, 0.00961, see+a, 0.00990, controllers, 0.01000, Subject*part, 0.99000, Date*02+Aug, 0.99000, everything+was, 0.01000, RAID, 0.01000, SCSI, 0.01000, be+relevant, 0.01000, now+that, 0.99000, on+FreeBSD, 0.01000, X-Spambayes-Classification: ham; 0.00 Received: from [192.168.3.2] (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id AC6BBDE2D4A for ; Thu, 2 Aug 2012 20:41:28 +0200 (CEST) Message-ID: <501AC9D6.6080409@fsn.hu> Date: Thu, 02 Aug 2012 20:41:26 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: ZFS thinks that a newly inserted emtpy disk is part of the pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2012 18:48:32 -0000 Hi, What I do (did always on FreeBSD 8): - wait for a disk to malfunction (its SCSI device disappears) or when I know its bad (SMART info, checksum errors etc), pull it out from the enclosure - insert a new disk, straight from the shop (has a lot of null bytes on it) - zpool replace pool daX when the device comes up again This has previously resulted in zfs to resilver the replaced disk and everything was OK. We have switched those machines to 9 sometimes in the near past (r237433) and the above has changed. The disk disappears, gets physically replaced, reappears, and zpool replace says now that the disk is already part of the pool. I can even see a zfs signature on it with dd. After rebooting the machine, I can issue the zpool replace command without any problems, and zfs starts to rebuild its contents. (I have no dd-data from this state, sorry) Additional information which may be relevant: the drives are hooked up to Smart Array (ciss) controllers, and they are RAID 0 volumes (one logical drive per physical drive). I thought about a ciss firmware bug (caching the zfs metadata even after the disk has been replaced), but this is so weird, and should affect both FreeBSD 8 and 9. So from this I guess it's a FreeBSD bug, which I couldn't see on 8. Any ideas about what could cause this?