From owner-freebsd-fs@freebsd.org Tue Dec 12 14:42:58 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E585E9AAC9 for ; Tue, 12 Dec 2017 14:42:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6B1F58051D for ; Tue, 12 Dec 2017 14:42:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCEgwrA031696 for ; Tue, 12 Dec 2017 14:42:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224272] zfs replace doesn't detach old device Date: Tue, 12 Dec 2017 14:42:58 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 14:42:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224272 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 16:09:09 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4CBB9E9D62C; Tue, 12 Dec 2017 16:09:09 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com [209.85.215.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A79C063913; Tue, 12 Dec 2017 16:09:06 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f45.google.com with SMTP id f13so23768288lff.12; Tue, 12 Dec 2017 08:09:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=rEjAXKiNhiLxxae9bIErnts23buG/BnfSx+DA8Wip2U=; b=FkBVPhDto8eHlfyGECibI4RgYCLpzZqWPXeGDiFxqzzW5XYnU058w6JvMuHpttEUJV GL0I8ZjEVtfr4gPNqKC0Ip0TnraQYWtccbHmIdF7L95quQt35zgUIozoBQHc7DpFkxAo T0Cjkzc4SzC2SQy7s+x3O+37AOVxTD2NnQwQxMod1cE9gy85AkN0dKzNwwZVdfU1Peip NqbzyOos2i0uB4XZBhufNaJDCxoRHUixqkwn9zUbyoxv5YGrYJ+zKv96vjgHOteLjx7t AZ0ueZ2Y+SuWlKwoWtt5JwrtasLFhu569seIfUUhBvv7suwJixo0x4+7rEs0DvKsr4sI uH1A== X-Gm-Message-State: AKGB3mLWBCAh6fJfJkstj/J9P5dNvwmmK+sI9bSBLdYA742sPRnAZM6M rvYFTa2JlcTX9uufTiGYfX/8SYX9 X-Google-Smtp-Source: ACJfBouOA4QjF577to/UTgbuAvsNVNaMONlIUeldMj8WleFv6as/M9oWVvTHFVdkTE6Je5zWXKCHVA== X-Received: by 10.46.42.130 with SMTP id q124mr2129583ljq.59.1513094937404; Tue, 12 Dec 2017 08:08:57 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id q11sm3349169lje.87.2017.12.12.08.08.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Dec 2017 08:08:55 -0800 (PST) Subject: Re: add BIO_NORETRY flag, implement support in ata_da, use in ZFS vdev_geom To: Poul-Henning Kamp , Scott Long Cc: FreeBSD FS , Warner Losh , freebsd-geom@FreeBSD.org References: <391f2cc7-0036-06ec-b6c9-e56681114eeb@FreeBSD.org> <64f37301-a3d8-5ac4-a25f-4f6e4254ffe9@FreeBSD.org> <39E8D9C4-6BF3-4844-85AD-3568A6D16E64@samsco.org> <30379.1511609830@critter.freebsd.dk> From: Andriy Gapon Message-ID: <38078290-ce16-d3a6-2256-c9b7fec17e72@FreeBSD.org> Date: Tue, 12 Dec 2017 18:08:54 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <30379.1511609830@critter.freebsd.dk> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 16:09:09 -0000 On 25/11/2017 13:37, Poul-Henning Kamp wrote: > The real fundamental deficiency is that we do not have a way to say "give up > if this bio cannot be completed in X time" which is what people actually want. Indeed. And I think that that was also what Warner tried to help me understand. That it is not about absolute retry count, but about a time budget for a request. > That is suprisingly hard to provide, there are far too many > corner-cases for me to enumerate them all, but let me just give one > example: This is true and this is a good example. I think that we might want to try first to handle simpler cases like deciding whether to retry a request if we get a transient error Dealing with a request that just doesn't come back is the much harder piece, of course. > Imagine you issue a deadlined write to a RAID5 thing. Thee component > writes happen smoothly, but the last two fail the deadline, with > no way to predict how long time it will take before they complete > or fail. > > * Does the bio write transaction fail ? > > * Does the bio write transaction time out ? > > * Do you attempt to complete the write to the RAID5 ? > > * Where do you store a copy of the data if you do ? > > * What happens next time a read happens on this bio's extent ? > > Then for an encore, imagine it was a read bio: Three DMAs go smoothly, > two are outstanding and you don't know if/when they will complete/fail. > > * If you fail or time out the bio, how do you "taint" the space > being read into until the two remaining DMAs are outstanding? > > * What if that space is mapped into userland ? > > * What if that space is being executed ? > > * What if one of the two outstanding DMAs later return garbage ? > > My conclusion back when I did GEOM, was that the only way to > do something like this sanely, is to have a special GEOM do it > for you, which always allocates a temp-space: > > allocate temp buffer > if (write) > copy write data to temp buffer > issue bio downwards on temp buffer > if timeout > park temp buffer until biodone > return(timeout) > if (read) > copy temp buffer to read space > return (ok/error) -- Andriy Gapon From owner-freebsd-fs@freebsd.org Tue Dec 12 16:20:13 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36C5AE9DBA9; Tue, 12 Dec 2017 16:20:13 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f48.google.com (mail-lf0-f48.google.com [209.85.215.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D08A6640B0; Tue, 12 Dec 2017 16:20:12 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f48.google.com with SMTP id x20so23869243lff.1; Tue, 12 Dec 2017 08:20:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=pvyxcyeBsAeXccLTKSTHyn8rJf8X375ECowLUb6YoL8=; b=cJiZLKP6eZXYIHwjDf2xbjeCoEiYE40Rvath+CbmVoFnMdL49zL0IGcNab7lJgkgoC wj26joQFYJLlBelrs9dJg2MBB+CwI3GwDeinkl4ZmLfnhizqjzO1IEi4c0X4fEC8aLf3 KH8+b4Kfl82OLBpKTKlk9LH71hLdHHYlGyZnGfvrIHNScIsTapVqJjrKDnqKDsc5B87X BsuZllnGavfy2hHIjvLwkMlM1w72VxpO50M57n9eQ1I2sAkI544X4e6FeZj7VhMgGox0 8e+7P8d+pLqKRXYB0bxpeUWFe5iL4SgJNY3LzyF9NaGmBYliQOiylOPkPav2JI5D/HUO WgRQ== X-Gm-Message-State: AKGB3mK1EhkqIwke6Vxns3eZ27dKPTgizP+wevdnotiUHG0bpa3esAV+ aZdsoYFuFGrSgzaz1oTdmQFlo9mG X-Google-Smtp-Source: ACJfBosN7UGwbsGu3w0DX4edklyH7oX2rwxJhKs8v1aXezug/WriC8eki5Asqs1gV8j48iW0+3EhOA== X-Received: by 10.25.242.65 with SMTP id d1mr1968326lfk.18.1513095149607; Tue, 12 Dec 2017 08:12:29 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id r90sm3352936lje.80.2017.12.12.08.12.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Dec 2017 08:12:29 -0800 (PST) Subject: Re: add BIO_NORETRY flag, implement support in ata_da, use in ZFS vdev_geom To: Scott Long Cc: Warner Losh , FreeBSD FS , freebsd-geom@freebsd.org References: <391f2cc7-0036-06ec-b6c9-e56681114eeb@FreeBSD.org> <64f37301-a3d8-5ac4-a25f-4f6e4254ffe9@FreeBSD.org> <39E8D9C4-6BF3-4844-85AD-3568A6D16E64@samsco.org> From: Andriy Gapon Message-ID: Date: Tue, 12 Dec 2017 18:12:28 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 16:20:13 -0000 On 25/11/2017 12:54, Scott Long wrote: > Why is overloading EIO so bad? brelse() will call bdirty() when a BIO_WRITE > command has failed with EIO. Calling bdirty() has the effect of retrying the I/O. > This disregards the fact that disk drivers only return EIO when they’ve decided > that the I/O cannot be retried. It has no termination condition for the retries, and > will endlessly retry I/O in vain; I’ve seen this quite frequently. It also disregards > the fact that I/O marked as B_PAGING can’t be retried in this fashion, and will > trigger a panic. Because we pretend that EIO can be retried, we are left with > a system that is very fragile when I/O actually does fail. Instead of adding > more special cases and blurred lines, I want to go back to enforcing strict > contracts between the layers and force the core parts of the system to respect > those contracts and handle errors properly, instead of just retrying and > hoping for the best. I agree with your intention. But let's not project what I consider to be a bug in the buffer cache code on all consumers of bio / geom interface. In fact, I am much surprised that there is any code that treats EIO as retriable. I don't know of any other such code except for specialized disk recovery tools. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Tue Dec 12 16:26:41 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 116A7E9E078; Tue, 12 Dec 2017 16:26:41 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f53.google.com (mail-lf0-f53.google.com [209.85.215.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AAE21645D9; Tue, 12 Dec 2017 16:26:40 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f53.google.com with SMTP id 74so23874302lfs.0; Tue, 12 Dec 2017 08:26:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=D+fvP8SIW1g26aBGiBUCVq4Dn+6VuY2SQaiBWoALppU=; b=DuQrKGzFwTDtSCjfyUUTn8BHN2wTCJO9y2Z2j3MSoaRLQoCzXEjaAiz7AcZ884qXMi ut2EGqhjKFVtyOY4jFgZ8iCAKrgTjitkDxr9M9TFDiCrXqerj+iAqtlPHdVI5td8FzJe dapj0JMz852e76qq/MXJv6dULTh7Gs8yCgXHexK42mn/CXU6fO1BG2EVlb4YEMx2zbMv 3zJOnkppDiN2Q8HB3XvZ1X3zJMFHAqAI7+2bDEVxWX3mDrDiuY1m/zr+wJWpIzAM85Ce aF98bQYDE3naLL5oPWLbF5HPv9b3PPlDt63u0VOIQ6Kxoo7OJmVMlZkbJ00LsnCOwsTJ qljg== X-Gm-Message-State: AKGB3mIsd9NvZvJXFUX/3E2gzokdV6cBe3/9z/TVndXcapfj8SMI6KCI xnyaKZo+GrudeWqTUd6q3PAk4EmW X-Google-Smtp-Source: ACJfBouT0adJBAlON6E0hEBJ4IdIfRDYPAxZcYbXVvbgFowQSatuBtfr8MIGowbYlMXbfHjE2i69mA== X-Received: by 10.46.27.24 with SMTP id b24mr2313996ljb.54.1513095992314; Tue, 12 Dec 2017 08:26:32 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id q81sm3252603lfb.3.2017.12.12.08.26.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Dec 2017 08:26:31 -0800 (PST) Subject: Re: add BIO_NORETRY flag, implement support in ata_da, use in ZFS vdev_geom To: Warner Losh Cc: Scott Long , FreeBSD FS , freebsd-geom@freebsd.org References: <391f2cc7-0036-06ec-b6c9-e56681114eeb@FreeBSD.org> <64f37301-a3d8-5ac4-a25f-4f6e4254ffe9@FreeBSD.org> <39E8D9C4-6BF3-4844-85AD-3568A6D16E64@samsco.org> <33101e6c-0c74-34b7-ee92-f9c4a11685d5@FreeBSD.org> From: Andriy Gapon Message-ID: <8fde1a9e-ea32-af9e-de7f-30e7fe1738cd@FreeBSD.org> Date: Tue, 12 Dec 2017 18:26:30 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 16:26:41 -0000 On 25/11/2017 19:57, Warner Losh wrote: > Let's walk through this. You see that it takes a long time to fail an I/O. > Perfectly reasonable observation. There's two reasons for this. One is that the > disks take a while to make an attempt to get the data. The second is that the > system has a global policy that's biased towards 'recover the data' over 'fail > fast'. These can be fixed by reducing the timeouts, or lowing the read-retry > count for a given drive or globally as a policy decision made by the system > administrator. > > It may be perfectly reasonable to ask the lower layers to 'fail fast' and have > either a hard or a soft deadline on the I/O for a subset of I/O. A hard deadline > would return ETIMEDOUT or something when it's passed and cancel the I/O. This > gives better determinism in the system, but some systems can't cancel just 1 I/O > (like SATA drives), so we have to flush the whole queue. If we get a lot of > these, performance suffers. However, for some class of drives, you know that if > it doesn't succeed in 1s after you submit it to the drive, it's unlikely to > complete successfully and it's worth the performance hit on a drive that's > already acting up. > > You could have a soft timeout, which says 'don't do any additional action after > X time has elapsed and you get word about this I/O. This is similar to the hard > timeout, but just stops retrying after the deadline has passed. This scenario is > better on the other users of the drive, assuming that the read-recovery > operations aren't starving them. It's also easier to implement, but has worse > worst case performance characteristics. > > You aren't asking to limit retries. You're really asking to the I/O subsystem to > limit, where it can, the amount of time on an I/O so you can try another one. > You're means to doing this is to tell it not to retry. That's the wrong means. > It shouldn't be listed in the API that it's a 'NO RETRY' request. It should be a > QoS request flag: fail fast. I completely agree. 'NO RETRY' was a bad name and now I see it with painful clarity. Just to clarify, I agree not only on the name, but also on everything else you said above. > Part of why I'm being so difficult is that you don't understand this and are > proposing a horrible API. It should have a different name. I completely agree. > The other reason is > that I  absolutely do not want to overload EIO. You must return a different > error back up the stack. You've show no interest in this past, which is also a > needless argument. We've given good reasons, and you've poopooed them with bad > arguments. I still honestly don't understand this. I think that bio_error and bio_flags are sufficient to properly interpret the "fail-fast EIO". And I never intended for that error to be ever propagated by any means other than in bio_error. > Also, this isn't the data I asked for. I know things can fail slowly. I was > asking for how it would improve systems running like this. As in "I implemented > it, and was able to fail over to this other drive faster" or something like > that. Actual drive failure scenarios vary widely, and optimizing for this one > failure is unwise. It may be the right optimization, but it may not. There's > lots of tricky edges in this space. Well, I implemented my quick hack (as you absolutely correctly characterized it) in response to something that I observed happening in the past and that hasn't happen to me since then. But, realistically, I do not expect myself to be able to reproduce and test every tricky failure scenario. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Tue Dec 12 16:36:59 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2DD17E9E63F; Tue, 12 Dec 2017 16:36:59 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f43.google.com (mail-lf0-f43.google.com [209.85.215.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D333C64E29; Tue, 12 Dec 2017 16:36:58 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f43.google.com with SMTP id x204so23863680lfa.11; Tue, 12 Dec 2017 08:36:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=YRt6rkLRn6TLNgYxe3Sd2HrmEMbOa3nQmThHZmE1NhA=; b=JGP4BBO1/SthiEYwMJNjP/hyOpkGrshgsrW4YMWTVNpQoxAZ6jKsOngzcMsijuElbf YJRrU8IVU/ogC6c80oeWUt38EURtXEVUy5n2dfZQH5Osvcw4IyAu9KaBbBAv1zCw6YaT khUE0KG1EZKH/Vo4x3teR4IBYEmqAWkPHJeNqebo9M5s5xRPXwUYgbxYApszo62Ut9WB wIXQ5DB+UkevMRNGldXS/tlIOQG5ZqMxLvTvk3Em+w3xYexAgyHB+yLRdQi6yAT7doYV cuWmNMwtvF09KVjLfQ9mD/az3qwu3vh6lIrIkIYv048qwnIUTr8hhwxncke2mkmmXXZx jOEA== X-Gm-Message-State: AKGB3mIKfHuYt44OGt8vM0x0rn+4XAL8q44tSaj0QyLqAYv/CeItlLpp wWgsVTkepLyOSfldHiRH2Dc= X-Google-Smtp-Source: ACJfBotCEvH1N0NRbhohYdAxW4Kz1Jn40hOAP1fDk/naRm/qt9d5CTPR+UaCS6nsTStO/pVtcIaSVg== X-Received: by 10.46.83.9 with SMTP id h9mr2445871ljb.68.1513096611383; Tue, 12 Dec 2017 08:36:51 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id h11sm3226856lfd.54.2017.12.12.08.36.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Dec 2017 08:36:50 -0800 (PST) Subject: Re: add BIO_NORETRY flag, implement support in ata_da, use in ZFS vdev_geom To: Warner Losh Cc: FreeBSD FS , freebsd-geom@freebsd.org, Scott Long References: <391f2cc7-0036-06ec-b6c9-e56681114eeb@FreeBSD.org> <64f37301-a3d8-5ac4-a25f-4f6e4254ffe9@FreeBSD.org> <9f23f97d-3614-e4d2-62fe-99723c5e3879@FreeBSD.org> From: Andriy Gapon Message-ID: <38122ea9-ab20-f18a-90a2-04d2681e2ac9@FreeBSD.org> Date: Tue, 12 Dec 2017 18:36:49 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 16:36:59 -0000 On 26/11/2017 00:17, Warner Losh wrote: > > > On Sat, Nov 25, 2017 at 10:40 AM, Andriy Gapon > wrote: > > > Before anything else, I would like to say that I got an impression that we speak > from so different angles that we either don't understand each other's words or, > even worse, misinterpret them. > > > I understand what you are suggesting. Don't take my disagreement with your > proposal as willful misinterpretation. You are proposing something that's a > quick hack. Very true. > Maybe a useful one, but it's still problematical because it has the > upper layers telling the lower layers what to do (don't do your retry), rather > than what service to provide (I prefer a fast error exit to over every effort to > recover the data). Also true. > And it also does it by overloading the meaning of EIO, which > has real problems which you've not been open to listening, I assume due to your > narrow use case apparently blinding you to the bigger picture issues with that > route. Quite likely. > However, there's a way forward which I think that will solve these objections. > First, designate that I/O that fails due to short-circuiting the normal recovery > process, return ETIMEDOUT. The I/O stack currently doesn't use this at all (it > was introduced for the network side of things). This is a general catch-all for > an I/O that we complete before the lower layers have given it the maximum amount > of effort to recover the data, at the user request. Next, don't use a flag. > Instead add a 32-bit field that is call bio_qos for quality of service hints and > another 32-bit field for bio_qos_param. This allows us to pass down specific > quality of service desires from the filesystem to the lower layers. The > parameter will be unused in your proposal. BIO_QOS_FAIL_EARLY may be a good name > for a value to set it to (at the moment, just use 1). We'll assign the other QOS > values later for other things. It would allow us to implement the other sorts of > QoS things I talked about as well. That's a very interesting and workable suggestion. I will try to work on it. > As for B_FAILFAST, it's quite unlike what you're proposing, except in one > incidental detail. It's a complicated state machine that the sd driver in > solaris implemented. It's an entire protocol. When the device gets errors, it > goes into this failfast state machine. The state machine makes a determination > that the errors are indicators the device is GONE, at least for the moment, and > it will fail I/Os in various ways from there. Any new I/Os that are submitted > will be failed (there's conditional behavior here: depending on a global setting > it's either all I/O or just B_FAILFAST I/O). Yeah, I realized that B_FAILFAST was quite different from the first impression that I got from its name. Thank you for doing and sharing your analysis of how it actually works. > ZFS appears to set this bit for its > discovery code only, when a device not being there would significantly delay > things. I think that ZFS sets the bit for all 'first-attempt' I/O. It's the various retries / recovery where this bit is not set. > Anyway, when the device returns (basically an I/O gets through or maybe > some other event happens), the driver exists this mode and returns to normal > operation. It appears to be designed not for the use case that you described, > but rather for a drive that's failing all over the place so that any pending > I/Os get out of the way quickly. Your use case is only superficially similar to > that use case, so the Solaris / Illumos experiences are mildly interesting, but > due to the differences not a strong argument for doing this. This facility in > Illumos is interesting, but would require significantly more retooling of the > lower I/O layers in FreeBSD to implement fully. Plus Illumos (or maybe just > Solaris) a daemon that looks at failures to manage them at a higher level, which > might make for a better user experience for FreeBSD, so that's something that > needs to be weighed as well. Okay. > We've known for some time that HDD retry algorithms take a long time. Same is > true of some SSD or NVMe algorithms, but not all. The other objection I have to > 'noretry' naming  is that it bakes the current observed HDD behavior and > recovery into the API. This is undesirable as other storage technologies have > retry mechanisms that happen quite quickly (and sometimes in the drive itself). > The cutoff between fast and slow recovery is device specific, as are the methods > used. For example, there's new proposals out in NVMe (and maybe T10/T13 land) to > have new types of READ commands that specify the quality of service you expect, > including providing some sort of deadline hint to clip how much effort is > expended in trying to recover the data. It would be nice to design a mechanism > that allows us to start using these commands when drives are available with > them, and possibly using timeouts to allow for a faster abort. Most of your HDD > I/O will complete within maybe ~150ms, with a long tail out to maybe as long as > ~400ms. It might be desirable to set a policy that says 'don't let any I/Os > remain in the device longer than a second' and use this mechanism to enforce > that. Or don't let any I/Os last more than 20x the most recent median I/O time. > A single bit is insufficiently expressive to allow these sorts of things, which > is another reason for my objection to your proposal. With the QOS fields being > independent, the clone routines just copies them and makes no judgement value on > them. I now agree with this. Thank you for the detailed explanation. > So, those are my problems with your proposal, and also some hopefully useful > ways to move forward. I've chatted with others for years about introducing QoS > things into the I/O stack, so I know most of the above won't be too contentious > (though ETIMEDOUT I haven't socialized, so that may be an area of concern for > people). Thank you! -- Andriy Gapon From owner-freebsd-fs@freebsd.org Tue Dec 12 19:05:11 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CD65E80531 for ; Tue, 12 Dec 2017 19:05:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7500E6C52F for ; Tue, 12 Dec 2017 19:05:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCJ5AZT009481 for ; Tue, 12 Dec 2017 19:05:11 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 104406] [ufs] Processes get stuck in "ufs" state under persistent CPU load Date: Tue, 12 Dec 2017 19:05:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 7.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: dependson Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 19:05:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D104406 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |224292 Referenced Bugs: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 [Bug 224292] processes are hanging in state ufs / possible deadlock in file system --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 19:10:35 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 553DBE807B2 for ; Tue, 12 Dec 2017 19:10:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 40FF96C853 for ; Tue, 12 Dec 2017 19:10:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCJAZGX052698 for ; Tue, 12 Dec 2017 19:10:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 223491] fsck_ufs: Directory XXXX name not found Date: Tue, 12 Dec 2017 19:10:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 19:10:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223491 --- Comment #3 from Wolfram Schneider --- Correction: the VM is running on Virtio (kvm/linux) https://www.linux-kvm.org/page/Virtio --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 19:11:35 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58335E8082C for ; Tue, 12 Dec 2017 19:11:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46C246CA92 for ; Tue, 12 Dec 2017 19:11:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCJBZFB068362 for ; Tue, 12 Dec 2017 19:11:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 223491] fsck_ufs: Directory XXXX name not found Date: Tue, 12 Dec 2017 19:11:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: dependson Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 19:11:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223491 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |224292 Referenced Bugs: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 [Bug 224292] processes are hanging in state ufs / possible deadlock in file system --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 19:13:18 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B17AEE80A4B for ; Tue, 12 Dec 2017 19:13:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9F3906CC55 for ; Tue, 12 Dec 2017 19:13:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCJDIH1086541 for ; Tue, 12 Dec 2017 19:13:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Tue, 12 Dec 2017 19:13:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 19:13:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 19:30:46 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EECFE8177F for ; Tue, 12 Dec 2017 19:30:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F023C6E67D for ; Tue, 12 Dec 2017 19:30:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCJUjUU060251 for ; Tue, 12 Dec 2017 19:30:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Tue, 12 Dec 2017 19:30:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 19:30:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mckusick@FreeBSD.org --- Comment #2 from Kirk McKusick --- It is possible that this patch will fix this problem: r326657 | kib | 2017-12-07 01:05:34 -0800 (Thu, 07 Dec 2017) | 15 lines Fix livelock in ufsdirhash_create(). When more than one thread enters ufsdirhash_create() for the same directory and the inode dirhash is instantiated, but the dirhash' hash is not, all of them lock the dirhash shared and then try to upgrade. Since there are several threads owning the lock shared, upgrade fails and the same attempt is repeated, ad infinitum. To break the lockstep, lock the dirhash in exclusive mode after the failed try-upgrade. Reported and tested by: pho Sponsored by: Mellanox Technologies MFC after: 1 week It is also known that dirhash tuning as described by Wolfram Schneider above significantly mitigates this problem. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 20:56:24 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA555E84B87 for ; Tue, 12 Dec 2017 20:56:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C730C72AA0 for ; Tue, 12 Dec 2017 20:56:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCKuOOQ088578 for ; Tue, 12 Dec 2017 20:56:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Tue, 12 Dec 2017 20:56:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 20:56:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress --- Comment #3 from Wolfram Schneider --- Hi Kirk, thanks for the fast response. I can run the tests again and see if the mach= ine survives the next 3 days. I'm wondering if we have a simple test program to trigger this bug?=20 Any suggestion to tune dirhash, to get the the broken lock faster? my current setup: sysctl -a|egrep vfs.ufs.dirhash vfs.ufs.dirhash_reclaimpercent: 10 vfs.ufs.dirhash_lowmemcount: 188 vfs.ufs.dirhash_docheck: 0 vfs.ufs.dirhash_mem: 510163 vfs.ufs.dirhash_maxmem: 2097152 vfs.ufs.dirhash_minsize: 2560 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 21:26:43 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E0995E854C2 for ; Tue, 12 Dec 2017 21:26:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CED00737DB for ; Tue, 12 Dec 2017 21:26:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCLQhYq023710 for ; Tue, 12 Dec 2017 21:26:43 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Tue, 12 Dec 2017 21:26:43 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 21:26:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 --- Comment #4 from Kirk McKusick --- (In reply to Wolfram Schneider from comment #3) Unfortunately I do not know the correlation of tuning to triggering the livelock bug. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 21:37:56 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95E49E85A95 for ; Tue, 12 Dec 2017 21:37:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 83460740CE for ; Tue, 12 Dec 2017 21:37:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCLbux8040739 for ; Tue, 12 Dec 2017 21:37:56 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Tue, 12 Dec 2017 21:37:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: pho@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 21:37:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 --- Comment #5 from Peter Holm --- (In reply to Kirk McKusick from comment #2) I have booted a test box with amd64, 1 CPU and 1GB of RAM to see if I can reproduce the problem. The issue fixed by r326657 looked like this: https://people.freebsd.org/~pho/stress/log/vfs_cache.txt --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Dec 12 23:34:24 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6BADE892A8 for ; Tue, 12 Dec 2017 23:34:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D487278F16 for ; Tue, 12 Dec 2017 23:34:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBCNYOtN024937 for ; Tue, 12 Dec 2017 23:34:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224272] zfs replace doesn't detach old device Date: Tue, 12 Dec 2017 23:34:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eborisch+FreeBSD@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 23:34:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224272 eborisch+FreeBSD@gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |eborisch+FreeBSD@gmail.com --- Comment #1 from eborisch+FreeBSD@gmail.com --- Perhaps because it is already marked as a spare (part of the pool) and the = ZFS later hasn=E2=80=99t had any errors yet? zpool remove the spare and then re= -issue the replace? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Dec 13 00:31:22 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1DE7E8AF43 for ; Wed, 13 Dec 2017 00:31:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9E4E67A7B6 for ; Wed, 13 Dec 2017 00:31:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBD0VMGX014128 for ; Wed, 13 Dec 2017 00:31:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Wed, 13 Dec 2017 00:31:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jeff@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 00:31:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #31 from Jeff Roberson --- Created attachment 188780 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D188780&action= =3Dedit Proposed fix Please test this fix for arclowmem et all --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Dec 13 02:50:46 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28872E8E687 for ; Wed, 13 Dec 2017 02:50:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B4137EEF2 for ; Wed, 13 Dec 2017 02:50:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBD2oiYf092718 for ; Wed, 13 Dec 2017 02:50:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Wed, 13 Dec 2017 02:50:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 02:50:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #32 from Cy Schubert --- Currently running on four amd64 machines. Initial tests indicate this PR is fixed. Thanks Jeff. I can test on an i386 machine this weekend. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Dec 13 03:06:26 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0513CE8EC97 for ; Wed, 13 Dec 2017 03:06:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB5877FA3D for ; Wed, 13 Dec 2017 03:06:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBD36P1Q042185 for ; Wed, 13 Dec 2017 03:06:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Wed, 13 Dec 2017 03:06:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 03:06:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #33 from Cy Schubert --- On i386 this probably needs a cast.=20 /opt/src/svn-current/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:6= 115:2 1: error: ordered comparison between pointer and integer ('uint64_t' (aka 'unsig ned long long') and 'long (*)(void)') [-Werror] available_memory =3D MIN(available_memory, uma_avail); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /opt/src/svn-current/sys/sys/param.h:305:23: note: expanded from macro 'MIN' #define MIN(a,b) (((a)<(b))?(a):(b)) ~~~^~~~ --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Dec 13 03:40:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71DCBE8F862 for ; Wed, 13 Dec 2017 03:40:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5F26580963 for ; Wed, 13 Dec 2017 03:40:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBD3e4l0081984 for ; Wed, 13 Dec 2017 03:40:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Wed, 13 Dec 2017 03:40:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 03:40:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #34 from Cy Schubert --- arc.c line 6115 s/b, available_memory =3D MIN(available_memory, uma_avail()); ^^ That'll fix i386 build. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Dec 13 07:41:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 763EAE94329 for ; Wed, 13 Dec 2017 07:41:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5AB7666288 for ; Wed, 13 Dec 2017 07:41:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBD7f5lu064226 for ; Wed, 13 Dec 2017 07:41:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224272] zfs replace doesn't detach old device Date: Wed, 13 Dec 2017 07:41:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: matthias.pfaller@familie-pfaller.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 07:41:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224272 --- Comment #2 from Matthias Pfaller = --- I already did what you suggested and the system did what I wanted to achiev= e.=20 Its still surprising to watch it resilver for 14h and then ending up in the same state as before... I think when you request a replace (even when replacing with an hot spare) = it should just detach the replaced drive when finished. So maybe that's not a bug report but a feature request. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Dec 13 08:33:14 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F503E95B25 for ; Wed, 13 Dec 2017 08:33:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D4F267950 for ; Wed, 13 Dec 2017 08:33:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBD8XD5p093218 for ; Wed, 13 Dec 2017 08:33:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Wed, 13 Dec 2017 08:33:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: nbe@renzel.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 08:33:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #35 from Nils Beyer --- (In reply to Jeff Roberson from comment #31) thank you. Confirmed - with this patch, "arc_reclaim_thread" stays quiet and ARC usage raises accordingly. Disk usage has become normal and CPU power goes into us= er processes. "arclowmem" tests anyone? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Dec 13 21:43:39 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C29F4E88704 for ; Wed, 13 Dec 2017 21:43:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0B6765A5C for ; Wed, 13 Dec 2017 21:43:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBDLhcoR031426 for ; Wed, 13 Dec 2017 21:43:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Wed, 13 Dec 2017 21:43:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jeff@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 21:43:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #36 from Jeff Roberson --- I can comfirm that uma_avail) should've been uma_avail()) on all architectu= res. pho has reported that this patch gives him identical performance to code pre-dating the change. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 04:49:42 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B9D9E93D91 for ; Thu, 14 Dec 2017 04:49:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D5147269A for ; Thu, 14 Dec 2017 04:49:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBE4neZe092875 for ; Thu, 14 Dec 2017 04:49:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Thu, 14 Dec 2017 04:49:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: markmi@dsl-only.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 04:49:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 Mark Millard changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |markmi@dsl-only.net --- Comment #37 from Mark Millard --- There is also the rpi2 and UFS tied list submittals against -r326347 causing booting to hang, starting with: https://lists.freebsd.org/pipermail/freebsd-hackers/2017-December/051939.ht= ml and ending with: https://lists.freebsd.org/pipermail/freebsd-hackers/2017-December/051947.ht= ml There are tied to boot hangs with the kernel thread (100001) involving: . . . uma_zcreate() at uma_zcreate+0x10c pc =3D 0xc052b024 lr =3D 0xc050cc90 (ffs_mount+0x80) sp =3D 0xd42a5858 fp =3D 0xd42a5980 r4 =3D 0xd7844000 r5 =3D 0x00000000 r6 =3D 0x00000003 r7 =3D 0xc09a99c0 r8 =3D 0x00000000 r9 =3D 0xd429f000 r10 =3D 0xd828c400 ffs_mount() at ffs_mount+0x80 pc =3D 0xc050cc90 lr =3D 0xc032e1d4 (vfs_donmount+0xeec) sp =3D 0xd42a5988 fp =3D 0xd42a5b38 r4 =3D 0xffffffff r5 =3D 0xd74ec120 r6 =3D 0xd429f000 r7 =3D 0x00000000 r8 =3D 0x00000000 r9 =3D 0xc425acd0 r10 =3D 0xd828c400 . . . and the uma thread involving: . . . uma_reclaim_locked() at uma_reclaim_locked+0x200 pc =3D 0xc052d9f8 lr =3D 0xc052de70 (uma_reclaim_worker+0x4c) sp =3D 0xd87dadf0 fp =3D 0xd87dae20 r4 =3D 0xc09a9c68 r5 =3D 0xc4242d80 r6 =3D 0xc06df86e r7 =3D 0x00000000 r8 =3D 0x00000100 r9 =3D 0xc09b3d08 r10 =3D 0xc09a9c7c uma_reclaim_worker() at uma_reclaim_worker+0x4c pc =3D 0xc052de70 lr =3D 0xc0230f10 (fork_exit+0xa0) sp =3D 0xd87dae28 fp =3D 0xd87dae40 r4 =3D 0xd7ad8740 r5 =3D 0xd70ce000 r6 =3D 0xc052de24 r7 =3D 0x00000000 r8 =3D 0xd87dae48 r9 =3D 0xd7ada3a0 r10 =3D 0xc0824e8c . . . (extracted from the last of those messages). bisect activity was used to isolate -r326347 as the version the problem started in. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 04:59:38 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FD65E9483D for ; Thu, 14 Dec 2017 04:59:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8974172D18 for ; Thu, 14 Dec 2017 04:59:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBE4xbAI019674 for ; Thu, 14 Dec 2017 04:59:38 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Thu, 14 Dec 2017 04:59:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cy@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 04:59:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #38 from Cy Schubert --- (In reply to Mark Millard from comment #37) It probably makes sense to track this as a separate incident. It's easier to find this through keywords. What do you think? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 05:14:01 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D20D4E952B5 for ; Thu, 14 Dec 2017 05:14:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BEE3D73690 for ; Thu, 14 Dec 2017 05:14:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBE5E0bV064449 for ; Thu, 14 Dec 2017 05:14:01 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Thu, 14 Dec 2017 05:14:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: markmi@dsl-only.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 05:14:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #39 from Mark Millard --- (In reply to Cy Schubert from comment #38) I can extract from the prior list messages and make a submittal if desired. There is also a complaint for pine64 from Emmanuel Vadot: https://lists.freebsd.org/pipermail/svn-src-head/2017-December/106954.html reporting that as of -r326347 : The board just hang when plugin usb device That would seem to match my rpi2 reports: I have a USB SSD on a powered hub for the root file system on the rpi2 that I was using. (Recently the mmcsd0 mechanism to hold in the sdcard is broken but the eject mechanism still works. I hold in the sdcard during the early boot. But I've used a USB SSD for such small boards for a long time.) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 05:46:14 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBB2EE96575 for ; Thu, 14 Dec 2017 05:46:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C8EB2748A4 for ; Thu, 14 Dec 2017 05:46:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBE5kDVV038163 for ; Thu, 14 Dec 2017 05:46:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Thu, 14 Dec 2017 05:46:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: markmi@dsl-only.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 05:46:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #40 from Mark Millard --- (In reply to Cy Schubert from comment #38) I've submitted bugzilla 224330 for Emmanuel Vadot and my list messages tied to USB on pine64 (Emmanual) and rpi2 (me). Mostly I referenced the list messages instead of copying over the information. I do not have access to the Keywords field there. Nor to See Also here in bugzilla 224080. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 06:08:39 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C80D7E97496 for ; Thu, 14 Dec 2017 06:08:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B34277571E for ; Thu, 14 Dec 2017 06:08:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBE68dWv002045 for ; Thu, 14 Dec 2017 06:08:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Thu, 14 Dec 2017 06:08:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: markmi@dsl-only.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 06:08:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #41 from Mark Millard --- Another -r326347 and zfs combination list report is part of the material in the pair: https://lists.freebsd.org/pipermail/freebsd-current/2017-November/067613.ht= ml https://lists.freebsd.org/pipermail/freebsd-current/2017-November/067618.ht= ml These folks might be able to test for if their problems have been fixed when things are ready for such. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 09:41:01 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BE2CE9E30F; Thu, 14 Dec 2017 09:41:01 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f51.google.com (mail-lf0-f51.google.com [209.85.215.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9D937CC46; Thu, 14 Dec 2017 09:41:00 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f51.google.com with SMTP id f20so5882672lfe.3; Thu, 14 Dec 2017 01:41:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=nNTnuQswS1AYZc2U/wyOpdL6RvGXoegKvnU8CBYS+MI=; b=Hgjxq/feua8HLMHSadq6aN9sHLgNhaCUtBYGZIYsqk2xrTCbVBGB5ngR2kfdFowKMo SSshsj3Kab2nNU4j/ogv1dLNpFeYdPdMX/KW1tQOXFRLtoWNnRagJMjyhx7KSaif959c rPhV7pQ/dL+Qd/l9KasWZeMnttJUZVD26KXzFhoM4KkO7gQdeEUKZ7i60cSBsK4V49VV QzYPxsFXURjqsyi8jwQJlptpQ+1GLLCrJbCJfOOo/Mj7HqaNSONha6PPFmnA5FfpYrbO hqRiyzewEEgTMRsYVIgZuzqgzvFC5mLNRnwHI8JNEYhG1xHqCBxAW5+hxW06NtaEaVgc ZlsA== X-Gm-Message-State: AKGB3mLSOrrBMkT0zcQMtHGn7xEhupWTDzFd3derrgmb9L8ryhJZGcmq a/P8P8gU0wWij1kQwYw+6N4UOUeM X-Google-Smtp-Source: ACJfBosUjL00d1OG34TV0tBHXHmAxQMGU7GzUz2h1q7g0H3NhwA2SOI0hvH2bzAYqRNQgtiSaUIJvQ== X-Received: by 10.46.55.20 with SMTP id e20mr3482902lja.118.1513244452499; Thu, 14 Dec 2017 01:40:52 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id h142sm711545lfh.37.2017.12.14.01.40.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Dec 2017 01:40:51 -0800 (PST) From: Andriy Gapon Subject: Re: add BIO_NORETRY flag, implement support in ata_da, use in ZFS vdev_geom To: Scott Long Cc: Warner Losh , FreeBSD FS , freebsd-geom@freebsd.org References: <391f2cc7-0036-06ec-b6c9-e56681114eeb@FreeBSD.org> <64f37301-a3d8-5ac4-a25f-4f6e4254ffe9@FreeBSD.org> <39E8D9C4-6BF3-4844-85AD-3568A6D16E64@samsco.org> <33101e6c-0c74-34b7-ee92-f9c4a11685d5@FreeBSD.org> <783CA790-C0D3-442D-A5A2-4CB424FCBD62@samsco.org> Message-ID: <59654c50-b24e-ac87-2154-681cc57627de@FreeBSD.org> Date: Thu, 14 Dec 2017 11:40:49 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <783CA790-C0D3-442D-A5A2-4CB424FCBD62@samsco.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 09:41:01 -0000 This email is rather large, with a lot of contexts. Replying just to a single piece here. On 27/11/2017 17:29, Scott Long wrote: > > >> On Nov 25, 2017, at 10:36 AM, Andriy Gapon wrote: >> Let's assume that I am talking about the case of not being able to read an HDD >> sector that is gone bad. >> Here is a real world example: >> >> Jun 16 10:40:18 trant kernel: ahcich0: NCQ error, slot = 20, port = -1 >> Jun 16 10:40:18 trant kernel: (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 >> 00 00 58 62 40 2c 00 00 08 00 00 >> Jun 16 10:40:18 trant kernel: (ada0:ahcich0:0:0:0): CAM status: ATA Status Error >> Jun 16 10:40:18 trant kernel: (ada0:ahcich0:0:0:0): ATA status: 41 (DRDY ERR), >> error: 40 (UNC ) >> Jun 16 10:40:18 trant kernel: (ada0:ahcich0:0:0:0): RES: 41 40 68 58 62 40 2c 00 >> 00 00 00 >> Jun 16 10:40:18 trant kernel: (ada0:ahcich0:0:0:0): Retrying command >> Jun 16 10:40:20 trant kernel: ahcich0: NCQ error, slot = 22, port = -1 > ... >> I do not see anything wrong in what CAM / ahci /ata_da did here. >> They did what I would expect them to do. They tried very hard to get data that >> I told them I need. > > Two things I see here. The first is that the drive is trying for 2 seconds to get good > data off of the media. The second is that it’s failing and reporting the error as > uncorrectable. I think that retries at the OS/driver > layer are therefore useless; the drive is already doing a bunch of its own retries and > is failing, and is telling you that it’s failing. In the past, the conventional wisdom has > been to do retries, because 30 years ago drives had minimal firmware and didn’t do > a good job at managing data integrity. Now they do an extensive amount of > analysis-driven error correction and retries, so I think it’s time to change the > conventional wisdom. I’d propose that for direct-attach SSDs and HDDs we treat this > error as non-retriable. Normally this would be a one-line change, but I think it needs > to be nuanced to distinguish between optical drives, simple flash media drives, and > regular HDDs and SSDs. > > An interim solution would be to just set the kern.cam.ada.retry_count to 0. I went through some ada errors that I have in my logs and I think that there can be a difference between HDDs and SSDs too. I thought that the HDD internal retry mechanism would be thorough enough, but, to my surprise, I see that majority of read failures are recovered by the first retry. Sometimes it's the second retry that's successful, in all other cases the standard four retries do not help. So, it may be too early to set ada.retry_count to 0 for all types of supported disks. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Thu Dec 14 19:19:34 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F728E8C1F3 for ; Thu, 14 Dec 2017 19:19:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8D53774007 for ; Thu, 14 Dec 2017 19:19:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBEJJXCF071673 for ; Thu, 14 Dec 2017 19:19:34 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224347] Boot erroneously mounts severely damaged ufs file system Date: Thu, 14 Dec 2017 19:19:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cem@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 19:19:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224347 Conrad Meyer changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cem@freebsd.org Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 21:32:56 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2684E90870 for ; Thu, 14 Dec 2017 21:32:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A08857AA10 for ; Thu, 14 Dec 2017 21:32:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBELWta1090050 for ; Thu, 14 Dec 2017 21:32:56 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Thu, 14 Dec 2017 21:32:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jeff@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 21:32:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #42 from Jeff Roberson --- The first report says r326347 was the last known good kernel. There may be other unrelated bugs. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 22:08:47 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79A77E916C9 for ; Thu, 14 Dec 2017 22:08:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 678917BB90 for ; Thu, 14 Dec 2017 22:08:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBEM8ldu077233 for ; Thu, 14 Dec 2017 22:08:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Thu, 14 Dec 2017 22:08:47 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: markmi@dsl-only.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 22:08:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #43 from Mark Millard --- (In reply to Jeff Roberson from comment #42) You may well be correct about unrelated bugs. But. . . Looks like I Parse the 067613.html report differently than you do: A) First it says: Most recent CURRENT, as with r326363, crashes on booting the kernel This is different than it later reports about -r326347 as I parse the statements. B) Then it says=20 The last known half-ways good kernel is 12.0-CURRENT #157 r326347 Note the "half-ways" part. It then continues with what I interpreted as a description of "half-ways" in a booted -r326347 context (so, not for -r326363). When performing buildworld/buildkernel/relaese/package or poudriere on a ZFS filesystem, I can now bring down the system almost in a reproducible way. I could be wrong but I do not see that as a report of the "the last known good kernel". --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Dec 14 22:17:13 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E86AE91B95 for ; Thu, 14 Dec 2017 22:17:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0C6947C3E4 for ; Thu, 14 Dec 2017 22:17:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBEMHCFu000955 for ; Thu, 14 Dec 2017 22:17:12 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224080] ZFS: "arc_dnlc_evicts_thr" + "arc_reclaim_thread" - high CPU load during poudriere build... Date: Thu, 14 Dec 2017 22:17:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jeff@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 22:17:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224080 --- Comment #44 from Jeff Roberson --- I have sent the patch and a short description to the person who made the report. I agree I may have misparsed. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Dec 15 15:08:50 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 361EEE83A52 for ; Fri, 15 Dec 2017 15:08:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2415078EA8 for ; Fri, 15 Dec 2017 15:08:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBFF8nGU018779 for ; Fri, 15 Dec 2017 15:08:50 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Fri, 15 Dec 2017 15:08:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2017 15:08:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 --- Comment #6 from Wolfram Schneider --- The machine crashed again, with a kernel panic. Unfortunately, the kernel d= ump was not configured. I can access the kernel debug with a VNC web console, a= nd take some screenshots. You can see the screenshots here: https://people.freebsd.org/~wosch/tmp/pr-224292/kernel-panic-deadlock-1/ actually, on https://people.freebsd.org/~wosch/tmp/pr-224292/kernel-panic-deadlock-1/Scr= een%20Shot%202017-12-15%20at%2011.57.55.png you can see on the top procstat, sh and perl5.26.1 are in state "ufs" and here is the perl process pid 28098 with sys_openat() https://people.freebsd.org/~wosch/tmp/pr-224292/kernel-panic-deadlock-1/Scr= een%20Shot%202017-12-15%20at%2011.59.47.png --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Dec 15 15:09:23 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8D50E83AC1 for ; Fri, 15 Dec 2017 15:09:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9740678F4C for ; Fri, 15 Dec 2017 15:09:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBFF9NVo025506 for ; Fri, 15 Dec 2017 15:09:23 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Fri, 15 Dec 2017 15:09:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2017 15:09:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 --- Comment #7 from Wolfram Schneider --- (In reply to Wolfram Schneider from comment #6) I forgot to mention the commands I run on the kernel debugger: show allpcpu show alllocks show lockedvnods show mount show page ps allt --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Dec 15 19:34:20 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 370D6E8A8D2 for ; Fri, 15 Dec 2017 19:34:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2532B34F0 for ; Fri, 15 Dec 2017 19:34:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBFJYJmD027364 for ; Fri, 15 Dec 2017 19:34:20 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224353] NFS mounts should count as local in a diskless system. Date: Fri, 15 Dec 2017 19:34:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2017 19:34:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224353 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rmacklem@FreeBSD.org Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Dec 15 19:47:15 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8209EE8AF2E for ; Fri, 15 Dec 2017 19:47:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D78D3F02 for ; Fri, 15 Dec 2017 19:47:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBFJlF3b056371 for ; Fri, 15 Dec 2017 19:47:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224353] NFS mounts should count as local in a diskless system. Date: Fri, 15 Dec 2017 19:47:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: amvandemore@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2017 19:47:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224353 amvandemore@gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |amvandemore@gmail.com --- Comment #1 from amvandemore@gmail.com --- I understand the current infrastructure may be lacking for the this use cas= e, but surely there is some better way to address than a glaring POLA violatio= n. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Dec 15 19:56:18 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D7B8E8B147 for ; Fri, 15 Dec 2017 19:56:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EFEEB63578 for ; Fri, 15 Dec 2017 19:56:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBFJuHdf078781 for ; Fri, 15 Dec 2017 19:56:17 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224353] NFS mounts should count as local in a diskless system. Date: Fri, 15 Dec 2017 19:56:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: parakleta@darkreality.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2017 19:56:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224353 --- Comment #2 from Duane --- (In reply to amvandemore from comment #1) I don't know what POLA means, could you please explain what you mean by tha= t? The other solution I considered was adding mount behaviour to the `rc.initdiskless` script through some modification of the `conf` mechanism = but this seemed more complicated and likely to lead to duplication of effort. I was also concerned that thr more "special" the behaviour is the more likely= it is to "rot" through disuse as time goes on. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Dec 15 23:44:54 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B85D4E8F95A for ; Fri, 15 Dec 2017 23:44:54 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-135.reflexion.net [208.70.210.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 724376A352 for ; Fri, 15 Dec 2017 23:44:53 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 3556 invoked from network); 15 Dec 2017 23:44:47 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 15 Dec 2017 23:44:47 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Fri, 15 Dec 2017 18:44:47 -0500 (EST) Received: (qmail 27857 invoked from network); 15 Dec 2017 23:44:47 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 15 Dec 2017 23:44:47 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Fri, 15 Dec 2017 15:44:46 -0800 From: Shiva Bhanujan To: Youzhong Yang , Andriy Gapon CC: "freebsd-fs@freebsd.org" , Shiva Bhanujan Subject: RE: zio_done panic in 10.3 Thread-Topic: zio_done panic in 10.3 Thread-Index: AdNiPrZNqbG0j29/RtqtjwQQWT08KgAsk0+A//+0h/eAAMUCgP//oow5gACblACAASwOAIAAC+cAgAARrQCACkl8K4AZ0DTe Date: Fri, 15 Dec 2017 23:44:45 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3683E11@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C367D3D1@QLEXC01.Quorum.local> <5021a016-9193-b626-78cf-54ffa3929e22@FreeBSD.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C367D562@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C367D636@QLEXC01.Quorum.local> <41e2465d-e1b5-33ce-57b5-49bea6087d9a@FreeBSD.org> <78d712d9-dda3-0411-262e-bb64f9ab46eb@FreeBSD.org>, , <3A5A10BE32AC9E45B4A22F89FC90EC0701C3680A9C@QLEXC01.Quorum.local> In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3680A9C@QLEXC01.Quorum.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [61.3.162.42] Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2017 23:44:54 -0000 I've updated both of the bug reports. I was hoping that setting = secondacache=3Dmetadata on the destination ZFS where the snapshots are = being received would restrict performance to only the receive side. that = isn't the case, and the crash has started again. I was really hoping that = there could be a solution to this. From: owner-freebsd-fs=40freebsd.org =5Bowner-freebsd-fs=40freebsd.org=5D = on behalf of Shiva Bhanujan =5Bshiva.bhanujan=40quorum.net=5D Sent: Wednesday, November 29, 2017 5:32 AM To: Youzhong Yang; Andriy Gapon Cc: freebsd-fs=40freebsd.org Subject: RE: zio_done panic in 10.3 Hi Andriy, Could you please let me know when could a fix for this be available? Regards, Shiva From: Youzhong Yang =5Byouzhong=40gmail.com=5D Sent: Wednesday, November 22, 2017 8:26 AM To: Andriy Gapon Cc: Shiva Bhanujan;=20 cem=40freebsd.org;=20 freebsd-fs=40freebsd.org Subject: Re: zio_done panic in 10.3 Thanks Andriy. Two bug reports filed: https://www.illumos.org/issues/8857 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223803 On Wed, Nov 22, 2017 at 10:22 AM, Andriy Gapon=20 wrote: On 22/11/2017 16:40, Youzhong Yang wrote: > Hi Andriy, > > This is nice=21 I am 100% sure it's exactly the same issue I experienced = and then > reported to illumos mailing list. In all the crash dumps zio->io_done =3D > l2arc_read_done, so I thought the crash must be related to L2ARC. Once I = set > secondarycache=3Dmetadata, the frequency of crash went from one per 2 = days down to > one per week. I've been puzzled by what could have caused a zio being = destroyed > while there's still child zio. Your explanation definitely makes sense=21 Oh, I now recall seeing your report: https://illumos.topicbox.com/groups/zfs/Tccd8b4463865899e I remember that it raised my interest, but then I forgot about it and didn't correlate it with the latest reports. > By the way, is there a FreeBSD bug report or an illumos bug number = tracking this > issue? I would be more than happy to create one if needed, and also test = your > potential fix here in our environment. I am not aware of any existing bug report. It would be great if you could open one =5B or two :-) =5D If you open an illumos issue, please also add George Wilson as a watcher. I think that George is also interested in fixing this issue and he knows the relevant code better than me. Thank you=21 > On Tue, Nov 21, 2017 at 3:46 PM, Andriy Gapon > wrote: > > > > > On 21/11/2017 21:30, Shiva Bhanujan wrote: > > it did get compressed to 0.5G - still too big to send via email. I did = send some more debug information by running kgdb on the core file to = Andriy, and I'm waiting for any analysis that he might provide. > > Yes, kgdb-over-email turned out to be a far more efficient compression :-) > I already have an analysis based on the information provided by Shiva = and by > another user who has the same problem and contacted me privately. > I am discussing possible ways to fix the problem with George Wilson who = was very > kind to double-check the analysis, complete it and suggest possible fixes. > > A short version is that dbuf_prefetch and dbuf_prefetch_indirect_done = functions > chain new zio-s under the same parent zio (a completion of one child zio = may > create another child zio). They do it using arc_read which can create = either a > logical zio in most cases or a vdev zio for a read from a cache device = (2arc). > zio_done() has a check for the completion of a parent zio's children but = that > check is not completely safe and can be broken by the pattern that = dbuf_prefetch > can create. So, under some specific circumstances the parent zio may = complete > and get destroyed while there is a child zio. > > I believe this problem to be rather rare, but there could be = configurations and > workloads where it's triggered more often. > The problem does not happen if there are no cache devices. > > > From: Conrad Meyer =5Bcem=40freebsd.org =5D > > > > Sent: Tuesday, November 21, 2017 9:04 AM > > > > To: Shiva Bhanujan > > > > Cc: Andriy Gapon;=20 freebsd-fs=40freebsd.org > > > > Subject: Re: zio_done panic in 10.3 > > > > > > > > > > > > > > > > Have you tried compressing it with e.g. xz or zstd? > > > > -- > Andriy Gapon > _______________________________________________ >=20 freebsd-fs=40freebsd.org mailing list >=20 https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to =22freebsd-fs-unsubscribe=40freebsd.org > =22 > > -- Andriy Gapon _______________________________________________ freebsd-fs=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to =22freebsd-fs-unsubscribe=40freebsd.org=22 From owner-freebsd-fs@freebsd.org Sat Dec 16 09:54:52 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14D70EA2CE0 for ; Sat, 16 Dec 2017 09:54:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 02EFC7EEBC for ; Sat, 16 Dec 2017 09:54:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBG9sp6B011726 for ; Sat, 16 Dec 2017 09:54:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Sat, 16 Dec 2017 09:54:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Dec 2017 09:54:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 --- Comment #8 from Wolfram Schneider --- (In reply to Wolfram Schneider from comment #6) As fare we can see from the test logs, it crashed in the same perl module as before. I'm sorry to say, but r326657 didn't solved the issue. The witch is= not dead. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Dec 16 14:11:36 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B55DE83677 for ; Sat, 16 Dec 2017 14:11:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7907D65152 for ; Sat, 16 Dec 2017 14:11:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vBGEBaGZ028147 for ; Sat, 16 Dec 2017 14:11:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 223803] zio_done() panic on system with cache devices Date: Sat, 16 Dec 2017 14:11:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Dec 2017 14:11:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223803 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.=