From owner-freebsd-fs@freebsd.org Thu May 17 17:39:20 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAF20EDAB0B for ; Thu, 17 May 2018 17:39:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 455C56C036 for ; Thu, 17 May 2018 17:39:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id F3287EDAB07; Thu, 17 May 2018 17:39:19 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E085CEDAB03 for ; Thu, 17 May 2018 17:39:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3FDB46C033 for ; Thu, 17 May 2018 17:39:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w4HHd4kN028145 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 17 May 2018 20:39:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w4HHd4kN028145 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w4HHd42p028144; Thu, 17 May 2018 20:39:04 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 17 May 2018 20:39:04 +0300 From: Konstantin Belousov To: Ryan Stone Cc: "freebsd-fs@freebsd.org" Subject: Re: "NFS append race" Message-ID: <20180517173904.GR6887@kib.kiev.ua> References: <20180513184120.GY6887@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.5 (2018-04-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 17:39:20 -0000 On Thu, May 17, 2018 at 11:45:45AM -0400, Ryan Stone wrote: > Do you have a patch you can try, or some pointers as to where to fix > it and how? I'm not very familiar with NFS or VFS but I could give it > a shot. No, I do not have patches. You might take a look at the calls to ncl_asyncio() and the ncl_asyncio() code itself. It does BUF_KERNPROC() to get rid of the lock ownership for the buffer lock, essentially the lock is not a lock but semaphore. Then the io occurs in the context of the nfsiod thread. Vnode lock is not transferred to the nfsiod thread, and since it typically needs to access the nfs node, things are somewhat messy. Note that comparing with the local filesystems, after the buffer logical block number b_lblkno is translated to the device physical block number b_blkno, buffer completion typically does not require interaction with the vnode, so BUF_KERNPROC() is more sane there.