From owner-freebsd-current@freebsd.org Sun Feb 18 07:28:42 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C73D8F1A16B for ; Sun, 18 Feb 2018 07:28:42 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 5EE4187FA9 for ; Sun, 18 Feb 2018 07:28:42 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 1FEC7F1A166; Sun, 18 Feb 2018 07:28:42 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED001F1A165 for ; Sun, 18 Feb 2018 07:28:41 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f46.google.com (mail-lf0-f46.google.com [209.85.215.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D88787FA3; Sun, 18 Feb 2018 07:28:41 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f46.google.com with SMTP id q194so9119013lfe.13; Sat, 17 Feb 2018 23:28:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=gyjFeFG1MDYt2sco42JS8X/p1yBsPt/60AMQdHHTdg8=; b=kGZdfs+RgBcuGSDjex/1iDa7tZscW8ZzqtTx2HC6paTf5qX5Nl5JRZNJ6lioaPaExN 7Ni6f0w72kRjafJ/65xlZ2YNuyqQ0wY/wdnIV3ZeYBrEZOzTSHUF4yFZzr9hYs5lET0r fUvetYLp3uYSG1G3o2vSKvH2O1eDXr1t0VFJtEu9pvbRUJ5DuRJyz5qaxLK7zmaFA7lI 2a7+rt8M95kVxLjN3mVmnFGDv+M/ZwHh5W6sy681RcupZiEag3jlCbbDnjCTMwc6lest 4Rx819oZCZ0FTbkauPIbxeF9sD+E1m8HlyUbRNBKPECz1r7WNN7Ib+YZFoNCGJEN6g4/ K0zw== X-Gm-Message-State: APf1xPCi9xKZ6WE2JnVxA8ZFngzh+fSIRRExMkHgqI9uT5NUY919wXRd 3KalE1HGZWzXfoR0k1EZec0InxRV X-Google-Smtp-Source: AH8x2260N70HRvKQm6Sq2W4gzStvOlPPzVDMiRjAKLDETniqZNdBWo4puGJB6RxKhePiKopCNruAMw== X-Received: by 10.25.87.193 with SMTP id l184mr7119448lfb.2.1518938913145; Sat, 17 Feb 2018 23:28:33 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id q133sm3573758lfe.60.2018.02.17.23.28.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Feb 2018 23:28:32 -0800 (PST) Subject: Re: Since last week (today) current on my Ryzen box is unstable To: Gleb Smirnoff Cc: Andrew Reilly , kib@FreeBSD.org, current@FreeBSD.org References: <0CEA9D55-D488-42EC-BBDE-D0B7CE58BAEA@bigpond.net.au> <20180218023545.GE93303@FreeBSD.org> From: Andriy Gapon Message-ID: <431f3e00-c66a-8e2e-6c61-a315a6353d1d@FreeBSD.org> Date: Sun, 18 Feb 2018 09:28:30 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180218023545.GE93303@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Feb 2018 07:28:43 -0000 On 18/02/2018 04:35, Gleb Smirnoff wrote: > Andriy, > > On Sun, Feb 18, 2018 at 12:54:21AM +0200, Andriy Gapon wrote: > A> > Today's rebuild has given me uptimes of below an hour, usually. The box will stay up in single user mode long enough to rebuild world/kernel, but multi-user it is panicking at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:1592 > A> > > A> > The backtrace shows that it gets to this panic from a sendfile() syscall. The line above is in the middle of a big edit that's part of svn revision 329363. The tripping assertion seems to suggest that m->valid != 0, for whatever that's worth. > A> > A> I am doing a bit of an offline investigation with Andrew and it seems that the > A> actual panic message is this: > A> > A> panic: vm_page_assert_xbusied: page 0xfffff807ebbd8f98 not exclusive busy @ > A> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:1592 > A> > A> The stack is this: > A> vpanic() at vpanic/frame 0xfffffe00b3c36390 > A> dmu_read_pages() at dmu_read_pages+0x535/frame 0xfffffe00b3c36460 > A> zfs_freebsd_getpages() at zfs_freebsd_getpages+0x24c/frame 0xfffffe00b3c36510 > A> VOP_GETPAGES_APV() at VOP_GETPAGES_APV+0xd9/frame 0xfffffe00b3c36540 > A> vop_stdgetpages_async() at vop_stdgetpages_async+0x49/frame 0xfffffe00b3c36590 > A> VOP_GETPAGES_ASYNC_APV() at VOP_GETPAGES_ASYNC_APV+0xd9/frame 0xfffffe00b3c365c0 > A> vnode_pager_getpages_async() at vnode_pager_getpages_async+0x81/frame > A> 0xfffffe00b3c36650 > A> vn_sendfile() at vn_sendfile+0xe70/frame 0xfffffe00b3c368e0 > A> sendfile() at sendfile+0x149/frame 0xfffffe00b3c36980 > A> amd64_syscall() at amd64_syscall+0x79b/frame 0xfffffe00b3c36ab0 > A> fast_syscall_common() at fast_syscall_common+0x101/frame 0x7fffffffdb00 > A> > A> I looked at sendfile_swapin() code and it seems that it uses the pager API in an > A> undocumented way. Specifically, it inserts bogus_page into the array of > A> requested pages. For starters, bogus_page is not busied and VOP_GETPAGES is > A> documented to have all requested pages exclusively busied. Second, I always had > A> an impression that bogus_page is an implementation detail of the unified buffer > A> / page cache and that other code need not be aware of it. > A> > A> So, my opinion is that the sendfile code uses a "clever hack" that happens to > A> work with the buffer cache based filesystems, but that that hack is a bug. > A> So, I'd prefer that the problem is fixed in that code. > A> But I am open to being convinced that all VOP_GETPAGES implementations, > A> including that in ZFS, must be made aware of bogus_page. Or, at least, that > A> they should not verify that the requested pages are busied. > > This is optimization that improves throughput when file memory cache is > fragmented. Why don't you like adding the code to zfs_freebsd_getpages()? I cited two reasons above and expected to hear some counter-points rather than them being ignored :-) If we settle upon allowing bogus_page to be used in ma[], then that will obviously need to be documented. -- Andriy Gapon