From owner-freebsd-arch@FreeBSD.ORG Mon Jun 15 11:31:05 2015 Return-Path: Delivered-To: arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0EDB589 for ; Mon, 15 Jun 2015 11:31:04 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7064C1AA for ; Mon, 15 Jun 2015 11:31:03 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t5FBV1rP014435 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 15 Jun 2015 14:31:01 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t5FBV0il014434; Mon, 15 Jun 2015 14:31:00 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 15 Jun 2015 14:31:00 +0300 From: Gleb Smirnoff To: Alan Cox Cc: Konstantin Belousov , arch@FreeBSD.org Subject: Re: more strict KPI for vm_pager_get_pages() Message-ID: <20150615113100.GW73119@glebius.int.ru> References: <20150430142408.GS546@nginx.com> <20150506114549.GS34544@FreeBSD.org> <20150610183047.GT2499@kib.kiev.ua> <20150610184251.GP73119@glebius.int.ru> <20150611024706.GW2499@kib.kiev.ua> <20150611103743.GV73119@glebius.int.ru> <20150612040959.GB2080@kib.kiev.ua> <557B3311.40908@rice.edu> <20150612204442.GL73119@glebius.int.ru> <557B6044.7060103@rice.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <557B6044.7060103@rice.edu> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 11:31:05 -0000 Alan, On Fri, Jun 12, 2015 at 05:42:12PM -0500, Alan Cox wrote: A> > A> On the other hand, I'm not ready to make a guarantee about the state of A> > A> the array entries for the non-request pages. In the general case, the A> > A> I/O completion handlers will unbusy and place the pages in a paging A> > A> queue. So, in principle, they could be reclaimed before control was A> > A> returned to vm_pager_get_pages()'s caller, and even if the pager updated A> > A> the array entries, they would no longer be valid. For example, the page A> > A> daemon could reclaim them, or another thread could simply decide to free A> > A> them for some arbitrary reason. A> > A> A> > A> In a nutshell, I'm fine with all of the changes except the one to A> > A> vm_thread_swapin(). The change to vm_thread_swapin() is only safe A> > A> because the pages have been wired and the pages are used in a particular A> > A> way, i.e., the implementation of a thread stack. A> > A> > Replying to the last two paragraphs: A> > A> > Yes, and this lack of guarantee is the inconsistency, that I'd like to A> > address. The patch committed is only a first step of a bigger A> > vm_pager_get_pages KPI strictening. A> > A> > Let's get back to the patch that started this topic: A> > A> > https://lists.freebsd.org/pipermail/freebsd-arch/2015-April/017154.html A> A> I'm not sure that I understand what inconsistency you're referring to A> here. That the request page is handled differently from the non-request A> pages? A> A> Again, I'm happy with the changes to the handling of the request page. A> However, I'm still on the fence about the other proposed changes, and I A> feel like the change to vm_thread_swapin() in the patch we are A> discussing is qualitatively different from the other changes in that A> same patch. In particular, it is the only part of that patch that A> touches non-request pages. As such, I didn't think it belongs. The vm_thread_swapin() is different from other consumers of vm_pager_get_pages() since it is interested in non-request pages as much as it is interested in the request page. The vm_thread_swapin() is rewritten to utilize the new KPI. Before rewrite, it could request for pages, that couldn't be read. The older KPI allowed that, since succeeding only with the reqpage meant success. The new KPI would fail on such request. Now, if we want to do multiple page request to vm_pager_get_pages(), we first need to determine how many pages can we ask for via vm_pager_has_page(). -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Mon Jun 15 16:04:54 2015 Return-Path: Delivered-To: freebsd-arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AE86BAF for ; Mon, 15 Jun 2015 16:04:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f175.google.com (mail-ie0-f175.google.com [209.85.223.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 563C11C3 for ; Mon, 15 Jun 2015 16:04:53 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by iecrd14 with SMTP id rd14so34130217iec.3 for ; Mon, 15 Jun 2015 09:04:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=lY/oJGvz5FGwIL0d24Hrldu2m6A5P2flScjk2L4bOVs=; b=FGwYvQHeWmXuV4KsCqbYae0QimvSUWAsczvHv6yM34tjLh7l8s9JHACXPLhNYeAzxW nlhBoLfiswQ404dAlHqW4kxrmVsaW9Gr8t4qYIZWqUHAisUT/7gMw2wvsZ7qXJW1QXrZ 11wwEXJjidwe9Qnh6krMQaHoS1agGO3UoxSpInFOWw75NRKv96J0nblOdeMkfjSe07bV XxBUFMrfy7rtjDjb/xyynmR0DJhkHPnnBZZq1FEbFjLDSk88vGHg9NkiU2RnIF6mEmDs 294lS0ZutsgZM6aRK3Cw1DvkYZk7PbYc49riMLd+pl5aoY6mjEfGuZHAOKkFWMy7Rbcv Qh0g== X-Gm-Message-State: ALoCoQmbiAuRqxs/tnv/hKyyiX0dfruAUED2Sj3cDloDYyEZuSAg3p+Brd29xzZzqaHhNh8ZA0fH X-Received: by 10.107.137.22 with SMTP id l22mr3606699iod.21.1434384293218; Mon, 15 Jun 2015 09:04:53 -0700 (PDT) Received: from netflix-mac-wired.bsdimp.com ([50.253.99.174]) by mx.google.com with ESMTPSA id z15sm6145735igp.12.2015.06.15.09.04.51 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 15 Jun 2015 09:04:52 -0700 (PDT) Sender: Warner Losh Subject: Re: [rfc] add MK_TELNET_SSL as a build option Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\)) Content-Type: multipart/signed; boundary="Apple-Mail=_7C768891-0016-4C56-BA5A-C605A3656658"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b6 From: Warner Losh In-Reply-To: Date: Mon, 15 Jun 2015 10:04:51 -0600 Cc: "freebsd-arch@freebsd.org" Message-Id: <14F5785C-E07D-4573-8659-AFBB90271B3F@bsdimp.com> References: To: Adrian Chadd X-Mailer: Apple Mail (2.2098) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 16:04:54 -0000 --Apple-Mail=_7C768891-0016-4C56-BA5A-C605A3656658 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jun 13, 2015, at 4:40 PM, Adrian Chadd wrote: > I'd appreciate feedback here as it does involve addin ga new MK, and I > really dislike it. :( Me too. However, in this case it seems necessary. You need to have a = system with SSL, but without SSL telnet. Perhaps, though, you on=E2=80=99t = really need telnet at all on these boxes? > -.if ${MK_OPENSSL} !=3D "no" > +.if ${MK_OPENSSL} !=3D "no" && ${MK_TELNET_SSL} !=3D =E2=80=9Cno=E2=80=9D= ... > -.if ${MK_OPENSSL} !=3D "no" > +.if ${MK_OPENSSL} !=3D "no" && ${MK_TELNET_SSL} !=3D =E2=80=9Cno" etc If MK_OPENSSL is =E2=80=9Cno=E2=80=9D, then MK_TELNET_SSL is necessarily = no. Reduce this to just testing for that. > .if ${MK_CRYPT} =3D=3D "no" > MK_OPENSSL:=3D no > +MK_TELNET_SSL:=3D no > MK_OPENSSH:=3D no > MK_KERBEROS:=3D no The last two (now three) are redunant because they are handled below with the OPENSSL dependency. > .endif > @@ -312,6 +314,7 @@ > .endif >=20 > .if ${MK_OPENSSL} =3D=3D "no" > +MK_TELNET_SSL:=3D no > MK_OPENSSH:=3D no > MK_KERBEROS:=3D no > .endif here... Warner --Apple-Mail=_7C768891-0016-4C56-BA5A-C605A3656658 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJVfvekAAoJEGwc0Sh9sBEAYbUQAKdjMEdqVtyWXzFMrCX4BMb6 UcoyYeNUan7szMhFIAbfAHe3jKlWFsPhDAcnRgxp8hH+Yo3/m06HW+Pjyx1V5ukm HKFjLd/pF7AX7kWkRvWwAvYe6IwyhjyTSXr0oK7hx3b4AimHCP52kjKATDEUg1+1 ofn+FqCbVOsg9MH6Xj7GHNQ26sriZkWCuVayB+nSCMzUn2XMmQ3Bms8k5d02anSc ix5AYr9/UpeD2NuwOjffYQ9ovXyg5eBamri4kXSipCs8sA/cmAw0/vH9i4nXpIVS KQicr3vcfYTTmlpEMwJYu1cvyV/s/fNsRjm1mcJbGbaG7fC9NSdV16mgY4EYxXI3 TKOikB2eh1A2aAY3vSrD+AsbXyKYrzQIU3Bo8IlegGCPoBo7FscvncxZOzZ+69j9 YOj4J2jfzd2dvDkMR4MW6oXTxotZX5r09WQ4dE07s3HVqGQhSiMCU+TxQ2o+PMo+ HJo98kpwNophrXqnrHn4c319x+JrpFdyk535Wqry9gP2qy6K2BuM6gZ9ZwX2kw/H 5E6kA8j/AnKKyctnFQJEVQpKF5BalWfL2LcHh52CCOO9WY4GRAuCdj4ocJBb35rh iVjLIuPmY70p0n7QraykK7AcwGcOUG9TmuDBblzbA1sVI42jX6uKQsBr8cwYwhrA JF8kzf0ZQ1ZeCqQCHVbH =5Dgc -----END PGP SIGNATURE----- --Apple-Mail=_7C768891-0016-4C56-BA5A-C605A3656658-- From owner-freebsd-arch@FreeBSD.ORG Mon Jun 15 16:19:07 2015 Return-Path: Delivered-To: freebsd-arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD5FA4A0 for ; Mon, 15 Jun 2015 16:19:07 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ig0-x230.google.com (mail-ig0-x230.google.com [IPv6:2607:f8b0:4001:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 86C3A686 for ; Mon, 15 Jun 2015 16:19:07 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by igbiq7 with SMTP id iq7so22224565igb.1 for ; Mon, 15 Jun 2015 09:19:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=pKt6s2mABBmhL/G7clO3K8XTaZ++LEOOXU58R8Prqsw=; b=IgS5sPtyfE1SBZIsMstNDoTIv+JXuJyz3Ns889hOL4V67f4k4X8sGT+0xlbL5PYWao +Jdwj7hdIE/jsXxH+mQda5TROdEvTt3JaB/ajhp2DqzDUawJIkc8iDA6NdhqzPywSQrL wcH4FOhHnGny50L18AL1zbhgkg9pCkgZbsW3wWNY3AWG4ovzZ6asLdH5ZJSWcb+0RP56 SEUXGc6luZnb21G8dlY64otR2bANa1sz2OC2+WnEenQQp5EiicKH6bPNRn6f671l4k+d qyuprv7eGZemhT7sKfehHk/R2v1OyhwxBUq+Vbzivm430Ekb09vIX2Pttbw/l2Ic5OKF mi8Q== MIME-Version: 1.0 X-Received: by 10.107.155.74 with SMTP id d71mr35185556ioe.29.1434385147016; Mon, 15 Jun 2015 09:19:07 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.38.133 with HTTP; Mon, 15 Jun 2015 09:19:06 -0700 (PDT) In-Reply-To: <14F5785C-E07D-4573-8659-AFBB90271B3F@bsdimp.com> References: <14F5785C-E07D-4573-8659-AFBB90271B3F@bsdimp.com> Date: Mon, 15 Jun 2015 09:19:06 -0700 X-Google-Sender-Auth: Uf-rhPvEfB5ND_ot0SbRmhFInEY Message-ID: Subject: Re: [rfc] add MK_TELNET_SSL as a build option From: Adrian Chadd To: Warner Losh Cc: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 16:19:07 -0000 On 15 June 2015 at 09:04, Warner Losh wrote: > >> On Jun 13, 2015, at 4:40 PM, Adrian Chadd wrote: >> I'd appreciate feedback here as it does involve addin ga new MK, and I >> really dislike it. :( > > Me too. However, in this case it seems necessary. You need to have a syst= em > with SSL, but without SSL telnet. Perhaps, though, you on=E2=80=99t reall= y need telnet > at all on these boxes? Well, we need telnet/telnetd on them right until we can build dropbear as part of the base system or base package system to go into the image. Until then, it's telnet/telnetd. -a From owner-freebsd-arch@FreeBSD.ORG Mon Jun 15 16:36:10 2015 Return-Path: Delivered-To: freebsd-arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D2E0212; Mon, 15 Jun 2015 16:36:10 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id D6A0EC24; Mon, 15 Jun 2015 16:36:09 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.60.3]) by phk.freebsd.dk (Postfix) with ESMTP id 2F6AB3BD35; Mon, 15 Jun 2015 16:36:03 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.9/8.14.9) with ESMTP id t5FGa13t087669; Mon, 15 Jun 2015 16:36:02 GMT (envelope-from phk@phk.freebsd.dk) To: Warner Losh cc: Adrian Chadd , "freebsd-arch@freebsd.org" Subject: Re: [rfc] add MK_TELNET_SSL as a build option In-reply-to: <14F5785C-E07D-4573-8659-AFBB90271B3F@bsdimp.com> From: "Poul-Henning Kamp" References: <14F5785C-E07D-4573-8659-AFBB90271B3F@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <87667.1434386161.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Mon, 15 Jun 2015 16:36:01 +0000 Message-ID: <87668.1434386161@critter.freebsd.dk> X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 16:36:10 -0000 -------- In message <14F5785C-E07D-4573-8659-AFBB90271B3F@bsdimp.com>, Warner Losh = write s: >> On Jun 13, 2015, at 4:40 PM, Adrian Chadd wrote: >> I'd appreciate feedback here as it does involve addin ga new MK, and I >> really dislike it. :( This reminds me: Somebody should really adopt src/tools/tools/build_option_survey and run it periodically so people can find out the space cost of the various build options. Here is a very old output to show what it produces: http://phk/misc/build_options_stable_8_210741/ -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-arch@FreeBSD.ORG Mon Jun 15 21:29:36 2015 Return-Path: Delivered-To: arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32E46613 for ; Mon, 15 Jun 2015 21:29:36 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C2762FD6 for ; Mon, 15 Jun 2015 21:29:34 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t5FLTVMS017921 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 16 Jun 2015 00:29:31 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t5FLTVf0017920; Tue, 16 Jun 2015 00:29:31 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 16 Jun 2015 00:29:31 +0300 From: Gleb Smirnoff To: Alan Cox , Konstantin Belousov Cc: arch@FreeBSD.org Subject: Step 2 Was: more strict KPI for vm_pager_get_pages() Message-ID: <20150615212931.GG73119@glebius.int.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="t5NgoZwlhlUmGr82" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 21:29:36 -0000 --t5NgoZwlhlUmGr82 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi! This is step 2 of the "more strict pager KPI" patch: o Uninline vm_pager_get_pages() into vm_pager.c. o Keep all KASSERTs that enforce the KPI of pagers and of their consumers in one place: vm_pager_get_pages(). o Keep the code that zeroes out partially valid pages in one place in vm_pager_get_pages(). -- Totus tuus, Glebius. --t5NgoZwlhlUmGr82 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="pagers.step.2.diff" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 284409) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -5734,8 +5734,6 @@ zfs_getpages(struct vnode *vp, vm_page_t *m, int c object = mreq->object; error = 0; - KASSERT(vp->v_object == object, ("mismatching object")); - if (pcount > 1 && zp->z_blksz > PAGESIZE) { startoff = rounddown(IDX_TO_OFF(mreq->pindex), zp->z_blksz); reqstart = OFF_TO_IDX(round_page(startoff)); @@ -5771,14 +5769,6 @@ zfs_getpages(struct vnode *vp, vm_page_t *m, int c vm_page_unlock(m[i]); } - if (mreq->valid && reqsize == 1) { - if (mreq->valid != VM_PAGE_BITS_ALL) - vm_page_zero_invalid(mreq, TRUE); - zfs_vmobject_wunlock(object); - ZFS_EXIT(zfsvfs); - return (zfs_vm_pagerret_ok); - } - PCPU_INC(cnt.v_vnodein); PCPU_ADD(cnt.v_vnodepgsin, reqsize); Index: sys/fs/fuse/fuse_vnops.c =================================================================== --- sys/fs/fuse/fuse_vnops.c (revision 284409) +++ sys/fs/fuse/fuse_vnops.c (working copy) @@ -1761,29 +1761,6 @@ fuse_vnop_getpages(struct vop_getpages_args *ap) npages = btoc(count); /* - * If the requested page is partially valid, just return it and - * allow the pager to zero-out the blanks. Partially valid pages - * can only occur at the file EOF. - */ - - VM_OBJECT_WLOCK(vp->v_object); - fuse_vm_page_lock_queues(); - if (pages[ap->a_reqpage]->valid != 0) { - for (i = 0; i < npages; ++i) { - if (i != ap->a_reqpage) { - fuse_vm_page_lock(pages[i]); - vm_page_free(pages[i]); - fuse_vm_page_unlock(pages[i]); - } - } - fuse_vm_page_unlock_queues(); - VM_OBJECT_WUNLOCK(vp->v_object); - return 0; - } - fuse_vm_page_unlock_queues(); - VM_OBJECT_WUNLOCK(vp->v_object); - - /* * We use only the kva address for the buffer, but this is extremely * convienient and fast. */ Index: sys/fs/nfsclient/nfs_clbio.c =================================================================== --- sys/fs/nfsclient/nfs_clbio.c (revision 284409) +++ sys/fs/nfsclient/nfs_clbio.c (working copy) @@ -129,23 +129,6 @@ ncl_getpages(struct vop_getpages_args *ap) npages = btoc(count); /* - * Since the caller has busied the requested page, that page's valid - * field will not be changed by other threads. - */ - vm_page_assert_xbusied(pages[ap->a_reqpage]); - - /* - * If the requested page is partially valid, just return it and - * allow the pager to zero-out the blanks. Partially valid pages - * can only occur at the file EOF. - */ - if (pages[ap->a_reqpage]->valid != 0) { - vm_pager_free_nonreq(object, pages, ap->a_reqpage, npages, - FALSE); - return (VM_PAGER_OK); - } - - /* * We use only the kva address for the buffer, but this is extremely * convienient and fast. */ Index: sys/fs/smbfs/smbfs_io.c =================================================================== --- sys/fs/smbfs/smbfs_io.c (revision 284409) +++ sys/fs/smbfs/smbfs_io.c (working copy) @@ -436,7 +436,7 @@ smbfs_getpages(ap) struct smbnode *np; struct smb_cred *scred; vm_object_t object; - vm_page_t *pages, m; + vm_page_t *pages; vp = ap->a_vp; if ((object = vp->v_object) == NULL) { @@ -453,27 +453,6 @@ smbfs_getpages(ap) npages = btoc(count); reqpage = ap->a_reqpage; - /* - * If the requested page is partially valid, just return it and - * allow the pager to zero-out the blanks. Partially valid pages - * can only occur at the file EOF. - */ - m = pages[reqpage]; - - VM_OBJECT_WLOCK(object); - if (m->valid != 0) { - for (i = 0; i < npages; ++i) { - if (i != reqpage) { - vm_page_lock(pages[i]); - vm_page_free(pages[i]); - vm_page_unlock(pages[i]); - } - } - VM_OBJECT_WUNLOCK(object); - return 0; - } - VM_OBJECT_WUNLOCK(object); - scred = smbfs_malloc_scred(); smb_makescred(scred, td, cred); Index: sys/vm/swap_pager.c =================================================================== --- sys/vm/swap_pager.c (revision 284409) +++ sys/vm/swap_pager.c (working copy) @@ -1118,10 +1118,6 @@ swap_pager_getpages(vm_object_t object, vm_page_t mreq = m[reqpage]; - KASSERT(mreq->object == object, - ("swap_pager_getpages: object mismatch %p/%p", - object, mreq->object)); - /* * Calculate range to retrieve. The pages have already been assigned * their swapblks. We require a *contiguous* range but we know it to Index: sys/vm/vm_pager.c =================================================================== --- sys/vm/vm_pager.c (revision 284409) +++ sys/vm/vm_pager.c (working copy) @@ -251,7 +251,90 @@ vm_pager_deallocate(object) } /* - * vm_pager_get_pages() - inline, see vm/vm_pager.h + * Retrieve pages from the VM system in order to map them into an object + * ( or into VM space somewhere ). If the pagein was successful, we + * must fully validate it. + */ +int +vm_pager_get_pages(vm_object_t object, vm_page_t *m, int count, int reqpage) +{ + int r; + + VM_OBJECT_ASSERT_WLOCKED(object); + KASSERT(count > 0, ("%s: 0 count", __func__)); + + /* + * If the request page is partially valid, just return it and zero-out + * the blanks. Partially valid pages can only occur at the file EOF. + */ + if (m[reqpage]->valid != 0) { + vm_page_zero_invalid(m[reqpage], TRUE); + vm_pager_free_nonreq(object, m, reqpage, count, TRUE); + return (VM_PAGER_OK); + } + +#ifdef INVARIANTS + /* + * All pages must be busied, not mapped, not valid, not dirty + * and belong to the proper object. + */ + for (int i = 0 ; i < count; i++) { + vm_page_assert_xbusied(m[i]); + KASSERT(!pmap_page_is_mapped(m[i]), + ("%s: page %p is mapped", __func__, m[i])); + KASSERT(m[i]->valid == 0, + ("%s: request for a valid page %p", __func__, m[i])); + KASSERT(m[i]->dirty == 0, + ("%s: page %p is dirty", __func__, m[i])); + KASSERT(m[i]->object == object, + ("%s: wrong object %p/%p", __func__, object, m[i]->object)); + } +#endif + + r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); + if (r != VM_PAGER_OK) + return (r); + + /* + * If pager has replaced the page, assert that it had + * updated the array. + */ + KASSERT(m[reqpage] == vm_page_lookup(object, m[reqpage]->pindex), + ("%s: mismatch page %p pindex %ju", __func__, + m[reqpage], (uintmax_t )m[reqpage]->pindex - 1)); + /* + * Pager didn't fill up entire page. Zero out + * partially filled data. + */ + if (m[reqpage]->valid != VM_PAGE_BITS_ALL) + vm_page_zero_invalid(m[reqpage], TRUE); + + return (VM_PAGER_OK); +} + +int +vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, + int reqpage, pgo_getpages_iodone_t iodone, void *arg) +{ + + VM_OBJECT_ASSERT_WLOCKED(object); + KASSERT(count > 0, ("%s: 0 count", __func__)); + + /* + * If the last page is partially valid, just return it and zero-out + * the blanks. Partially valid pages can only occur at the file EOF. + */ + if (m[reqpage]->valid != 0) { + vm_page_zero_invalid(m[reqpage], TRUE); + iodone(arg, m, reqpage, 0); + return (VM_PAGER_OK); + } + + return ((*pagertab[object->type]->pgo_getpages_async)(object, m, + count, reqpage, iodone, arg)); +} + +/* * vm_pager_put_pages() - inline, see vm/vm_pager.h * vm_pager_has_page() - inline, see vm/vm_pager.h */ Index: sys/vm/vm_pager.h =================================================================== --- sys/vm/vm_pager.h (revision 284409) +++ sys/vm/vm_pager.h (working copy) @@ -106,9 +106,9 @@ vm_object_t vm_pager_allocate(objtype_t, void *, v vm_ooffset_t, struct ucred *); void vm_pager_bufferinit(void); void vm_pager_deallocate(vm_object_t); -static __inline int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); -static inline int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, - int, pgo_getpages_iodone_t, void *); +int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); +int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, int, + pgo_getpages_iodone_t, void *); static __inline boolean_t vm_pager_has_page(vm_object_t, vm_pindex_t, int *, int *); void vm_pager_init(void); vm_object_t vm_pager_object_lookup(struct pagerlst *, void *); @@ -115,40 +115,6 @@ vm_object_t vm_pager_object_lookup(struct pagerlst void vm_pager_free_nonreq(vm_object_t object, vm_page_t ma[], int reqpage, int npages, boolean_t object_locked); -/* - * vm_page_get_pages: - * - * Retrieve pages from the VM system in order to map them into an object - * ( or into VM space somewhere ). If the pagein was successful, we - * must fully validate it. - */ -static __inline int -vm_pager_get_pages( - vm_object_t object, - vm_page_t *m, - int count, - int reqpage -) { - int r; - - VM_OBJECT_ASSERT_WLOCKED(object); - r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); - if (r == VM_PAGER_OK && m[reqpage]->valid != VM_PAGE_BITS_ALL) { - vm_page_zero_invalid(m[reqpage], TRUE); - } - return (r); -} - -static inline int -vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, - int reqpage, pgo_getpages_iodone_t iodone, void *arg) -{ - - VM_OBJECT_ASSERT_WLOCKED(object); - return ((*pagertab[object->type]->pgo_getpages_async)(object, m, - count, reqpage, iodone, arg)); -} - static __inline void vm_pager_put_pages( vm_object_t object, Index: sys/vm/vnode_pager.c =================================================================== --- sys/vm/vnode_pager.c (revision 284409) +++ sys/vm/vnode_pager.c (working copy) @@ -84,8 +84,6 @@ static int vnode_pager_addr(struct vnode *vp, vm_o static int vnode_pager_input_smlfs(vm_object_t object, vm_page_t m); static int vnode_pager_input_old(vm_object_t object, vm_page_t m); static void vnode_pager_dealloc(vm_object_t); -static int vnode_pager_local_getpages0(struct vnode *, vm_page_t *, int, int, - vop_getpages_iodone_t, void *); static int vnode_pager_getpages(vm_object_t, vm_page_t *, int, int); static int vnode_pager_getpages_async(vm_object_t, vm_page_t *, int, int, vop_getpages_iodone_t, void *); @@ -714,7 +712,7 @@ int vnode_pager_local_getpages(struct vop_getpages_args *ap) { - return (vnode_pager_local_getpages0(ap->a_vp, ap->a_m, ap->a_count, + return (vnode_pager_generic_getpages(ap->a_vp, ap->a_m, ap->a_count, ap->a_reqpage, NULL, NULL)); } @@ -722,42 +720,10 @@ int vnode_pager_local_getpages_async(struct vop_getpages_async_args *ap) { - return (vnode_pager_local_getpages0(ap->a_vp, ap->a_m, ap->a_count, + return (vnode_pager_generic_getpages(ap->a_vp, ap->a_m, ap->a_count, ap->a_reqpage, ap->a_iodone, ap->a_arg)); } -static int -vnode_pager_local_getpages0(struct vnode *vp, vm_page_t *m, int bytecount, - int reqpage, vop_getpages_iodone_t iodone, void *arg) -{ - vm_page_t mreq; - - mreq = m[reqpage]; - - /* - * Since the caller has busied the requested page, that page's valid - * field will not be changed by other threads. - */ - vm_page_assert_xbusied(mreq); - - /* - * The requested page has valid blocks. Invalid part can only - * exist at the end of file, and the page is made fully valid - * by zeroing in vm_pager_get_pages(). Free non-requested - * pages, since no i/o is done to read its content. - */ - if (mreq->valid != 0) { - vm_pager_free_nonreq(mreq->object, m, reqpage, - round_page(bytecount) / PAGE_SIZE, FALSE); - if (iodone != NULL) - iodone(arg, m, reqpage, 0); - return (VM_PAGER_OK); - } - - return (vnode_pager_generic_getpages(vp, m, bytecount, reqpage, - iodone, arg)); -} - /* * This is now called from local media FS's to operate against their * own vnodes if they fail to implement VOP_GETPAGES. --t5NgoZwlhlUmGr82-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 17 12:33:10 2015 Return-Path: Delivered-To: arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4849CFFF for ; Wed, 17 Jun 2015 12:33:10 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6626534F for ; Wed, 17 Jun 2015 12:33:08 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t5HAEjHC028265 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 17 Jun 2015 13:14:45 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t5HAEjCE028264; Wed, 17 Jun 2015 13:14:45 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 17 Jun 2015 13:14:45 +0300 From: Gleb Smirnoff To: Alan Cox , Konstantin Belousov Cc: arch@FreeBSD.org Subject: Re: Step 2 Was: more strict KPI for vm_pager_get_pages() Message-ID: <20150617101444.GW73119@glebius.int.ru> References: <20150615212931.GG73119@glebius.int.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="FX+Db2fp7WJhXKrW" Content-Disposition: inline In-Reply-To: <20150615212931.GG73119@glebius.int.ru> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 12:33:10 -0000 --FX+Db2fp7WJhXKrW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi! On Tue, Jun 16, 2015 at 12:29:31AM +0300, Gleb Smirnoff wrote: T> This is step 2 of the "more strict pager KPI" patch: T> T> o Uninline vm_pager_get_pages() into vm_pager.c. T> o Keep all KASSERTs that enforce the KPI of pagers and of their T> consumers in one place: vm_pager_get_pages(). T> o Keep the code that zeroes out partially valid pages in one T> place in vm_pager_get_pages(). As Konstantin pointed out, it is very likely that right now NFS and SMD do the zeroing of partially valid pages incorrecrtly. So this code is removed from the generic vm_pager_get_pages() and this reduces step 2 patch to a very small change: o Uninline vm_pager_get_pages() into vm_pager.c. o Keep all KASSERTs that enforce the KPI of pagers and of their consumers in one place: vm_pager_get_pages(). o Remove some KASSERTs from pagers that are already there in vm_pager_get_pages(). Patch attached. -- Totus tuus, Glebius. --FX+Db2fp7WJhXKrW Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="pagers.step.2.diff" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 284504) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -5734,8 +5734,6 @@ zfs_getpages(struct vnode *vp, vm_page_t *m, int c object = mreq->object; error = 0; - KASSERT(vp->v_object == object, ("mismatching object")); - if (pcount > 1 && zp->z_blksz > PAGESIZE) { startoff = rounddown(IDX_TO_OFF(mreq->pindex), zp->z_blksz); reqstart = OFF_TO_IDX(round_page(startoff)); Index: sys/fs/nfsclient/nfs_clbio.c =================================================================== --- sys/fs/nfsclient/nfs_clbio.c (revision 284504) +++ sys/fs/nfsclient/nfs_clbio.c (working copy) @@ -129,12 +129,6 @@ ncl_getpages(struct vop_getpages_args *ap) npages = btoc(count); /* - * Since the caller has busied the requested page, that page's valid - * field will not be changed by other threads. - */ - vm_page_assert_xbusied(pages[ap->a_reqpage]); - - /* * If the requested page is partially valid, just return it and * allow the pager to zero-out the blanks. Partially valid pages * can only occur at the file EOF. Index: sys/vm/swap_pager.c =================================================================== --- sys/vm/swap_pager.c (revision 284504) +++ sys/vm/swap_pager.c (working copy) @@ -1118,10 +1118,6 @@ swap_pager_getpages(vm_object_t object, vm_page_t mreq = m[reqpage]; - KASSERT(mreq->object == object, - ("swap_pager_getpages: object mismatch %p/%p", - object, mreq->object)); - /* * Calculate range to retrieve. The pages have already been assigned * their swapblks. We require a *contiguous* range but we know it to Index: sys/vm/vm_pager.c =================================================================== --- sys/vm/vm_pager.c (revision 284504) +++ sys/vm/vm_pager.c (working copy) @@ -251,7 +251,88 @@ vm_pager_deallocate(object) } /* - * vm_pager_get_pages() - inline, see vm/vm_pager.h + * Retrieve pages from the VM system in order to map them into an object + * ( or into VM space somewhere ). If the pagein was successful, we + * must fully validate it. + */ +int +vm_pager_get_pages(vm_object_t object, vm_page_t *m, int count, int reqpage) +{ + int r; + + VM_OBJECT_ASSERT_WLOCKED(object); + KASSERT(count > 0, ("%s: 0 count", __func__)); + +#ifdef INVARIANTS + /* + * All pages must be busied, not mapped, not valid (save the last one), + * not dirty and belong to the proper object. + */ + for (int i = 0 ; i < count; i++) { + vm_page_assert_xbusied(m[i]); + KASSERT(!pmap_page_is_mapped(m[i]), + ("%s: page %p is mapped", __func__, m[i])); + KASSERT(m[i]->valid == 0 || i == count - 1, + ("%s: request for a valid page %p", __func__, m[i])); + KASSERT(m[i]->dirty == 0, + ("%s: page %p is dirty", __func__, m[i])); + KASSERT(m[i]->object == object, + ("%s: wrong object %p/%p", __func__, object, m[i]->object)); + } +#endif + + r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); + if (r != VM_PAGER_OK) + return (r); + + /* + * If pager has replaced the page, assert that it had + * updated the array. + */ + KASSERT(m[reqpage] == vm_page_lookup(object, m[reqpage]->pindex), + ("%s: mismatch page %p pindex %ju", __func__, + m[reqpage], (uintmax_t )m[reqpage]->pindex - 1)); + /* + * Pager didn't fill up entire page. Zero out + * partially filled data. + */ + if (m[reqpage]->valid != VM_PAGE_BITS_ALL) + vm_page_zero_invalid(m[reqpage], TRUE); + + return (VM_PAGER_OK); +} + +int +vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, + int reqpage, pgo_getpages_iodone_t iodone, void *arg) +{ + + VM_OBJECT_ASSERT_WLOCKED(object); + KASSERT(count > 0, ("%s: 0 count", __func__)); + +#ifdef INVARIANTS + /* + * All pages must be busied, not mapped, not valid (save the last one), + * not dirty and belong to the proper object. + */ + for (int i = 0 ; i < count; i++) { + vm_page_assert_xbusied(m[i]); + KASSERT(!pmap_page_is_mapped(m[i]), + ("%s: page %p is mapped", __func__, m[i])); + KASSERT(m[i]->valid == 0 || i == count - 1, + ("%s: request for a valid page %p", __func__, m[i])); + KASSERT(m[i]->dirty == 0, + ("%s: page %p is dirty", __func__, m[i])); + KASSERT(m[i]->object == object, + ("%s: wrong object %p/%p", __func__, object, m[i]->object)); + } +#endif + + return ((*pagertab[object->type]->pgo_getpages_async)(object, m, + count, reqpage, iodone, arg)); +} + +/* * vm_pager_put_pages() - inline, see vm/vm_pager.h * vm_pager_has_page() - inline, see vm/vm_pager.h */ Index: sys/vm/vm_pager.h =================================================================== --- sys/vm/vm_pager.h (revision 284504) +++ sys/vm/vm_pager.h (working copy) @@ -106,9 +106,9 @@ vm_object_t vm_pager_allocate(objtype_t, void *, v vm_ooffset_t, struct ucred *); void vm_pager_bufferinit(void); void vm_pager_deallocate(vm_object_t); -static __inline int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); -static inline int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, - int, pgo_getpages_iodone_t, void *); +int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); +int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, int, + pgo_getpages_iodone_t, void *); static __inline boolean_t vm_pager_has_page(vm_object_t, vm_pindex_t, int *, int *); void vm_pager_init(void); vm_object_t vm_pager_object_lookup(struct pagerlst *, void *); @@ -115,40 +115,6 @@ vm_object_t vm_pager_object_lookup(struct pagerlst void vm_pager_free_nonreq(vm_object_t object, vm_page_t ma[], int reqpage, int npages, boolean_t object_locked); -/* - * vm_page_get_pages: - * - * Retrieve pages from the VM system in order to map them into an object - * ( or into VM space somewhere ). If the pagein was successful, we - * must fully validate it. - */ -static __inline int -vm_pager_get_pages( - vm_object_t object, - vm_page_t *m, - int count, - int reqpage -) { - int r; - - VM_OBJECT_ASSERT_WLOCKED(object); - r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); - if (r == VM_PAGER_OK && m[reqpage]->valid != VM_PAGE_BITS_ALL) { - vm_page_zero_invalid(m[reqpage], TRUE); - } - return (r); -} - -static inline int -vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, - int reqpage, pgo_getpages_iodone_t iodone, void *arg) -{ - - VM_OBJECT_ASSERT_WLOCKED(object); - return ((*pagertab[object->type]->pgo_getpages_async)(object, m, - count, reqpage, iodone, arg)); -} - static __inline void vm_pager_put_pages( vm_object_t object, --FX+Db2fp7WJhXKrW-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 17 13:45:46 2015 Return-Path: Delivered-To: arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 018F3CC5; Wed, 17 Jun 2015 13:45:45 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7C7678DB; Wed, 17 Jun 2015 13:45:45 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t5HDjcX8025834 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 17 Jun 2015 16:45:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t5HDjcX8025834 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t5HDjcCk025832; Wed, 17 Jun 2015 16:45:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 17 Jun 2015 16:45:38 +0300 From: Konstantin Belousov To: Gleb Smirnoff Cc: Alan Cox , arch@FreeBSD.org Subject: Re: Step 2 Was: more strict KPI for vm_pager_get_pages() Message-ID: <20150617134538.GY2080@kib.kiev.ua> References: <20150615212931.GG73119@glebius.int.ru> <20150617101444.GW73119@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150617101444.GW73119@glebius.int.ru> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 13:45:46 -0000 On Wed, Jun 17, 2015 at 01:14:45PM +0300, Gleb Smirnoff wrote: > Hi! > > On Tue, Jun 16, 2015 at 12:29:31AM +0300, Gleb Smirnoff wrote: > T> This is step 2 of the "more strict pager KPI" patch: > T> > T> o Uninline vm_pager_get_pages() into vm_pager.c. > T> o Keep all KASSERTs that enforce the KPI of pagers and of their > T> consumers in one place: vm_pager_get_pages(). > T> o Keep the code that zeroes out partially valid pages in one > T> place in vm_pager_get_pages(). > > As Konstantin pointed out, it is very likely that right now NFS and SMD > do the zeroing of partially valid pages incorrecrtly. So this code > is removed from the generic vm_pager_get_pages() and this reduces > step 2 patch to a very small change: > > o Uninline vm_pager_get_pages() into vm_pager.c. > o Keep all KASSERTs that enforce the KPI of pagers and of their > consumers in one place: vm_pager_get_pages(). > o Remove some KASSERTs from pagers that are already there in > vm_pager_get_pages(). > > Patch attached. > > -- > Totus tuus, Glebius. > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > =================================================================== > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 284504) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) > @@ -5734,8 +5734,6 @@ zfs_getpages(struct vnode *vp, vm_page_t *m, int c > object = mreq->object; > error = 0; > > - KASSERT(vp->v_object == object, ("mismatching object")); > - > if (pcount > 1 && zp->z_blksz > PAGESIZE) { > startoff = rounddown(IDX_TO_OFF(mreq->pindex), zp->z_blksz); > reqstart = OFF_TO_IDX(round_page(startoff)); > Index: sys/fs/nfsclient/nfs_clbio.c > =================================================================== > --- sys/fs/nfsclient/nfs_clbio.c (revision 284504) > +++ sys/fs/nfsclient/nfs_clbio.c (working copy) > @@ -129,12 +129,6 @@ ncl_getpages(struct vop_getpages_args *ap) > npages = btoc(count); > > /* > - * Since the caller has busied the requested page, that page's valid > - * field will not be changed by other threads. > - */ > - vm_page_assert_xbusied(pages[ap->a_reqpage]); > - > - /* > * If the requested page is partially valid, just return it and > * allow the pager to zero-out the blanks. Partially valid pages > * can only occur at the file EOF. > Index: sys/vm/swap_pager.c > =================================================================== > --- sys/vm/swap_pager.c (revision 284504) > +++ sys/vm/swap_pager.c (working copy) > @@ -1118,10 +1118,6 @@ swap_pager_getpages(vm_object_t object, vm_page_t > > mreq = m[reqpage]; > > - KASSERT(mreq->object == object, > - ("swap_pager_getpages: object mismatch %p/%p", > - object, mreq->object)); > - > /* > * Calculate range to retrieve. The pages have already been assigned > * their swapblks. We require a *contiguous* range but we know it to > Index: sys/vm/vm_pager.c > =================================================================== > --- sys/vm/vm_pager.c (revision 284504) > +++ sys/vm/vm_pager.c (working copy) > @@ -251,7 +251,88 @@ vm_pager_deallocate(object) > } > > /* > - * vm_pager_get_pages() - inline, see vm/vm_pager.h > + * Retrieve pages from the VM system in order to map them into an object > + * ( or into VM space somewhere ). If the pagein was successful, we > + * must fully validate it. > + */ > +int > +vm_pager_get_pages(vm_object_t object, vm_page_t *m, int count, int reqpage) > +{ > + int r; > + > + VM_OBJECT_ASSERT_WLOCKED(object); > + KASSERT(count > 0, ("%s: 0 count", __func__)); > + > +#ifdef INVARIANTS > + /* > + * All pages must be busied, not mapped, not valid (save the last one), > + * not dirty and belong to the proper object. > + */ > + for (int i = 0 ; i < count; i++) { > + vm_page_assert_xbusied(m[i]); > + KASSERT(!pmap_page_is_mapped(m[i]), > + ("%s: page %p is mapped", __func__, m[i])); > + KASSERT(m[i]->valid == 0 || i == count - 1, > + ("%s: request for a valid page %p", __func__, m[i])); Are you sure that e.g. the requested page cannot be partially valid ? I am surprised. > + KASSERT(m[i]->dirty == 0, > + ("%s: page %p is dirty", __func__, m[i])); > + KASSERT(m[i]->object == object, > + ("%s: wrong object %p/%p", __func__, object, m[i]->object)); > + } > +#endif Create a helper function with the loop, and use it, instead of copying the block twice. > + > + r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); > + if (r != VM_PAGER_OK) > + return (r); > + > + /* > + * If pager has replaced the page, assert that it had > + * updated the array. > + */ > + KASSERT(m[reqpage] == vm_page_lookup(object, m[reqpage]->pindex), > + ("%s: mismatch page %p pindex %ju", __func__, > + m[reqpage], (uintmax_t )m[reqpage]->pindex - 1)); Why -1 ? Need an empty line after assert, to deliniate the code which is commented about. Also, you may assert that the requested page is still busied. > + /* > + * Pager didn't fill up entire page. Zero out > + * partially filled data. > + */ > + if (m[reqpage]->valid != VM_PAGE_BITS_ALL) > + vm_page_zero_invalid(m[reqpage], TRUE); > + > + return (VM_PAGER_OK); > +} > + > +int > +vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, > + int reqpage, pgo_getpages_iodone_t iodone, void *arg) > +{ > + > + VM_OBJECT_ASSERT_WLOCKED(object); > + KASSERT(count > 0, ("%s: 0 count", __func__)); > + > +#ifdef INVARIANTS > + /* > + * All pages must be busied, not mapped, not valid (save the last one), > + * not dirty and belong to the proper object. > + */ > + for (int i = 0 ; i < count; i++) { > + vm_page_assert_xbusied(m[i]); > + KASSERT(!pmap_page_is_mapped(m[i]), > + ("%s: page %p is mapped", __func__, m[i])); > + KASSERT(m[i]->valid == 0 || i == count - 1, > + ("%s: request for a valid page %p", __func__, m[i])); > + KASSERT(m[i]->dirty == 0, > + ("%s: page %p is dirty", __func__, m[i])); > + KASSERT(m[i]->object == object, > + ("%s: wrong object %p/%p", __func__, object, m[i]->object)); > + } > +#endif > + > + return ((*pagertab[object->type]->pgo_getpages_async)(object, m, > + count, reqpage, iodone, arg)); > +} > + > +/* > * vm_pager_put_pages() - inline, see vm/vm_pager.h > * vm_pager_has_page() - inline, see vm/vm_pager.h > */ > Index: sys/vm/vm_pager.h > =================================================================== > --- sys/vm/vm_pager.h (revision 284504) > +++ sys/vm/vm_pager.h (working copy) > @@ -106,9 +106,9 @@ vm_object_t vm_pager_allocate(objtype_t, void *, v > vm_ooffset_t, struct ucred *); > void vm_pager_bufferinit(void); > void vm_pager_deallocate(vm_object_t); > -static __inline int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); > -static inline int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, > - int, pgo_getpages_iodone_t, void *); > +int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); > +int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, int, > + pgo_getpages_iodone_t, void *); > static __inline boolean_t vm_pager_has_page(vm_object_t, vm_pindex_t, int *, int *); > void vm_pager_init(void); > vm_object_t vm_pager_object_lookup(struct pagerlst *, void *); > @@ -115,40 +115,6 @@ vm_object_t vm_pager_object_lookup(struct pagerlst > void vm_pager_free_nonreq(vm_object_t object, vm_page_t ma[], int reqpage, > int npages, boolean_t object_locked); > > -/* > - * vm_page_get_pages: > - * > - * Retrieve pages from the VM system in order to map them into an object > - * ( or into VM space somewhere ). If the pagein was successful, we > - * must fully validate it. > - */ > -static __inline int > -vm_pager_get_pages( > - vm_object_t object, > - vm_page_t *m, > - int count, > - int reqpage > -) { > - int r; > - > - VM_OBJECT_ASSERT_WLOCKED(object); > - r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); > - if (r == VM_PAGER_OK && m[reqpage]->valid != VM_PAGE_BITS_ALL) { > - vm_page_zero_invalid(m[reqpage], TRUE); > - } > - return (r); > -} > - > -static inline int > -vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, > - int reqpage, pgo_getpages_iodone_t iodone, void *arg) > -{ > - > - VM_OBJECT_ASSERT_WLOCKED(object); > - return ((*pagertab[object->type]->pgo_getpages_async)(object, m, > - count, reqpage, iodone, arg)); > -} > - > static __inline void > vm_pager_put_pages( > vm_object_t object, From owner-freebsd-arch@FreeBSD.ORG Wed Jun 17 14:29:00 2015 Return-Path: Delivered-To: arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F1D2A6E for ; Wed, 17 Jun 2015 14:29:00 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A6C6E3D2 for ; Wed, 17 Jun 2015 14:28:58 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t5HESopx029558 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 17 Jun 2015 17:28:50 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t5HESoq8029557; Wed, 17 Jun 2015 17:28:50 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 17 Jun 2015 17:28:50 +0300 From: Gleb Smirnoff To: Konstantin Belousov Cc: Alan Cox , arch@FreeBSD.org Subject: Re: Step 2 Was: more strict KPI for vm_pager_get_pages() Message-ID: <20150617142850.GA73119@glebius.int.ru> References: <20150615212931.GG73119@glebius.int.ru> <20150617101444.GW73119@glebius.int.ru> <20150617134538.GY2080@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="NKtYx2Ppz7d1tORf" Content-Disposition: inline In-Reply-To: <20150617134538.GY2080@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 14:29:00 -0000 --NKtYx2Ppz7d1tORf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Konstantin, On Wed, Jun 17, 2015 at 04:45:38PM +0300, Konstantin Belousov wrote: K> > +++ sys/vm/vm_pager.c (working copy) K> > @@ -251,7 +251,88 @@ vm_pager_deallocate(object) K> > } K> > K> > /* K> > - * vm_pager_get_pages() - inline, see vm/vm_pager.h K> > + * Retrieve pages from the VM system in order to map them into an object K> > + * ( or into VM space somewhere ). If the pagein was successful, we K> > + * must fully validate it. K> > + */ K> > +int K> > +vm_pager_get_pages(vm_object_t object, vm_page_t *m, int count, int reqpage) K> > +{ K> > + int r; K> > + K> > + VM_OBJECT_ASSERT_WLOCKED(object); K> > + KASSERT(count > 0, ("%s: 0 count", __func__)); K> > + K> > +#ifdef INVARIANTS K> > + /* K> > + * All pages must be busied, not mapped, not valid (save the last one), K> > + * not dirty and belong to the proper object. K> > + */ K> > + for (int i = 0 ; i < count; i++) { K> > + vm_page_assert_xbusied(m[i]); K> > + KASSERT(!pmap_page_is_mapped(m[i]), K> > + ("%s: page %p is mapped", __func__, m[i])); K> > + KASSERT(m[i]->valid == 0 || i == count - 1, K> > + ("%s: request for a valid page %p", __func__, m[i])); K> Are you sure that e.g. the requested page cannot be partially valid ? K> I am surprised. All save the last one are always valid. This passed stress2 for me and pho@. K> > + KASSERT(m[i]->dirty == 0, K> > + ("%s: page %p is dirty", __func__, m[i])); K> > + KASSERT(m[i]->object == object, K> > + ("%s: wrong object %p/%p", __func__, object, m[i]->object)); K> > + } K> > +#endif K> Create a helper function with the loop, and use it, instead of copying K> the block twice. Done. K> > + K> > + r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); K> > + if (r != VM_PAGER_OK) K> > + return (r); K> > + K> > + /* K> > + * If pager has replaced the page, assert that it had K> > + * updated the array. K> > + */ K> > + KASSERT(m[reqpage] == vm_page_lookup(object, m[reqpage]->pindex), K> > + ("%s: mismatch page %p pindex %ju", __func__, K> > + m[reqpage], (uintmax_t )m[reqpage]->pindex - 1)); K> Why -1 ? K> Need an empty line after assert, to deliniate the code which is commented K> about. That's an artifact from future patch. Fixed. K> Also, you may assert that the requested page is still busied. Done. Updated patch attached. -- Totus tuus, Glebius. --NKtYx2Ppz7d1tORf Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="pagers.step.2.diff" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 284504) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -5734,8 +5734,6 @@ zfs_getpages(struct vnode *vp, vm_page_t *m, int c object = mreq->object; error = 0; - KASSERT(vp->v_object == object, ("mismatching object")); - if (pcount > 1 && zp->z_blksz > PAGESIZE) { startoff = rounddown(IDX_TO_OFF(mreq->pindex), zp->z_blksz); reqstart = OFF_TO_IDX(round_page(startoff)); Index: sys/fs/nfsclient/nfs_clbio.c =================================================================== --- sys/fs/nfsclient/nfs_clbio.c (revision 284504) +++ sys/fs/nfsclient/nfs_clbio.c (working copy) @@ -129,12 +129,6 @@ ncl_getpages(struct vop_getpages_args *ap) npages = btoc(count); /* - * Since the caller has busied the requested page, that page's valid - * field will not be changed by other threads. - */ - vm_page_assert_xbusied(pages[ap->a_reqpage]); - - /* * If the requested page is partially valid, just return it and * allow the pager to zero-out the blanks. Partially valid pages * can only occur at the file EOF. Index: sys/vm/swap_pager.c =================================================================== --- sys/vm/swap_pager.c (revision 284504) +++ sys/vm/swap_pager.c (working copy) @@ -1118,10 +1118,6 @@ swap_pager_getpages(vm_object_t object, vm_page_t mreq = m[reqpage]; - KASSERT(mreq->object == object, - ("swap_pager_getpages: object mismatch %p/%p", - object, mreq->object)); - /* * Calculate range to retrieve. The pages have already been assigned * their swapblks. We require a *contiguous* range but we know it to Index: sys/vm/vm_pager.c =================================================================== --- sys/vm/vm_pager.c (revision 284504) +++ sys/vm/vm_pager.c (working copy) @@ -250,8 +250,79 @@ vm_pager_deallocate(object) (*pagertab[object->type]->pgo_dealloc) (object); } +static void +vm_pager_assert_in(vm_object_t object, vm_page_t *m, int count) +{ +#ifdef INVARIANTS + + VM_OBJECT_ASSERT_WLOCKED(object); + KASSERT(count > 0, ("%s: 0 count", __func__)); + /* + * All pages must be busied, not mapped, not valid (save the last one), + * not dirty and belong to the proper object. + */ + for (int i = 0 ; i < count; i++) { + vm_page_assert_xbusied(m[i]); + KASSERT(!pmap_page_is_mapped(m[i]), + ("%s: page %p is mapped", __func__, m[i])); + KASSERT(m[i]->valid == 0 || i == count - 1, + ("%s: request for a valid page %p", __func__, m[i])); + KASSERT(m[i]->dirty == 0, + ("%s: page %p is dirty", __func__, m[i])); + KASSERT(m[i]->object == object, + ("%s: wrong object %p/%p", __func__, object, m[i]->object)); + } +#endif +} + /* - * vm_pager_get_pages() - inline, see vm/vm_pager.h + * Retrieve pages from the VM system in order to map them into an object + * ( or into VM space somewhere ). If the pagein was successful, we + * must fully validate it. + */ +int +vm_pager_get_pages(vm_object_t object, vm_page_t *m, int count, int reqpage) +{ + int r; + + vm_pager_assert_in(object, m, count); + + r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); + if (r != VM_PAGER_OK) + return (r); + + /* + * If pager has replaced the page, assert that it had + * updated the array. Also assert that page is still + * busied. + */ + KASSERT(m[reqpage] == vm_page_lookup(object, m[reqpage]->pindex), + ("%s: mismatch page %p pindex %ju", __func__, + m[reqpage], (uintmax_t )m[reqpage]->pindex)); + vm_page_assert_xbusied(m[reqpage]); + + /* + * Pager didn't fill up entire page. Zero out + * partially filled data. + */ + if (m[reqpage]->valid != VM_PAGE_BITS_ALL) + vm_page_zero_invalid(m[reqpage], TRUE); + + return (VM_PAGER_OK); +} + +int +vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, + int reqpage, pgo_getpages_iodone_t iodone, void *arg) +{ + + vm_pager_assert_in(object, m, count); + + return ((*pagertab[object->type]->pgo_getpages_async)(object, m, + count, reqpage, iodone, arg)); +} + +/* * vm_pager_put_pages() - inline, see vm/vm_pager.h * vm_pager_has_page() - inline, see vm/vm_pager.h */ Index: sys/vm/vm_pager.h =================================================================== --- sys/vm/vm_pager.h (revision 284504) +++ sys/vm/vm_pager.h (working copy) @@ -106,9 +106,9 @@ vm_object_t vm_pager_allocate(objtype_t, void *, v vm_ooffset_t, struct ucred *); void vm_pager_bufferinit(void); void vm_pager_deallocate(vm_object_t); -static __inline int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); -static inline int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, - int, pgo_getpages_iodone_t, void *); +int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); +int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, int, + pgo_getpages_iodone_t, void *); static __inline boolean_t vm_pager_has_page(vm_object_t, vm_pindex_t, int *, int *); void vm_pager_init(void); vm_object_t vm_pager_object_lookup(struct pagerlst *, void *); @@ -115,40 +115,6 @@ vm_object_t vm_pager_object_lookup(struct pagerlst void vm_pager_free_nonreq(vm_object_t object, vm_page_t ma[], int reqpage, int npages, boolean_t object_locked); -/* - * vm_page_get_pages: - * - * Retrieve pages from the VM system in order to map them into an object - * ( or into VM space somewhere ). If the pagein was successful, we - * must fully validate it. - */ -static __inline int -vm_pager_get_pages( - vm_object_t object, - vm_page_t *m, - int count, - int reqpage -) { - int r; - - VM_OBJECT_ASSERT_WLOCKED(object); - r = (*pagertab[object->type]->pgo_getpages)(object, m, count, reqpage); - if (r == VM_PAGER_OK && m[reqpage]->valid != VM_PAGE_BITS_ALL) { - vm_page_zero_invalid(m[reqpage], TRUE); - } - return (r); -} - -static inline int -vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, - int reqpage, pgo_getpages_iodone_t iodone, void *arg) -{ - - VM_OBJECT_ASSERT_WLOCKED(object); - return ((*pagertab[object->type]->pgo_getpages_async)(object, m, - count, reqpage, iodone, arg)); -} - static __inline void vm_pager_put_pages( vm_object_t object, --NKtYx2Ppz7d1tORf-- From owner-freebsd-arch@FreeBSD.ORG Thu Jun 18 12:34:21 2015 Return-Path: Delivered-To: arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D978798; Thu, 18 Jun 2015 12:34:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B5D72CCD; Thu, 18 Jun 2015 12:34:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t5ICYFRO077295 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 18 Jun 2015 15:34:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t5ICYFRO077295 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t5ICYEse077294; Thu, 18 Jun 2015 15:34:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 18 Jun 2015 15:34:14 +0300 From: Konstantin Belousov To: Gleb Smirnoff Cc: Alan Cox , arch@FreeBSD.org, rmacklem@freebsd.org Subject: Re: Step 2 Was: more strict KPI for vm_pager_get_pages() Message-ID: <20150618123414.GG2080@kib.kiev.ua> References: <20150615212931.GG73119@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150615212931.GG73119@glebius.int.ru> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 12:34:21 -0000 On Tue, Jun 16, 2015 at 12:29:31AM +0300, Gleb Smirnoff wrote: > Hi! > > This is step 2 of the "more strict pager KPI" patch: > > o Uninline vm_pager_get_pages() into vm_pager.c. > o Keep all KASSERTs that enforce the KPI of pagers and of their > consumers in one place: vm_pager_get_pages(). > o Keep the code that zeroes out partially valid pages in one > place in vm_pager_get_pages(). > I added Rick to Cc:, since there is something which I do not quite understand in the NFS client code. According to NFS v3 RFC, server may reply with the short read for a read RPC. In case the EOF flag is not set in the reply, this means that the data is available, and it is a transient server condition that reply was truncated. Do we handle the short reads ? I see the code in nfs read path which zeroes the absent parts of the buffer. Similarly, there is a code to zero the unread part of the page after VOP_GETPAGES(). But couldn't this result in the zeros instead of real data after the short read ? At least, I was not able to find code which would retry after reply without EOF. From owner-freebsd-arch@FreeBSD.ORG Fri Jun 19 13:08:22 2015 Return-Path: Delivered-To: freebsd-arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20D2C824; Fri, 19 Jun 2015 13:08:22 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ig0-x229.google.com (mail-ig0-x229.google.com [IPv6:2607:f8b0:4001:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E21A6333; Fri, 19 Jun 2015 13:08:21 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by igblz2 with SMTP id lz2so12655168igb.1; Fri, 19 Jun 2015 06:08:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=QVnMduCi2aMFsgoZZE/gaPNVR57nBYDO7KG+oVdQJsI=; b=wqFE+rm9alPPx0LqRKVfQr6I/FoE96otWWyEXqJPbbj4GFmdbGsguCKUC1TBZ9EVpf QXRt4l/5+vrvZHivEa9+cxDtmBspvfpLq7PC0jcl8lUdVMyOLuDZnA2hItXYefoRT4fC cZnkRdOLaPsD/nh/kieG8xUW68G73e2MeLdmnk0COjETBTzET+A74/7phAvEWN8yhGS7 lSkunz1MDt+AJXHZD9zMYzFS97QvsceB5TvdQ3o9SeBfe52v17jfUnwqDPYLEKpsd8JJ q0TU2qT8EbgREZ5btKK1SJqqv7OZuoLyuliJsLPJiY0ilD/A5Y7X55L2jKKrwQ65FmkZ crpg== MIME-Version: 1.0 X-Received: by 10.107.164.70 with SMTP id n67mr21849928ioe.8.1434719301325; Fri, 19 Jun 2015 06:08:21 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.38.133 with HTTP; Fri, 19 Jun 2015 06:08:21 -0700 (PDT) Date: Fri, 19 Jun 2015 06:08:21 -0700 X-Google-Sender-Auth: mTYnbIiBUPm5KAp3R711ZO_n5ys Message-ID: Subject: stale .depend during -HEAD builds? From: Adrian Chadd To: "Simon J. Gerraty" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jun 2015 13:08:22 -0000 Hi Simon/others, Is this a fallout from the meta mode changes? make[4]: /home/adrian/work/freebsd/head-embedded-2/src/../obj/mips//mips.mips/usr/home/adrian/work/freebsd/head-embedded-2/src/usr.sbin/praliases/.depend, 78: ignoring stale .depend for /home/adrian/work/freebsd/head-embedded-2/src/../obj/mips//mips.mips/usr/home/adrian/work/freebsd/head-embedded-2/src/lib/libsmdb/libsmutil.a -a From owner-freebsd-arch@FreeBSD.ORG Fri Jun 19 15:36:34 2015 Return-Path: Delivered-To: freebsd-arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1EB152E; Fri, 19 Jun 2015 15:36:34 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0117.outbound.protection.outlook.com [65.55.169.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18F71EA0; Fri, 19 Jun 2015 15:36:33 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from BL2PR05CA0023.namprd05.prod.outlook.com (10.255.226.23) by BY2PR05MB775.namprd05.prod.outlook.com (10.141.224.152) with Microsoft SMTP Server (TLS) id 15.1.195.15; Fri, 19 Jun 2015 15:21:50 +0000 Received: from BN1BFFO11OLC003.protection.gbl (2a01:111:f400:7c10::1:116) by BL2PR05CA0023.outlook.office365.com (2a01:111:e400:c04::23) with Microsoft SMTP Server (TLS) id 15.1.195.15 via Frontend Transport; Fri, 19 Jun 2015 15:21:50 +0000 Authentication-Results: spf=softfail (sender IP is 66.129.239.19) smtp.mailfrom=juniper.net; freebsd.org; dkim=none (message not signed) header.d=none; Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.19 as permitted sender) Received: from P-EMF02-SAC.jnpr.net (66.129.239.19) by BN1BFFO11OLC003.mail.protection.outlook.com (10.58.145.14) with Microsoft SMTP Server (TLS) id 15.1.190.9 via Frontend Transport; Fri, 19 Jun 2015 15:21:49 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Fri, 19 Jun 2015 08:21:48 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.21.16.28]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id t5JFLlD90958; Fri, 19 Jun 2015 08:21:47 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 41788580AA; Fri, 19 Jun 2015 08:21:47 -0700 (PDT) To: Adrian Chadd CC: "freebsd-arch@freebsd.org" Subject: Re: stale .depend during -HEAD builds? In-Reply-To: References: Comments: In-reply-to: Adrian Chadd message dated "Fri, 19 Jun 2015 06:08:21 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 8.0.3; nmh 1.3; GNU Emacs 22.3.1 Date: Fri, 19 Jun 2015 08:21:47 -0700 Message-ID: <19927.1434727307@chaos> MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BN1BFFO11OLC003; 1:4XFZFQKhFMcoMMskHx7j+VgiKo6wZsQLF2sSZ5HSmyKoUXb1aMy+cSjVBXQCfT2OjtkuWoF8XUeDE2DhyAHnp507XDp0ZDATnoKuofRtCOMOj+SnyTEMusMupzyoVt+QT7NZqWRap8RLXmeyiAMsWws9Dcq6KhHlZM858EpyoN47mEFLblSWH+CUK7luD5GNmCJf+1KXlRLCtNDBdljzK3tpST1ggm/p+A32TGwrEWjNAweyFzWXpcWj6bg1c/SOmudQbVXuCHonBH+loXltRclw/Y0dVBkZOoGVPDuosFs= X-Forefront-Antispam-Report: CIP:66.129.239.19; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(2980300002)(52314003)(24454002)(189002)(199003)(110136002)(189998001)(33716001)(6806004)(117636001)(5001960100002)(19580405001)(19580395003)(50226001)(86362001)(50986999)(76176999)(87936001)(50466002)(77096005)(57986006)(76506005)(450100001)(77156002)(62966003)(47776003)(106466001)(46102003)(92566002)(2950100001)(105596002)(48376002)(142923001)(62816006)(42262002); DIR:OUT; SFP:1102; SCL:1; SRVR:BY2PR05MB775; H:P-EMF02-SAC.jnpr.net; FPR:; SPF:SoftFail; MLV:ovrnspm; MX:1; A:1; PTR:InfoDomainNonexistent; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2PR05MB775; 2:Z3kf0eXza5adIp6xRl+AYye/srzyHct7+yBIyao/2HtBUNnqqkX9TDnoqhfI/0zX; 3:gOGbKENGh9AQVSmcSmNHMPHj2U1c0aCaNLXl6Y0H+KykBukY/OeHk77OOSTaD7wwpthRmD31KBP1gA7P6yT5jq51toy8KTB9dTerGxXlqnS5xOtdkrCIGk+OOylfDnu4umcnvV3IF92h92Xjd3nSrf9QYLyemnHjy6Fo/XkQxF2m7q5NAIcpjFbytz2YmqxGBVXCOJqRzKaoq7d+uT3XDogt1JtRE0hQr+yhKz/yQqY=; 20:NnUqGo4gb7YCa0GS2gHag2UqsCgUrFxNGlDm9OeL572bKMUE4mVZTiJ37M5/BnDoSlhtxgzRsbqLQHxjexVOgLDKr+CJR7hi7a076d/77rRtfUsZ6iS3WXafhfyE/8yChVoHToJCciEl0BWJ6auSlUUZ0ighhNPHtOCmCuXISkhVwAKuDyt6LKjElqXfruKpuXEQdhaPpUeZKlrQ4SvMpqi0/Yh7DYRPp0epgpI1/EXsmc9j+d0r3g7FZGJZjv39wFW/zK+iwDLCnGY+vnBgRlqtAnbZ4zy1Vfs+mykBZRFtY+Dm/4A2Xhm9iGTpT/85Ht0KhJSZR+DPYp7afFOTdG9H84v/fbS3W7KmJVT4Ma7qAalE81PghJw3ec9qBJ9ulGqdMSbTUNUk2F+WqSLeHXtTXWLTe2LaUyCnkRCMzHZ8uWZ6PbtTkMOeU6hnINluB6WxD/dxHdvYMidL3au/Y+YdMhLDJJ0DXbDfNzS6arbi+vR6a81cY8O4ELRkB/A2 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR05MB775; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(5005006)(3002001); SRVR:BY2PR05MB775; BCL:0; PCL:0; RULEID:; SRVR:BY2PR05MB775; X-Microsoft-Exchange-Diagnostics: 1; BY2PR05MB775; 4:HzikpmHYhFOrcCdkd907OtRlk5sPi7Kk4XdzotblmuEEM2Bw5dMMr9qT3s7Rqe57stBd9duUoz0FhtmO31BSkPPhh1Gx4PV3krp6om7dAe+Z6Wuz1eYp72N4NjPBlAT6Xflc9qX6n0iyHQW3+MWls61DkITJrW1qkhwhB/v9W7DYuPsuco/Rd3fBaiYvGdvmwHxN6tHDojLymYNQaR9SErqzbVcOHOEk1q7Awpvsd3Gj8jjke1jn+NN0bYujlVwBJadw6Ncgss37cH/iR/F+omm+3JyE/YdjNM7pXSWe3CE= X-Forefront-PRVS: 0612E553B4 X-Microsoft-Exchange-Diagnostics: 1; BY2PR05MB775; 23:Jm+hk+xCJhZjmtvtt3qaQtvyDXNA6LYmeQF+CfLsrWBAIv/kBGUVJkw2LzIVDuoWHNkJO6/XNSX7Vx8TOA37bc9CtzYWymgP/wGHDsEIySabmkNgXY0iQzfVczuJVd/piyEcMoCpV5+L13GvMSQJECybonhhHlOT+T80SFU/kct9DTuwdELnzV8Xq9T+ZfqbIkISc2Bj3BIzr8yb5k//zQsUPYMMNh90SsFjzdKZvtguraRiE6ZfIQX0WbnxALn9SQz8NoY11CpZOLIzciiyocvVGgWtug+tLYE55gw7gX8c3POoV3+y9swK7kpoYJngE4TNNLGZ9Yc8gYTi7/N6demTTmmMoIo5omgBRGt3e34JjdVpLoh9M7X5XNq0dH+2hZJ4fSUkF0rAfS4t3LWx/Q6e8sHwlDHcz2Gr+IQw4uVoGN/Ktv5N24yt71PBqgIfYvGg3ixUOdYA7q0HqrhmxL9BHcnes2LZGduKnp490GzOTtKiHYM3hCSkw4WgpqaLA7nWVmYQTB8oPjo25liKa5s+gNu3d4KhmFpq4+1vQV17aWm4ZVvqELSQYa6yrr0cfJcaBQAVU1xa1Gta2Zbfy8596ru6gN6srEcqn2FL4n4gWFdKSrspSYPACxHl9HpmqFQ1u5Zr/E3xFsbiqxodUOvnbTJa3+WQzInIKlLzOiSDkOe2ELImv2TBwTEMyL2Q82kX8kiy0n4LfpBiMgXndkLw7VpsSh4ukTAp3w8DPwuD8qpYJv2VTkSE7exuOYah74/r5BjNltX02UIT3B5zeVcepm5f6HXVJk6F9qxQewtnr53rWB390gUXX3z/JEEqkSAxhMrpoJzeLk6p2bEvhNXcPeM49X/D1bs4qkuY9wnzb7Itrb0KebGnQbDTPijeRdiYDSo/Ro0TG/xrLiI4bu+quIFfNQkCegPrk020Url1z9kDAlvzU5AtJtRP1fcd X-Microsoft-Exchange-Diagnostics: 1; BY2PR05MB775; 5:ndmkrJkVKFB6j7AV86aX5EzxqS9GRDQ+nMwaG4ktwxeyxoDHZSgNm1rav68iMJLS/UOujJiXRP/6RWX47tZGUllDBnAVXrLZ3V4BIS3x2c0hP8htTm1zJzp9OxJoVMMotxaTxgjxVVk7BUsKMEnaQw==; 24:6hGz41PKejvqtI7FRP4xxbc7hdVeCl0/OhhPxGxNkl/K6Pd94AaB7alOqgtlLetGtVA9CaaJL+KbRy9v2YOWqTMgK7GGal3k7aW0/oph9Yg=; 20:1fitkighuM4a1xlY5xSP84fsVF0P8n3NYErU2vzmakCpuW9BKYVaZ5vagSPYJxtC6lCKhMq8AgQHPbJySlzlyg== X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2015 15:21:49.7232 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.19]; Helo=[P-EMF02-SAC.jnpr.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR05MB775 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jun 2015 15:36:34 -0000 Adrian Chadd wrote: > Is this a fallout from the meta mode changes? Without more detail, its hard to be certain. > make[4]: /home/adrian/work/freebsd/head-embedded-2/src/../obj/mips//mips.mips/usr/home/adrian/work/freebsd/head-embedded-2/src/usr.sbin/praliases/.depend, > 78: ignoring stale .depend for > /home/adrian/work/freebsd/head-embedded-2/src/../obj/mips//mips.mips/usr/home/adrian/work/freebsd/head-embedded-2/src/lib/libsmdb/libsmutil.a > Is that what your objdirs normally look like? The message from make is standard bmake behavior. An unresolved and unresolvable dependency learned from .depend is ignored, in case it is just stale data. If it really is needed the build will fail anyway, but in 90% of cases it is just stale data and the build sails happily on. Normally you would see this sort of thing after some sort of tree re-org (eg things getting renamed or removed). Very handy for update builds, especially if doing any form of auto depend (which usually means depends lag by a build) From owner-freebsd-arch@FreeBSD.ORG Fri Jun 19 15:39:44 2015 Return-Path: Delivered-To: freebsd-arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 898226D2 for ; Fri, 19 Jun 2015 15:39:44 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [IPv6:2607:f8b0:4001:c03::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 54944ED1 for ; Fri, 19 Jun 2015 15:39:44 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by iecrd14 with SMTP id rd14so77300808iec.3 for ; Fri, 19 Jun 2015 08:39:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5U2QisFp73xld5ErvjKqlY5tLs3kUcRjRoji40vty88=; b=DWY1HCFvh5BGnqkeapGRLFM30fz2YnZmli7RButSyjIL+32JicwOOrydUsKrXDx1kl DJORslxyNW7u3JDvckrboy2zGulsgkPx8vL6R6k1A8Jtx7t4p7mNRP34Oti6OGmcISgo 17sKeLaZsfh0kkkPgyfPI9eWrFYk3rQJHy/m4+noRW2cy1HQWRAkTTemCyLhS6NNXPHB w4RTUdYj2Blgddf4F00h/Z5F4fyx3T7AGvuN/GiEjFykJMc6PF9+mMi1CDtCkuIpDiuV IUnZpT4RcJ6N4GJUjsEd5mHjYA27myRUGuXBxHawyDlvcz+CugjYyjXKLk777zKHOY2u H4UA== MIME-Version: 1.0 X-Received: by 10.42.176.8 with SMTP id bc8mr12515183icb.22.1434728383692; Fri, 19 Jun 2015 08:39:43 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.38.133 with HTTP; Fri, 19 Jun 2015 08:39:43 -0700 (PDT) In-Reply-To: <19927.1434727307@chaos> References: <19927.1434727307@chaos> Date: Fri, 19 Jun 2015 08:39:43 -0700 X-Google-Sender-Auth: ygbJ7hCrJLN_WhMfWx0SdeZsIME Message-ID: Subject: Re: stale .depend during -HEAD builds? From: Adrian Chadd To: "Simon J. Gerraty" Cc: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jun 2015 15:39:44 -0000 On 19 June 2015 at 08:21, Simon J. Gerraty wrote: > Adrian Chadd wrote: >> Is this a fallout from the meta mode changes? > > Without more detail, its hard to be certain. > >> make[4]: /home/adrian/work/freebsd/head-embedded-2/src/../obj/mips//mips.mips/usr/home/adrian/work/freebsd/head-embedded-2/src/usr.sbin/praliases/.depend, >> 78: ignoring stale .depend for >> /home/adrian/work/freebsd/head-embedded-2/src/../obj/mips//mips.mips/usr/home/adrian/work/freebsd/head-embedded-2/src/lib/libsmdb/libsmutil.a >> > > Is that what your objdirs normally look like? Yup: github.com/freebsd/freebsd-wifi-build You can use that to generate MIPS AP images using freebsd-head ,and the obj paths are relative. > > The message from make is standard bmake behavior. > An unresolved and unresolvable dependency learned from .depend > is ignored, in case it is just stale data. > If it really is needed the build will fail anyway, but in 90% of cases > it is just stale data and the build sails happily on. > > Normally you would see this sort of thing after some sort of tree re-org > (eg things getting renamed or removed). > > Very handy for update builds, especially if doing any form of auto > depend (which usually means depends lag by a build) Ok. I just didn't see it until I updated my local MIPS tree to include the meta make stuf, (and did kill obj/root.) -a