From owner-freebsd-hackers@freebsd.org Thu Apr 13 04:59:37 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 402ECD3BBA7 for ; Thu, 13 Apr 2017 04:59:37 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-43.reflexion.net [208.70.210.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0296680C for ; Thu, 13 Apr 2017 04:59:36 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 18806 invoked from network); 13 Apr 2017 04:59:34 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 13 Apr 2017 04:59:34 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Thu, 13 Apr 2017 00:59:34 -0400 (EDT) Received: (qmail 15295 invoked from network); 13 Apr 2017 04:59:34 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 13 Apr 2017 04:59:34 -0000 Received: from [192.168.1.106] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 65DFFEC8B66; Wed, 12 Apr 2017 21:59:33 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: The arm64 fork-then-swap-out-then-swap-in failures: a program source for exploring them From: Mark Millard In-Reply-To: <7adada71-e089-e105-eec8-6136d4b8c083@bsd.com.br> Date: Wed, 12 Apr 2017 21:59:32 -0700 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <4DEA2D76-9F27-426D-A8D2-F07B16575FB9@dsl-only.net> <163B37B0-55D6-498E-8F52-9A95C036CDFA@dsl-only.net> <08E7A5B0-8707-4479-9D7A-272C427FF643@dsl-only.net> <20170409122715.GF1788@kib.kiev.ua> <9D152170-5F19-47A2-A06A-66F83CA88A09@dsl-only.net> <9DCAF95B-39A5-4346-88FC-6AFDEE8CF9BB@dsl-only.net> <8FFE95AA-DB40-4D1E-A103-4BA9FCC6EDEE@dsl-only.net> <89D6D677-3BE2-45E2-A902-CC6A0305F3F9@dsl-only.net> <585B43F7-D4C8-431A-BFFE-68B48C3214AE@dsl-only.net> <876EA1E4-E5A9-411C-AFFD-989713037C19@dsl-only.net> <7adada71-e089-e105-eec8-6136d4b8c083@bsd.com.br> To: =?utf-8?B?T3RhY8OtbGlv?= X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Apr 2017 04:59:37 -0000 On 2017-Apr-12, at 6:25 PM, Otac=C3=ADlio = wrote: > Em 10/04/2017 17:15, Mark Millard escreveu: >> On 2017-Apr-10, at 2:51 AM, Mark Millard = wrote: >>=20 >>> On 2017-Apr-9, at 5:10 PM, Mark Millard = wrote: >>>=20 >>>> On 2017-Apr-9, at 10:24 AM, Mark Millard = wrote: >>>>=20 >>>>> On 2017-Apr-9, at 5:27 AM, Konstantin Belousov = wrote: >>>>>> Hmm, could you try the following patch, I did not even compiled = it. >>>>> I'll try it later today. >>>>>=20 >>>>>> diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c >>>>>> index 3d5756ba891..55aa402eb1c 100644 >>>>>> --- a/sys/arm64/arm64/pmap.c >>>>>> +++ b/sys/arm64/arm64/pmap.c >>>>>> @@ -2481,6 +2481,11 @@ pmap_protect(pmap_t pmap, vm_offset_t sva, = vm_offset_t eva, vm_prot_t prot) >>>>>> sva +=3D L3_SIZE) { >>>>>> l3 =3D pmap_load(l3p); >>>>>> if (pmap_l3_valid(l3)) { >>>>>> + if ((l3 & ATTR_SW_MANAGED) && >>>>>> + pmap_page_dirty(l3)) { >>>>>> + = vm_page_dirty(PHYS_TO_VM_PAGE(l3 & >>>>>> + ~ATTR_MASK)); >>>>>> + } >>>>>> pmap_set(l3p, ATTR_AP(ATTR_AP_RO)); >>>>>> PTE_SYNC(l3p); >>>>>> /* XXX: Use pmap_invalidate_range */ >>>>=20 >>>> Preliminary testing indicates that this fixes the >>>> some-pages-become-zero problem for fork-then-swapout/in. >>>>=20 >>>> Thanks! >>>>=20 >>>> I'll see if a buildworld can go through without being stopped >>>> by the type of issue. But that will take a while. (It is how >>>> I originally ran into the problem(s) that others had been >>>> reporting on the lists.) >>> buildworld buildkernel completed non-stop for the first time >>> on a BPI-M3 board. >> I had been thinking of the BPI-M3 for other reasons >> and typed that instead of the correct: Pine64+ 2GB. >> (True elsewhere as well.) I do really mean arm64 >> here, not armv7. >>=20 >>> Looks good for a check-in to svn to me (head and stable/11). >>>=20 >>> This combined with 2017-Feb-15's -r313772's fix to the fork >>> trampline code's updating of sp_el0 makes arm64 far more stable >>> for my purposes. >>>=20 >>> -r313772 was never MFC'd to stable/11. In my view it should be. >> =3D=3D=3D >> Mark Millard >> markmi at dsl-only.net >>=20 > Dears >=20 > Will this patch be committed to HEAD? It was: Author: kib Date: Mon Apr 10 15:32:26 2017 New Revision: 316679 URL:=20 https://svnweb.freebsd.org/changeset/base/316679 Log: Do not lose dirty bits for removing PROT_WRITE on arm64. =20 Arm64 pmap interprets accessed writable ptes as modified, since ARMv8.0 does not track Dirty Bit Modifier in hardware. If writable bit is removed, page must be marked as dirty for MI VM. =20 This change is most important for COW, where fork caused losing content of the dirty pages which were not yet scanned by pagedaemon. =20 Reviewed by: alc, andrew Reported and tested by: Mark Millard PR: 217138, 217239 Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Modified: head/sys/arm64/arm64/pmap.c Modified: head/sys/arm64/arm64/pmap.c = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D --- head/sys/arm64/arm64/pmap.c Mon Apr 10 12:35:58 2017 = (r316678) +++ head/sys/arm64/arm64/pmap.c Mon Apr 10 15:32:26 2017 = (r316679) @@ -2481,6 +2481,11 @@ pmap_protect(pmap_t pmap, vm_offset_t sv sva +=3D L3_SIZE) { l3 =3D pmap_load(l3p); if (pmap_l3_valid(l3)) { + if ((l3 & ATTR_SW_MANAGED) && + pmap_page_dirty(l3)) { + vm_page_dirty(PHYS_TO_VM_PAGE(l3 = & + ~ATTR_MASK)); + } pmap_set(l3p, ATTR_AP(ATTR_AP_RO)); PTE_SYNC(l3p); /* XXX: Use pmap_invalidate_range */ There was a patch ( -r313772 ) committed to head back in Feb. for interrupts sometimes trashing a special register during fork. It takes both of these patches to get fork working reliably. [stable/11 should eventually get both of these patches so that fork becomes reliable there for aarch64 (armv8.0).] =3D=3D=3D Mark Millard markmi at dsl-only.net