From owner-freebsd-hackers@freebsd.org Sun Apr 30 19:22:18 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7905DD57C2D for ; Sun, 30 Apr 2017 19:22:18 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "m5p.com", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B0AC1308 for ; Sun, 30 Apr 2017 19:22:18 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:418:3fd::1f] (haymarket.m5p.com [IPv6:2001:418:3fd::1f]) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTP id v3UJM9ai019484 for ; Sun, 30 Apr 2017 15:22:15 -0400 (EDT) (envelope-from george+freebsd@m5p.com) To: freebsd-hackers@FreeBSD.org From: George Mitchell Subject: TCP6 problem Message-ID: <82600781-7892-63ea-feac-35a140dbf95d@m5p.com> Date: Sun, 30 Apr 2017 15:22:03 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="B08votonULj2vlPOBBV9bvomAuf8BMFXi" X-Spam-Status: No, score=-1.0 required=10.0 tests=ALL_TRUSTED, RP_MATCHES_RCVD autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mattapan.m5p.com X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.1 (mailhost.m5p.com [IPv6:2001:418:3fd::f7]); Sun, 30 Apr 2017 15:22:16 -0400 (EDT) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Apr 2017 19:22:18 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --B08votonULj2vlPOBBV9bvomAuf8BMFXi Content-Type: multipart/mixed; boundary="vhFB3q1MLd9gbWQ85chcSTWwJ3aRbxQim"; protected-headers="v1" From: George Mitchell To: freebsd-hackers@FreeBSD.org Message-ID: <82600781-7892-63ea-feac-35a140dbf95d@m5p.com> Subject: TCP6 problem --vhFB3q1MLd9gbWQ85chcSTWwJ3aRbxQim Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable In certain cases, TCP6 stopped working between machines on my local net and the outside world when I updated to 10.3-RELEASE-p18 on my network. Here is my network setup: +--------------+ +---------------+ | sullivan | +--------+ | haymarket | | 6-core | | | | 8-core | | amd64 alc0 +------+ | switch +-------+ re0 amd64 | +--------------+ | | 2 | +---------------+ | +--+ +---+ +--------------+ | | | | | +---------------+ | scollay | | | +--------+ | | pi Raspberry | | 1-core | | | | | Pi (orig.) | | amd64 nfe0 +----+ | | +---+ ue0 arm | +--------------+ | | | +--------+ +---------------+ | | | | | +--------------+ | | +--+ switch | +---------------+ | parkstreet | | +----+ 1 | | mattapan | | 1-core | +------+ | | 1-core amd64 | | and64 nfe0 +-----------+ +-------+ alc0 gif0 +---IPv6 +--------------+ +------- + +---------------+ world Throughout this process, pi has been running 10.3-RELEASE and has not had any problems. All the other machines were running 10.3-RELEASE-p13 up until April 17, when I upgraded them to 10.3-RELEASE-p18. At this point, TCP6 still worked normally, except on sullivan and scollay. Even sullivan and scollay can perform TCP6 to other local machines, but it fails in the following manner when attempted to the outside world, as observed using tcpdump on the alc0 interface of mattapan and either the alc0 interface of sullivan or the nfe0 interface of scollay: sullivan (or scollay) sends SYN. outside machine replies with SYN/ACK, and sullivan (or scollay) receives the SYN/ACK. sullivan (or scollay) replies with ACK (as seen on sullivan or scollay), but the ACK is not seen at mattapan alc0. sullivan (or scollay) fruitlessly retransmits the ACK, but it is never seen at mattapan. mattapan is using pf filtering on gif0 (which physically is on the re0 interface), but no other machines are filtering packets. My best guess at the moment is that somehow tcpdump is lying to me about the captured packets, because I rebooted mattapan and sullivan with my /boot/kernel.old and it did not fix the problem. Any clues? -- George --vhFB3q1MLd9gbWQ85chcSTWwJ3aRbxQim-- --B08votonULj2vlPOBBV9bvomAuf8BMFXi Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAlkGOWEACgkQwRES3m+p 4fl8Xg/+Lya6VmXDhzlkXeDeaPEK5saQmAvLeVACrbyuYB4vAoKPjyxW1/i9Hraa N8XMF853j4GYz5Iwb++t6YYnwfcDL/ePFqam8z8xZusInD2MsuSb4gdg2NLxFljt rOHp1Q8xLEYFTll6g6ArZnyPK/53njwvrKcaSTDI1dsu/x6rUvuEH2vUDy+2BDZZ KORSGfPrgzrgpPXXNS2IFCKmwhaYBCKBFgoitsaqKd/O/1/jn655UqELPNXPRn/m 59/QbFYQB8lJxEnnlzX6JtJfe+oMl2PEr2dq6wmyy6NFfaBJovjH8aWbLAE6y8B7 nXQFKDbI/bR1Fe+z9bc1q9sSO7sosugy0fv2zIr+UyoqMK3blmTmJSUo53zH72YU DBHLCDf5Yogo+5FDIIKjPN+me9PVLqPVRE0AMutptbTwW3uA1eo+AkaELT6TWKVJ xCLoWxsXeU19M1LoOZWQEq8mLMTAaldg2QRj9kk3WJt1O4DghAPGG5cQnzwifaZE dNRY8HBxTssFSVowY/G3qCUPZduhwWNX9nynBhjiNQlqy34p+eBq047E/hHrTdxM +Sk+JKvB2nCV1R8R/0TZKpKERuhJcyEWTPFCJwiNdQG3+pzjbZD99t7cKSOyxC6E 08H0NTid0S+jdTICE4QSTJVBg79/vBtKuiBqmsZa6awKeSCJPyM= =A1fS -----END PGP SIGNATURE----- --B08votonULj2vlPOBBV9bvomAuf8BMFXi-- From owner-freebsd-hackers@freebsd.org Sun Apr 30 22:40:07 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05814CA1D75 for ; Sun, 30 Apr 2017 22:40:07 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 997231811 for ; Sun, 30 Apr 2017 22:40:05 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from [194.32.164.16] ([194.32.164.16]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id v3UMdvNq066163; Sun, 30 Apr 2017 23:39:57 +0100 (BST) (envelope-from rb@gid.co.uk) Subject: Re: TCP6 problem Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_E8881701-E78B-43F0-8EF0-7962ECDD9496"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.6b2 From: Bob Bishop In-Reply-To: <82600781-7892-63ea-feac-35a140dbf95d@m5p.com> Date: Sun, 30 Apr 2017 23:39:49 +0100 Cc: freebsd-hackers@freebsd.org Message-Id: <1D3B8C36-6144-4040-8939-61EB933B5080@gid.co.uk> References: <82600781-7892-63ea-feac-35a140dbf95d@m5p.com> To: George Mitchell X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Apr 2017 22:40:07 -0000 --Apple-Mail=_E8881701-E78B-43F0-8EF0-7962ECDD9496 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Hi, > On 30 Apr 2017, at 20:22, George Mitchell wrote: > > In certain cases, TCP6 stopped working between machines on my local > net and the outside world when I updated to 10.3-RELEASE-p18 on my > network. [etc] Try a quick test without pf, if only to eliminate it. -- Bob Bishop rb@gid.co.uk --Apple-Mail=_E8881701-E78B-43F0-8EF0-7962ECDD9496 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlkGZ70ACgkQvMaT6aS3xb8YSQCgp8D8Qb1d2EdGLUMvjNK8HOGX cFQAn06pLDSxVq7mqCz3DIxgh+WR9aY/ =fN6O -----END PGP SIGNATURE----- --Apple-Mail=_E8881701-E78B-43F0-8EF0-7962ECDD9496-- From owner-freebsd-hackers@freebsd.org Sun Apr 30 23:09:45 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7211FCA2934 for ; Sun, 30 Apr 2017 23:09:45 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "m5p.com", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F0E68F1 for ; Sun, 30 Apr 2017 23:09:45 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:418:3fd::1f] (haymarket.m5p.com [IPv6:2001:418:3fd::1f]) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTP id v3UN9aTB001172; Sun, 30 Apr 2017 19:09:41 -0400 (EDT) (envelope-from george+freebsd@m5p.com) Subject: Re: TCP6 problem To: Bob Bishop References: <82600781-7892-63ea-feac-35a140dbf95d@m5p.com> <1D3B8C36-6144-4040-8939-61EB933B5080@gid.co.uk> Cc: freebsd-hackers@freebsd.org From: George Mitchell Message-ID: <4438fbf6-e7db-fa6b-4f6a-96796c21aeb4@m5p.com> Date: Sun, 30 Apr 2017 19:09:30 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <1D3B8C36-6144-4040-8939-61EB933B5080@gid.co.uk> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="cuqn6m8QFLknlXBebXwrqIwCGjFe6cE9S" X-Spam-Status: No, score=-1.0 required=10.0 tests=ALL_TRUSTED, RP_MATCHES_RCVD autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mattapan.m5p.com X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.1 (mailhost.m5p.com [IPv6:2001:418:3fd::f7]); Sun, 30 Apr 2017 19:09:43 -0400 (EDT) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Apr 2017 23:09:45 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --cuqn6m8QFLknlXBebXwrqIwCGjFe6cE9S Content-Type: multipart/mixed; boundary="6q7aPDBshTwm9Lr1uaBOKp4jCK4pJeEfA"; protected-headers="v1" From: George Mitchell To: Bob Bishop Cc: freebsd-hackers@freebsd.org Message-ID: <4438fbf6-e7db-fa6b-4f6a-96796c21aeb4@m5p.com> Subject: Re: TCP6 problem References: <82600781-7892-63ea-feac-35a140dbf95d@m5p.com> <1D3B8C36-6144-4040-8939-61EB933B5080@gid.co.uk> In-Reply-To: <1D3B8C36-6144-4040-8939-61EB933B5080@gid.co.uk> --6q7aPDBshTwm9Lr1uaBOKp4jCK4pJeEfA Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 04/30/17 18:39, Bob Bishop wrote: > Hi, >=20 >> On 30 Apr 2017, at 20:22, George Mitchell wro= te: >> >> In certain cases, TCP6 stopped working between machines on my local >> net and the outside world when I updated to 10.3-RELEASE-p18 on my >> network. [etc] >=20 > Try a quick test without pf, if only to eliminate it. > [...] A quick naive test (service pf stop) seemed to just make things worse, so let me wait until a lower traffic time like tomorrow morning and reboot with "pf_enable" set false. -- George --6q7aPDBshTwm9Lr1uaBOKp4jCK4pJeEfA-- --cuqn6m8QFLknlXBebXwrqIwCGjFe6cE9S Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAlkGbrAACgkQwRES3m+p 4fmOwg/+IDpfbSiMfJfd+Yl8vIrwMS8o4zPEzmTSoeIIKh48WppmLEax4wLPb7kI /AXsw887sKDiQIQNsSyrbbNpAHZ1uk02XkBRd4vYXYSHS3xa6cOcASYoOCEkrI1d q1BdjZAas8qovSCNWyPg0H0et0JPMcz+HEtJYiK4TPENIU6BdBymmkp2yNsHh3/8 ZNi8mUzVnXzzY6IyvjMSVmjaq5M9A/hy92iaAGNXvHHTlwqgmJsDkeDYuPi5LNeY ktF+ulyi2wHsSon1Lsbc3ekBp0xqsY0rmbA92BfFnZDJjLD27Z1hLgzEYwp11Wq1 XADdnYcFijFPKMzilAqNvsQ2vB7mIUmlGLQKpKWuboZ1UrkT+TwOT64KAUS1HtMD RoVNRhu9A4dLsWpVuoTkM8ajz/Q+8QL/AQ/+xICwkwZ7yoanM8qiaKD/HbBhTv1K yCmM2AFA5D7HBCqaGz9YnkWFRGV4iKK/52t4CvnHx290Jj6EEAbrg/ZTJpteF1S8 6+v93+jb4+630/+KLcLXmBmpy84Kt2OGw6W9leuKB5YUw9tBh99LrVASg+hfMDjw xaFBvpD3ODbSc86iuKvQDDYmOMrEO2d1Mvr9ueqCALwP4xf5GxW2P6tcKfEBRkp3 AlndlRolkUoR5rvoW5xgYklT0usTl3ROK89xgqbH47x/iGpgJc8= =T5qk -----END PGP SIGNATURE----- --cuqn6m8QFLknlXBebXwrqIwCGjFe6cE9S-- From owner-freebsd-hackers@freebsd.org Mon May 1 01:31:07 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F3EED58717 for ; Mon, 1 May 2017 01:31:07 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-50.reflexion.net [208.70.210.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E816F11 for ; Mon, 1 May 2017 01:31:06 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 1176 invoked from network); 1 May 2017 01:34:20 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 1 May 2017 01:34:20 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Sun, 30 Apr 2017 21:31:05 -0400 (EDT) Received: (qmail 8158 invoked from network); 1 May 2017 01:31:05 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 1 May 2017 01:31:05 -0000 Received: from [192.168.1.106] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 66802EC8FFF; Sun, 30 Apr 2017 18:31:04 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: FYI: example "panic: ARM64TODO: reclaim_pv_chunk" on a Pine64+ 2GB with head -r317015 [mkimg okay; "cp -p" generates the bad context] Date: Sun, 30 Apr 2017 18:31:03 -0700 References: <4BC7E6BC-4BF9-4F5E-9851-E022AC9A3082@dsl-only.net> <51050A07-B951-45C0-82CE-73BB342F012E@dsl-only.net> <302D1255-4D34-4C1B-8F3A-9180A6AF8768@dsl-only.net> To: freebsd-arm , freebsd-hackers@freebsd.org, FreeBSD Current In-Reply-To: <302D1255-4D34-4C1B-8F3A-9180A6AF8768@dsl-only.net> Message-Id: <7C5C4725-984C-4461-B76C-9A6BE052BA00@dsl-only.net> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 May 2017 01:31:07 -0000 On 2017-Apr-27, at 10:26 PM, Mark Millard = wrote: > [As the text does not really follow from the earlier text > I'd sent directly I'm top posting a hypothesis about where > so much active memory came from to be so low in available > memory to get an reclaim_pv_chunk attempt.] >=20 > My hypothesis is that the "mdconfig -d"s from vm_copy_base > (from /usr/src/release/tools/vmimage.subr ) did not actually > release the memory resources involved (from vnode backed > mdconfig use): I watched with "vmstat 1" and "mdconfig -d" did release memory (including RAM) each time. > . . . > =46rom the prior top report for the failure, > partially repeated here: >=20 > . . . > Mem: 1618M Active, 17M Inact, 315M Wired, 204M Buf, 15M Free > Swap: 6144M Total, 34M Used, 6110M Free, 348K Out >=20 > PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND > 48988 root 4 31 0 651M 27048K 0K RUN 0 0:03 = 87.60% xz -T 0 = /usr/obj/DESTDIRs/vmimage-aarch64/vmimages/FreeBSD-12.0-CURRENT- > . . . >=20 > The combination 1618M Mem:Active but 34M Swap:Used > and 651M xz:SIZE but 27M xz:RES and 0K xz:SWAP just > seems very odd, like it should not happen. The 17M > Mem:Inact is odd for the context as well. (Mem:Wired, > Mem:Buf, and Mem:Free look normal.) >=20 > An alternate hypothesis would be the memory "leak" > is from mkimg not having it memory-use cleaned up. > This happens after vm_copy_base but before the cp/xz > sequence and is what produces vm.raw. mkimg also release its memory (including RAM) each time. But the later "cp -p" of the large vm.raw that I was producing ended up without leaving the free memory that is expected after it finished. I worked around the issue with (just a personal workaround that helps in other respects as well): Index: Makefile.vm =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- Makefile.vm (revision 317015) +++ Makefile.vm (working copy) @@ -119,15 +119,20 @@ vm-install: .if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) mkdir -p ${DESTDIR}/vmimages -. for FORMAT in ${VMFORMATS} - cp -p ${VMBASE}.${FORMAT} \ - ${DESTDIR}/vmimages/${OSRELEASE}.${FORMAT} -. endfor . if defined(WITH_COMPRESSED_VMIMAGES) && = !empty(WITH_COMPRESSED_VMIMAGES) . for FORMAT in ${VMFORMATS} - # Don't keep the originals. There is a copy in ${.OBJDIR} if = needed. - ${XZ_CMD} ${DESTDIR}/vmimages/${OSRELEASE}.${FORMAT} + # Tradeoff "cp -p" property for less memory, I/O, and time use. + # Also: -3 got > 30 MiByte/sec effective (source file) for 32 = GiByte, mostly-zero image + # on a Pine64+ 2GB and used about 600 MiBytes of "active virtual = memory". The about 16 + # minutes was vastly faster than the "cp -p" --and the sha512 = and sha256 are vastly + # faster on the compressed file as well. + ${XZ_CMD} -T 0 -3 --stdout --memlimit-compress=3D50% -v = ${VMBASE}.${FORMAT} > ${DESTDIR}/vmimages/${OSRELEASE}.${FORMAT}.xz . endfor +. else +. for FORMAT in ${VMFORMATS} + cp -p ${VMBASE}.${FORMAT} \ + ${DESTDIR}/vmimages/${OSRELEASE}.${FORMAT} +. endfor . endif cd ${DESTDIR}/vmimages && sha512 ${OSRELEASE}* > \ ${DESTDIR}/vmimages/CHECKSUM.SHA512 and using WITH_COMPRESSED_VMIMAGES to avoid a "cp -p" based copy of the large file. Prior reports from capturing text: On 2017-Apr-27, at 7:31 PM, Mark Millard wrote: > [Another example panic. Again no dump. But I have what > a top -PCwaopid froze at this time.] >=20 > On 2017-Apr-27, at 4:22 PM, Mark Millard = wrote: >=20 >> Unfortunately for this FYI the attempt to produce a dump >> failed and so all the information that I have is what I >> first captured from the console output: a backtrace. >>=20 >> The context was head -r317015 on a Pine64+ 2GB. At the >> time I was experimenting with trying to build a vm.raw >> from my own build of FreeBSD. The (root) file system >> is on a USB SSD off of a powered USB hub. >>=20 >> panic: ARM64TODO: reclaim_pv_chunk >> cpuid =3D 1 >> time =3D 1493332968 >> KDB: stack backtrace: >> db_trace_self() at db_trace_self_wrapper+0x28 >> pc =3D 0xffff000000605cc0 lr =3D 0xffff0000000869cc >> sp =3D 0xffff000083ba4f00 fp =3D 0xffff000083ba5110 >>=20 >> db_trace_self_wrapper() at vpanic+0x164 >> pc =3D 0xffff0000000869cc lr =3D 0xffff00000031d464 >> sp =3D 0xffff000083ba5120 fp =3D 0xffff000083ba5190 >>=20 >> vpanic() at panic+0x4c >> pc =3D 0xffff00000031d464 lr =3D 0xffff00000031d2fc >> sp =3D 0xffff000083ba51a0 fp =3D 0xffff000083ba5220 >>=20 >> panic() at reclaim_pv_chunk+0x10 >> pc =3D 0xffff00000031d2fc lr =3D 0xffff00000061a234 >> sp =3D 0xffff000083ba5230 fp =3D 0xffff000083ba5230 >>=20 >> reclaim_pv_chunk() at get_pv_entry+0x240 >> pc =3D 0xffff00000061a234 lr =3D 0xffff000000616184 >> sp =3D 0xffff000083ba5240 fp =3D 0xffff000083ba5260 >>=20 >> get_pv_entry() at pmap_enter+0x694 >> pc =3D 0xffff000000616184 lr =3D 0xffff0000006156a0 >> sp =3D 0xffff000083ba5270 fp =3D 0xffff000083ba5300 >>=20 >> pmap_enter() at vm_fault_hold+0x28c >> pc =3D 0xffff0000006156a0 lr =3D 0xffff0000005b9740 >> sp =3D 0xffff000083ba5310 fp =3D 0xffff000083ba5460 >>=20 >> vm_fault_hold() at vm_fault+0x70 >> pc =3D 0xffff0000005b9740 lr =3D 0xffff0000005b9464 >> sp =3D 0xffff000083ba5470 fp =3D 0xffff000083ba54a0 >>=20 >> vm_fault() at data_abort+0xe0 >> pc =3D 0xffff0000005b9464 lr =3D 0xffff00000061ad94 >> sp =3D 0xffff000083ba54b0 fp =3D 0xffff000083ba5560 >>=20 >> data_abort() at handle_el1h_sync+0x70 >> pc =3D 0xffff00000061ad94 lr =3D 0xffff000000607870 >> sp =3D 0xffff000083ba5570 fp =3D 0xffff000083ba5680 >>=20 >> handle_el1h_sync() at kern_select+0x9fc >> pc =3D 0xffff000000607870 lr =3D 0xffff00000037db3c >> sp =3D 0xffff000083ba5690 fp =3D 0xffff000083ba58f0 >>=20 >> kern_select() at sys_select+0x5c >> pc =3D 0xffff00000037db3c lr =3D 0xffff00000037dc58 >> sp =3D 0xffff000083ba5900 fp =3D 0xffff000083ba5930 >>=20 >> sys_select() at do_el0_sync+0xa48 >> pc =3D 0xffff00000037dc58 lr =3D 0xffff00000061b91c >> sp =3D 0xffff000083ba5940 fp =3D 0xffff000083ba5a70 >>=20 >> do_el0_sync() at handle_el0_sync+0x6c >> pc =3D 0xffff00000061b91c lr =3D 0xffff0000006079e8 >> sp =3D 0xffff000083ba5a80 fp =3D 0xffff000083ba5b90 >>=20 >> handle_el0_sync() at 0x4948c >> pc =3D 0xffff0000006079e8 lr =3D 0x000000000004948c >> sp =3D 0xffff000083ba5ba0 fp =3D 0x0000ffffffffd960 >=20 >=20 > This time I got to record from top: > (swap is on a swap partition) > (pid 49888's SIZE vs. RES and SWAP might be interesting) > (as might the Active figure) >=20 > last pid: 48988; load averages: 0.64, 0.44, 0.38 = = up 0+04:21:01 19:19:50 > 32 processes: 2 running, 30 sleeping > CPU 0: 13.2% user, 0.0% nice, 23.2% system, 0.3% interrupt, 63.3% = idle > CPU 1: 4.6% user, 0.0% nice, 23.9% system, 0.0% interrupt, 71.5% = idle > CPU 2: 2.1% user, 0.0% nice, 23.2% system, 0.0% interrupt, 74.8% = idle > CPU 3: 3.3% user, 0.0% nice, 23.8% system, 0.0% interrupt, 72.8% = idle > Mem: 1618M Active, 17M Inact, 315M Wired, 204M Buf, 15M Free > Swap: 6144M Total, 34M Used, 6110M Free, 348K Out >=20 > PID USERNAME THR PRI NICE SIZE RES SWAP STATE C TIME = CPU COMMAND > 48988 root 4 31 0 651M 27048K 0K RUN 0 0:03 = 87.60% xz -T 0 = /usr/obj/DESTDIRs/vmimage-aarch64/vmimages/FreeBSD-12.0-CURRENT-arm64-aarc= h64.raw > 11983 root 1 22 0 5068K 0K 0K wait 3 0:00 = 0.00% make vm-image vm-install DESTDIR=3D/usr/obj/DESTDIRs/vmimage-aarch64= () > 11981 root 1 42 0 7320K 0K 1516K wait 1 0:00 = 0.00% sh = /root/sys_build_scripts.aarch64-host/make_noscript_aarch64_nodebug_clang_b= ootstrap-aarch64-host.sh vm-image vm-install=20 > 11980 root 1 20 0 6656K 1548K 0K select 0 0:02 = 0.00% [script] > 11977 root 1 30 0 7320K 0K 1516K wait 3 0:00 = 0.00% /bin/sh = /root/sys_build_scripts.aarch64-host/make_aarch64_nodebug_clang_bootstrap-= aarch64-host.sh vm-image vm-install DEST > 2694 root 1 20 0 8804K 2072K 0K CPU2 2 0:07 = 0.17% top -PCwaopid > 827 root 1 20 0 7320K 0K 360K wait 0 0:00 = 0.00% su () > 826 markmi 1 22 0 10372K 0K 1532K wait 3 0:00 = 0.00% su () > 820 markmi 1 24 0 7320K 0K 1516K wait 1 0:00 = 0.00% -sh () > 819 markmi 1 20 0 18416K 1152K 0K select 1 0:21 = 0.00% sshd: markmi@pts/1 (sshd) > 816 root 1 20 0 18416K 3276K 0K select 0 0:00 = 0.00% sshd: markmi [priv] (sshd) > 765 root 1 20 0 7320K 0K 224K wait 2 0:00 = 0.00% su () > 764 markmi 1 23 0 10372K 0K 1532K wait 0 0:00 = 0.00% su () > 758 markmi 1 31 0 7320K 0K 1516K wait 1 0:00 = 0.00% -sh () > 757 markmi 1 20 0 18416K 228K 904K select 3 0:01 = 0.01% sshd: markmi@pts/0 (sshd) > 754 root 1 25 0 18416K 3276K 0K select 1 0:00 = 0.00% sshd: markmi [priv] (sshd) > 746 root 1 27 0 7320K 1532K 0K ttyin 0 0:00 = 0.00% -sh (sh) > 745 root 1 20 0 10372K 0K 1532K wait 1 0:00 = 0.00% login [pam] () > 700 root 1 20 0 6948K 0K 168K nanslp 1 0:00 = 0.00% /usr/sbin/cron -s () > 696 smmsp 1 20 0 10460K 0K 184K pause 0 0:00 = 0.00% sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue = () > 693 root 1 20 0 10460K 1392K 0K select 1 0:00 = 0.03% sendmail: accepting connections (sendmail) > 690 root 1 20 0 15800K 968K 0K select 2 0:00 = 0.00% /usr/sbin/sshd > 661 root 1 20 0 6656K 344K 0K select 2 0:01 = 0.00% /usr/sbin/powerd > 658 root 2 20 0 12788K 12672K 0K select 0 0:02 = 0.01% /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f = /var/db/ntpd.drift > 620 root 32 52 0 6384K 1100K 0K rpcsvc 1 0:00 = 0.00% nfsd: server (nfsd) > 619 root 1 52 0 6384K 704K 0K select 1 0:00 = 0.00% nfsd: master (nfsd) > 617 root 1 20 0 6684K 688K 0K select 1 0:00 = 0.00% /usr/sbin/mountd -r > 478 root 1 20 0 6676K 596K 0K select 3 0:00 = 0.00% /usr/sbin/rpcbind > 469 root 1 20 0 6680K 572K 0K select 2 0:00 = 0.00% /usr/sbin/syslogd -s > 396 root 1 20 0 9580K 32K 0K select 0 0:00 = 0.00% /sbin/devd > 308 _dhcp 1 20 0 6800K 532K 0K select 2 0:00 = 0.00% dhclient: awg0 (dhclient) > 307 root 1 52 0 6800K 424K 0K select 2 0:00 = 0.00% dhclient: awg0 [priv] (dhclient) >=20 > And here is the backtrace: >=20 > timeout stopping cpus > panic: ARM64TODO: reclaim_pv_chunk > cpuid =3D 0 > time =3D 1493345992 > KDB: stack backtrace: > db_trace_self() at db_trace_self_wrapper+0x28 > pc =3D 0xffff000000605cc0 lr =3D 0xffff0000000869cc > sp =3D 0xffff000083d301d0 fp =3D 0xffff000083d303e0 >=20 > db_trace_self_wrapper() at vpanic+0x164 > pc =3D 0xffff0000000869cc lr =3D 0xffff00000031d464 > sp =3D 0xffff000083d303f0 fp =3D 0xffff000083d30460 >=20 > vpanic() at panic+0x4c > pc =3D 0xffff00000031d464 lr =3D 0xffff00000031d2fc > sp =3D 0xffff000083d30470 fp =3D 0xffff000083d304f0 >=20 > panic() at reclaim_pv_chunk+0x10 > pc =3D 0xffff00000031d2fc lr =3D 0xffff00000061a234 > sp =3D 0xffff000083d30500 fp =3D 0xffff000083d30500 >=20 > reclaim_pv_chunk() at get_pv_entry+0x240 > pc =3D 0xffff00000061a234 lr =3D 0xffff000000616184 > sp =3D 0xffff000083d30510 fp =3D 0xffff000083d30530 >=20 > get_pv_entry() at pmap_enter+0x694 > pc =3D 0xffff000000616184 lr =3D 0xffff0000006156a0 > sp =3D 0xffff000083d30540 fp =3D 0xffff000083d305d0 >=20 > pmap_enter() at vm_fault_hold+0x28c > pc =3D 0xffff0000006156a0 lr =3D 0xffff0000005b9740 > sp =3D 0xffff000083d305e0 fp =3D 0xffff000083d30730 >=20 > vm_fault_hold() at vm_fault+0x70 > pc =3D 0xffff0000005b9740 lr =3D 0xffff0000005b9464 > sp =3D 0xffff000083d30740 fp =3D 0xffff000083d30770 >=20 > vm_fault() at data_abort+0xe0 > pc =3D 0xffff0000005b9464 lr =3D 0xffff00000061ad94 > sp =3D 0xffff000083d30780 fp =3D 0xffff000083d30830 >=20 > data_abort() at handle_el0_sync+0x6c > pc =3D 0xffff00000061ad94 lr =3D 0xffff0000006079e8 > sp =3D 0xffff000083d30840 fp =3D 0xffff000083d30950 >=20 > handle_el0_sync() at 0x400a3de4 > pc =3D 0xffff0000006079e8 lr =3D 0x00000000400a3de4 > sp =3D 0xffff000083d30960 fp =3D 0x0000ffffbfdfcd30 >=20 > KDB: enter: panic > [ thread pid 48988 tid 100230 ] > Stopped at kdb_enter+0x44: undefined d4200000 >=20 =3D=3D=3D Mark Millard markmi at dsl-only.net _______________________________________________ freebsd-arm@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-arm To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Wed May 3 17:36:53 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22FC2D5C418 for ; Wed, 3 May 2017 17:36:53 +0000 (UTC) (envelope-from Zhuojia.Shen@rochester.edu) Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0094.outbound.protection.outlook.com [104.47.33.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D169F126C for ; Wed, 3 May 2017 17:36:51 +0000 (UTC) (envelope-from Zhuojia.Shen@rochester.edu) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rochester.edu; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=n7f2UcKdezfrzGNICmKzbj2K6OdJg0Sxe0W83Ra5F/c=; b=pawaNzF1JosHdK87fynM6VEYUNfRRWReotxX6b0p1BNxuNWg5BX0TZcz+KnehgaRQJUj+oYYTkne3PxmFUbcmsPIyqXIKHEhhBZJXakbvLGNXh4wo4FtM+tAuBH1/3+VMmMIlfiV3cazyBHejGr8CDpldjp+9Fd4CzSWvcJYofw= Authentication-Results: freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=none action=none header.from=cs.rochester.edu; Received: from j13.cs.rochester.edu (128.151.67.93) by BN6PR07MB3521.namprd07.prod.outlook.com (10.161.153.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1061.12; Wed, 3 May 2017 17:36:49 +0000 To: freebsd-hackers@freebsd.org From: Zhuojia Shen Subject: Allocate huge chunk of pageable memory in FreeBSD kernel space Message-ID: <9f9ddba2-19c5-c182-8273-c173650b938c@cs.rochester.edu> Date: Wed, 3 May 2017 13:36:46 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [128.151.67.93] X-ClientProxiedBy: BN6PR16CA0048.namprd16.prod.outlook.com (10.172.26.34) To BN6PR07MB3521.namprd07.prod.outlook.com (10.161.153.37) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 09b62d09-846e-4f11-db29-08d4924afaa0 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:BN6PR07MB3521; X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3521; 3:RzgjFtKNkxfhzsnJaj4LOVISbZIwa0y3W34UOEeHbtkvKt3Sc1hExqVtiHAiVcnzjq0QGFo9/PAJVdIygQ7wkIUt9p0Nn1ggTBzGYARIbafoHN03lNpbPjbCt+GwG043pDfWVAiGFpqY4alJ4E1qrrYrEIa3+7a8KaqZ8TyiDve8/qV/9AFqGQO4bLLMFyYKtXjbtJvJq7YLtAx4Se6tonTWfY+05ruSwGmSKuzf9Ysr0oF/Ta8AhUtBJ0cfXsNsHvi5gsbauGRAbcOjrRfVTnRT13Y9AvINNsY3EEPxW3xNmRO3X16+urERUwCkhLUMnh+aLK5rIQHsexCK32NV5A==; 25:0mVIha/qxq4UFka8rnFrn4Zkm0xuoTK+XlTrFn4uAqm9XXztl7Bt6p2LnKB/sTQ85LxoeBca86NZaQmPUGJMNmEDNMWTq1cyVi0U+vTZYnFNscETBGtsxfzwsE28ExEe0iibe4OLdVTHr+EG2ojdTT39JnczOf0GyjXbj7pJBIOx+5JbIzBPv6Y2xm8xhjEpjnDSlhh0BvSF5crQ2N7IVl3t9pNVfWWI7FlB4i9JD2GEeiK7OBCgJrUl0cPCeGTL9B5g5fKqsCXKRbB21nblr54CumZ0D8tq513fpNfVTEIThvIjcQolG+HEHmOqXukJYXzcy0D1a77ZeHIkb1m6Kgc4B71v2gdLGUbkBWtsTd97vAFbgNFKyNUL9DoAhIXPI9QsJt6CLhPflzMaodlTUY+57XwSk19pXvDcPRGGi/whInFAQzmXvJ9HhP4CUjIL X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3521; 31:XadYEl/atVqojdyztKH9Nq6s63wJf8D7MirCaVK1N8wcVgwDgNZnmC/M6J9Mb/pvAedV6Q00Eir/RSyKteAmTiFHwrbnEH1TJjJ5oDnPmg39Vf90RiZWgda9jYGpG2z1qDX6SHyX+XwMfLVDtDFislOpuHj5kf1d2RlBPw34ewxR8Xcckl37wSk9OXl2SL1mgWmhY8PPScbzdn+ycpyU6bYl2UHtCZ2eg3usMzHIAPHNhk+zLKvg/97yH5cfF+oFMaOUva5QquFEFj1IYTFlPw==; 20:+NqBf6vTqb5lrVoDaSXlpxoTE5vtpPg1iRvKZdCtwcMvTDGr1REjQHcmu4kitZbr5xIKTIGf7sp6H8XX+bkmQlW2WlYuq+5/FjKvRb4Xdg9bklnijI8brj1nU8bLvyRurNFg3Yi8tDmE6DHCOYDgTtwIIQ4ZPoC+YZdsw9xRSY05/0UYNb5h58vJwU+SBnKPj7J8yFHmSckgV44DzL86rsNuNJZ4ZSAm1TXHVv0dW+27BdJhxYoQQ/E7yBwkwVZl5P5Qhpf1zFcPXTxhCbHSHWzvoWQZ5nEEMHL2mu9RhNGF9J8U0M4DKm3yyKliItA1v7jefJiO4u7OoEz2f9N1vNb7AJdWHeFqVfrNgdmwzpIRDamkg/YDt0+qRzZNEAxRati6NQZDIqH9OhDAMAkPNx9KXmJ8LW/t+cVGrw3JdeTkAllHiYxeTRZAk8nTyK0cv3TAaDbL7uiJh+PTCgB5dqO1ogxyqBJoUYAduRCWoAApBoiWVUFBfGp/8zm7XCif X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(192374486261705); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(6041248)(20161123562025)(20161123560025)(20161123564025)(20161123555025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(6072148); SRVR:BN6PR07MB3521; BCL:0; PCL:0; RULEID:; SRVR:BN6PR07MB3521; X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3521; 4:H0HW5j9SHpLltvh44sRVnv200WI0fDp1k3EErWU9yw0nHqtgjrws+61UYE9bsjNGPPN6Dzop6ZowDZDEB8fXgxF7ayoJWttd/AWR3Z4sSlM3FRCqI4P18GK+EwfdjNcdg6FtEdZgK3RypBVXklssn9LvaLl8WluV902kDHE+lYODCYDFVWnX8dFgN++UXclJHgiAYMv0q/4FKGs+BO85+bB8gWkT1MN+CwMF6Lex4Kxw7vrNXbHvVDp+LHAhvzaX0a0aMSAwheLK7jgWcRCB1cFwVbqV+lWghIzX9Hj+gfqW1lWNSlIjbaKX4kEi0ZvyEBVHUDM2Ip2lpyTR052LnX7V3/a5Xgh2ujli2ru/ChWZFfIRWYRjUZRA7JMShWTe61HO30AL0ISl+erwywJxbKMt68Cw1UQ7IDpinMGNEEE3qx2oV7bjSWiq8jhtAUOlG0c0R/Z6e9swL2kSrzIuciuE3p7t/F+9qLwn+YQAlo2MnRnSTQ/2rtRjVzKAv75KqMbFToWM8fx+WoUr3YFNCLi3I9C7FghIgfesDozGOVot0gVxAHoqpzAC31YYGw6p5SSyYGm3zPfLKkr0w9dRWP93gLI6Uce4wyal5KYRskHCq7IJ2z9x92TAHb60Ilv+4cikCMm+Nbhe1do3hUnd8L+rNzK5vitoPIxARf+mph4d++F4jos37kl1BG8AFG/dDInqkx2cqXP44D7axH5oAqce8Ab3Xqb39ssgMtBvYcN6LkNeBqG/uB0jjFeauU9L/EFab4lXWlaajnCctzFIfaUgGyhOoyKbYLMsWzWV8mXfsARFs09osl53/M4nVFUqVq3sjcG2u6A8k3VQOyUmzfGCFFwvesA6t2x7mvjohcE= X-Forefront-PRVS: 029651C7A1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6009001)(39450400003)(39410400002)(39850400002)(39840400002)(39400400002)(54356999)(38730400002)(50986999)(23676002)(2906002)(53936002)(8676002)(6486002)(81166006)(2351001)(2361001)(33646002)(50466002)(88552002)(478600001)(3846002)(31686004)(64126003)(42186005)(47776003)(6116002)(189998001)(110136004)(83506001)(31696002)(7736002)(230700001)(4001350100001)(5660300001)(6666003)(25786009)(75432002)(305945005)(6916009)(14583001); DIR:OUT; SFP:1102; SCL:1; SRVR:BN6PR07MB3521; H:j13.cs.rochester.edu; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTjZQUjA3TUIzNTIxOzIzOlpYZ0FFekZvelA2SEtKQnZQd2IySDloT0Nn?= =?utf-8?B?aXh1NmtsdWd2aThTVTVOMVhCak44ajBZUU9MM3VhQlFYNUFLU28yMi9TcFJQ?= =?utf-8?B?dVIvOTZpcUQrakNmMGExMDJHbEM3U09rUmJYWExaQ0pma0FCRlBKNzBLWFJJ?= =?utf-8?B?NFNVQWdVdlZOU2NEODJnQzJ0NnNaK0QzcWFCYUpldHVwR05UZG5INlY1Nzgr?= =?utf-8?B?VHlaRWowN3BBajBSU2RTQjY5R0tERnNCbSsvejZJOUd6UlVmaVY5TFZzcEVH?= =?utf-8?B?aHNlTEZBV2hDQXVZVDEzSlVDa2M0czMzYW5OQzM0S1JIdG4vMGk2RjZrRW5t?= =?utf-8?B?ZUpENUdCR2FyNUtwLzhXK1Joc0YvVWtvT3Nsc3ZBR0kxRGd5eTA3SGxXTTVE?= =?utf-8?B?SnRjMTlBZ1ZpUGZVRnFBUXE2emF1RlpOU0hoNFFJdUloMERmVnBROVZ2YmRT?= =?utf-8?B?QzdJNlVIR1VrUzFoRUhvNm96VGN6MHlVTVRsWWpSSkJ1MWhTVEwvTDJmRjVQ?= =?utf-8?B?QnB6aXBueXFaMnIzQ2xvdDhEdHVmbVR6eXNvMkNtQkVFbFRlcFRRMkZpRmVW?= =?utf-8?B?UTg1T0pxYnlkYUFLS0J6dVNaTnVmOTE5cXNBMHhPdzFtNVNxczlaa2RHRGVI?= =?utf-8?B?dm84aERxbVhBaDNwL0xzUDd3Qm5vNkFVSTZxYjlpR2trVitPYzBvcG5tRUxr?= =?utf-8?B?V1lkTTQxVUI3R0JuTk1hbE45SmlPS3hqcU9ZKy9PekVUSEVUWFg4ZXA5NE5m?= =?utf-8?B?VVp2U3JJK0NLZHo3bVVDT2hHcjM4UjRuZmc1OGJuK0Q2UVgrVnNUWWh6Vkt5?= =?utf-8?B?ZmthODhrejJSVXNlMXhVS1RkTGJBcnd2Um5QOENLdzNxeHlIUFV6L0Y2Unda?= =?utf-8?B?TVhrKzR5aWplL0w4Vi9jaG5qUTBPKzJrNWdBLzE0VjlXY1l5cXZqWW4wZU5m?= =?utf-8?B?TFN0eTU4R0xBbENvNGREa1pweS9sSUlBY2xMUkRibmRseTlveEdJaUhjOEIv?= =?utf-8?B?d01RWDNvdWduamR6dkVNMzBZblZrZVBDRHh2Nldrb3FLODZuMUJmeUE5NHF2?= =?utf-8?B?UERIK0p4SHR4TUVyNzY5NmUwYzdXTlkwbXVvTnJRQVoxMGovWEUzakh3VHFJ?= =?utf-8?B?eHhkWDVvZGIrSWFnQmZ0eUsxNkRaMFZyZ2tmbUdLMmVzWmVpczd5YjBKZm1p?= =?utf-8?B?KytpSzE0OUlGeUdmeURnWTJoNDc4Z25LUEIwTE5HaG52MGpCUkJ1ZnVGQzNj?= =?utf-8?B?bmpLRkJ5Z3VvV1RTNUREQUh6M1pkR0NvTFZsV1ZucVpxWHVVelZVUExJMzhQ?= =?utf-8?B?ZVZGOU1QZUVJWE9oOG01dzVzdU5CbjcySHZRVjhyS1dmbW9ERVpCczRKR2JL?= =?utf-8?B?Vkk2VFNIaHdJNWFyR2xjeE94T2VqQWo4UCtMWHVqQXdlS2R6K3FPVWw3U0dX?= =?utf-8?Q?W1h5wZyKyAdDotghBRlImEhXuT5?= X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3521; 6:HtQUV84v2LW9+/SGWlhDDDY7Kptti9ok2Tr25hwrit8t2ZcHUFUEXxMZNddFW51EirT7wIVZHU4Ajf9s+7uXOr7MDQmVLqZ8+MhUFnP9Cej1veyvkyskiyH+bsTYjeqNguIx1oVVhV+9Xb03lJwti8kPNHuuvs8wX6o1N9fbRtNkwh9m3cXLGxswEmWNOIHrEbTI7ULuAr26f3CsVaxfMshgkez/a+7txdR9HS0+iw5+n5hkw006V1tnlEcEX+29E0Ss67oGh0KznjM1WUStwF4I8shWiVNoiWXSAnmJHWERRAFhxFyyw0qyAEeQeVLSR2F/gM8isb65UGKT8vQncTAsDA4tH5vxKs230/UZrvZREz+zM7NIB9DfesjhLH3/OKxhplqzHPygBWq1a0IxaS09ArxGJoJwjhbsbEL5fzINuslO1DJzITDVmDenV88kQaPRPPDjHVfoJNxOoUxPZap5ysVvX2StoysHL3PhBOj9IoIOQJ2V+yoCCkv7os8PhuxLJ9MfNvcjDmhrAmqUiw==; 5:bls2/ZjQzTuduG5Z2L096P82042vUC3m1kXTGIRVfHLQfS3gXSd0SnJm5dVtK/fDgcRw+W0SasnCyt2Ppz+oW3vFW8u5I5RyUhj5Smi9J9A7a/pcLDmW/6iHnaOVzx9rhVBXVkHQZmmWcj0jlpSQ7A==; 24:W3x8a+bJaupiAkCOEMacJr3o0M65hfCSz9t/CuoeU3K0LAW8X2IMM7BRz460OuP+SX4TSs3/Z+lFml+etWLxDO+g3xnuKxqLriiz3I0zfEk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3521; 7:ZNP2BNmIU+Eb/MWrp19IxpKWaZ+TxjZlOnlbK1cMv134it/ma55xb9x0CNdatmbrLqEUwuyXLY1UlrF3/bGIAGWv/cc9YYuP90MJL5SorUF28DfPXyJH8ai/cZrdRWXmyWVV0AYbHF6kBOZULM9pYZ5MQlamRD+CEaPn8A4adn5ueZWa7Q/6QLzKuhlusDosWPXWy/DIoYVqlDLK3iVuysDgMk+RYZN6npzs0Nsi1VP7kxRBB4xRRP6lsvETf7gdjzzxKPnWBzC7C+LiHhXzJxm2dFOpBNPzg3Vf+6QTZy/Tvk/rIQivnnM1UB58dQNe6A8luIf//gXaN2h+NRBp/g== X-OriginatorOrg: cs.rochester.edu X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2017 17:36:49.5543 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR07MB3521 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 May 2017 17:36:53 -0000 Hi, I'm working on a project that uses FreeBSD 11.0 on amd64 to do some security research. Now what I want to achieve is to allocate an 8TB portion of memory within the kernel virtual address space using demand paging. I noticed that there is an unused portion of kernel space from 0xffff804020101000 to 0xfffff80000000000 where the 8TB structure would fit. I also noticed that there is a function kmap_alloc_wait() which can be used to allocate pageable memory. Do you suggest that I use this function and, if so, how would I use it? Thank you, Zhuojia -- Zhuojia Shen Graduate Student Department of Computer Science University of Rochester From owner-freebsd-hackers@freebsd.org Fri May 5 11:16:12 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14B8AD5FEE7 for ; Fri, 5 May 2017 11:16:12 +0000 (UTC) (envelope-from zec@fer.hr) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 057F01FBD for ; Fri, 5 May 2017 11:16:12 +0000 (UTC) (envelope-from zec@fer.hr) Received: by mailman.ysv.freebsd.org (Postfix) id 01DDFD5FEE6; Fri, 5 May 2017 11:16:12 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0181BD5FEE5 for ; Fri, 5 May 2017 11:16:12 +0000 (UTC) (envelope-from zec@fer.hr) Received: from mail.fer.hr (mail.fer.hr [161.53.72.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.fer.hr", Issuer "TERENA SSL CA 3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9101D1FBC for ; Fri, 5 May 2017 11:16:11 +0000 (UTC) (envelope-from zec@fer.hr) Received: from x23 (161.53.63.210) by MAIL.fer.hr (161.53.72.233) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 5 May 2017 13:16:01 +0200 Date: Fri, 5 May 2017 13:16:38 +0200 From: Marko Zec To: Subject: AMD CPU cache monitoring Message-ID: <20170505131638.4835dfce@x23> X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [161.53.63.210] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 May 2017 11:16:12 -0000 I'm looking for a simple, vendor-neutral tool for monitoring per-CPU-core data cache hits and misses (L2, L3). Adrian mentioned https://github.com/opcm/pcm in his article on profiling: http://adrianchadd.blogspot.hr/2013/08/profiling-on-superscalar-architectures.html This tool works fine on FBSD 11 and Intel silicon, and would be almost ideal for my needs, but unfortunately it doesn't work with AMD CPUs. OTOH, pmc(3) seems to be a more generic mechanism for working with performance counters, but I couldn't find useful examples on how to harvest per-cpu counters. Most of the pmc-related examples I bumped itno seem to be focused on call stack profiling, which isn't what I'm looking for. Any ideas on how to monitor CPU cache hits / misses on AMD CPUs? Thanks, Marko