From owner-freebsd-arm@freebsd.org Wed Jan 13 00:55:24 2021 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 14C2A4EC2AC for ; Wed, 13 Jan 2021 00:55:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DFprp6Xc4z3C6p for ; Wed, 13 Jan 2021 00:55:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1610499320; bh=e4ADgYEDx7BLzaFglTcKie2cVM76egizwYe2lfMs9q8=; h=Subject:From:Date:To:From:Subject:Reply-To; b=LGIJf1xmZCKchOEmXHjULaGNIZQl8943yAZyvEP1auGikHFzfe2A7ntJzeefkHSKPFGe+CNllnRngP9rSjYr3hOK/nsPbxYcSd4BE0HB5lhOIxMV6NGnmvgpoRmbIRH/b0qyax20qgMSOvNBYeuhwjcXBbtOJbqxtVxqGo/meGV2FjalAkXDNjVVE3ygUEJvIu1VxlluFiGtqfl5Jmmeqfe63oDVgSV07UoccNJpKrnOKVp4elWed/UBFKLGqVFQcRJD6RBI2M06ChaqW7qZYnrc+mXA8Yo6jLD+CgnnEEuIhv3qKwlU3BAfsap+v+rjl7vO1ptr4yAF2QOSeYSWPg== X-YMail-OSG: vFsPzPAVM1mIMH9X5lOCz5Xe1Ns68Ir3RNzyZvDb4yQBG4tFd9IQU_EymcyYwnU wCV4zdI4HSkuoAcZSIo2E0ffQjURY0rRaF5L4oEzldeWQCxbH6JA0ilYaMmMBjQLwsUk3oKMhr0R K7uHZjFhYBrtcnL30yKO1jNVy1CZjTyFqQ89sKZj7YCPiD25ykqCGaTfZHM9D5n6bZEV2j6CVlIB Yfk6wMn07ZcBrAD5yCdAEh0PzRKNDAhvPr9WFtr3r17vT3IbmVJIDj2SVh3Dj0.vEhPI.rKn.bN4 LXDeZhqr94LTErAEqOsUs8aSZUuehdZ8xSLLNOkq6NyB4Lmv02CTq75RoGCjGtK1crNrPRqPp3p2 AxldiJ_6dNz5qcT_4bUCPwAbh4Y2TM6KOTfP_ZVPPoiXBJURVcJczwHKAkMoJSq6..Kd1dEMCGUm TC2ON50XI6k8UhyLNinzuD0kybIayUsaWoVstNacvBOj2xQhBghIU4bWdzFBXgcGCtOcUKARhZ9o vAKt3HyReGcoL1uW508tw28Kk4NTJvHoHLGkofwHD_AYJgAMjm2kzv3ojNTxEymK2a9kCEmbUpmP k.A0CmwejlemYCjLeEEm1rC5FAxB4oZdqPJRTeCRxx_klHmrcHLZ2lFARvul6CzbtXFF7SRBcFh1 1O_cGWUNWt6.CEx_uV_G9UtchvOGM1ePHmujty2Fh3RC1k5wuoQrBjUaS7gKhcK5SmDIixstZI_Z u9LSjZYVK4Mg5Da05FtnL6Mv_1GUkikKWNwI5gEsacSoGVm1QlMl_ixtYFRB9bJCz3mmdODPvCzz sUwgBS6WK1Ea33rM7BmTamJVp5DPm3pyh_5dep9axBBwDorOOL7wIuFMRSTS.mpYoxutNUNglDYO lIfr4TphTpZuQh7TGELXaLll2THkDO12OEZtt2NcCfQNrdGFaDrtkLwbuXKyaiMSmTu7kEZEzaI_ CV2pTyiScAwmJIHWOxUc7XCNbfQnleMZAP0471jKEsNLwAFD2v_k8s6PgnaIQJFDbYhpKxPUx6VT gqtzdKu581f8lbmjYjpkLjdQgnuxbBgqbIivBlioPwEfldIRHbV2Ind5XYhsfDzz2nTWklFVctNB 1bW.vbGuTvkbpPtg2Fcx9ZLSDXtMlsbpeKHmYc6OD6G7pE4o0Z4GJIIeuHBaqCiA1ycDixs79W9N nfpT9e0MK56gtrCyClh5vtc4w84SoVQQFJHx3i6YKE1aeQGxc.izN1vORDmTnK1fPs1W7XaK3jyu 2n6sM4lmDcnYR9ZgeeRvY658vLz1JeE6yPTgJcV1YH3.YT4yg3XDJ2ywHZbkGPtEO9Cm_cFpQs1t gOb3hIyVzUF4SZeoMjSdm3T2eKuMyiv4HY_iXlWrcBRDgPVfA_hMOWXR0dWFD9Sx6HHH7GCMetb2 PiQ52oIjUEzRqlGlmj9Yj4qTrCidUlamV04Oawccfd9dZWhSAUrIZmQQpXBAHVmKmoevaizOeuaj y9vfQzeE1za8eqd4m9r79KyLbOWbxTtwZwzhUgrU92wCiE25NYTeYWSLG_2DAtCv2Trw970rdL4O B_fR3b17ItkEB2BhPsFmKJuMCpjydFlWqmLFBg2l2M.FBddSfUu.2OZxGm3Hcg9Fqk1ya21hO8fb EbFOKGxMtaiGJfFTinj4MQdKU0uwofaX2NR3002_ud8_VFPvKC3khHIriqR.w2uSWmPNzXkPizWs CJMQGDGl6Pilq4wpbtna1Xwf.ahDZ3L8Okut1Vo_x7shYJgWhWeYTeZu3U1pzO2tgmhs1Dfk0N8t ymug8u.pHMvgQrlJhs8yis7B1ro._Ks_hjCz4Prlf9CyXWUj7tpRaK.StIUe.hnzAbYt79pZybw3 mfPff31dNXYQmpj3QXi_b3vLWp0dulIlb3.QVDyHGkNlgImLPovpJ4N1lM1ULjEKRzRr4AGcTGBd vh9Jxw2ySWVjKgYobaor9LPK4D0IC_QblMq5K_ah3hWGG8HhDxUwyhJstpDZWdsbyELl_YPpYO6r CdKOEiyYs8yGjsafFVu1zzSxVbRtzBoCg1wNGA4MBkj7dN95MLnOFnqCgBUzwrk50HljXH91GcyV g0H7r8v0.OGWacbTN3OkS7xdAxMG6Wwar379L_lMHd_OHkP8BZluTFIKbzBQ_y1J0R1cJNAdOq.6 PyNva7ybsTdQAx78toEdBETgQfp8TJ7iFXxWaL.K.VYUaLhy_a70rSgtNUdEe0R8GS5JT1k6b9xu hWt9N7xb7VVj7.HOOjaeBVIU1aW6KriDEOtDZtDelqYk6qY.OpeRKx0hkCVY_NXURo.Weoii574P hj.UIHu2DS3efwJ483S2aqTI4oT2Gmr1rZkSdG9zCFYVRnaB6yIo7DoZh_YfXjQegSfKDlIR4zWl YWrz6HP.5DzRhMxEiLk2Tgrcknd8zLbOPqgAaTrOOkWlxqh_1qon0S6h92DgeYz3yii8dW9kp9oW tXXpVHI1b1N113EGQfJw..zn0nGz7n3xyaYO3NGaoOChq8yrHPxqAvAGBOFDx9auyHusrs9EGIoa mu3FPV7XmpF_vz0A84n1ijiA08t23d4mOsb2uDPGCvlgJxijKVkbqUozHfMclaBiGkAxQWEWz4gY VFm19YmRkjty89B3jKmKAsVZ6NBPYo.e7OtYMXPRPgU7_VkVt3IiCRmuUMsb0ClOxsEgif4e.DRT RHYssim6JqTJIaQv_Wal2989w_cgxs9X8OmGICxTYlR7oZQFJ05g0PqC7ZBsPqV_7TJmNIiXSO.. CiDTjKVRf667BBr9EnaLgUsrjC7n5jd8- Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Wed, 13 Jan 2021 00:55:20 +0000 Received: by smtp421.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 4a998bf65824f5e08bc660891ce2e540; Wed, 13 Jan 2021 00:55:14 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.40.0.2.32\)) Subject: Re: panic: Too many early devmap mappings From: Mark Millard In-Reply-To: <20210113002432.GA79600@www.zefox.net> Date: Tue, 12 Jan 2021 16:55:12 -0800 Cc: freebsd-arm , Gordon Bergling , =?utf-8?Q?Klaus_K=C3=BCchemann?= Content-Transfer-Encoding: quoted-printable Message-Id: <04FEAC11-5603-4D4E-8651-43AB37A10B46@yahoo.com> References: <20210112233607.GA79348@www.zefox.net> <90C90797-A8A5-457C-AF07-800EA82F5F12@yahoo.com> <20210113002432.GA79600@www.zefox.net> To: bob prohaska , "mhorne@freebsd.org" , "rwatson@freebsd.org" , Ed Maste X-Mailer: Apple Mail (2.3654.40.0.2.32) X-Rspamd-Queue-Id: 4DFprp6Xc4z3C6p X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.43 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCPT_COUNT_SEVEN(0.00)[7]; NEURAL_HAM_SHORT(-0.93)[-0.928]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.69.148:from]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.69.148:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.148:from]; FREEMAIL_CC(0.00)[freebsd.org,googlemail.com]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2021 00:55:24 -0000 [I have a git bisect result for the failure: bbfa199cbc16.] On 2021-Jan-12, at 16:24, bob prohaska wrote: > On Tue, Jan 12, 2021 at 03:59:44PM -0800, Mark Millard wrote: >>=20 >>=20 >> On 2021-Jan-12, at 15:49, bob prohaska wrote: >>=20 >>> An RPi3 running -current updated on Jan. 10 installed a new = world/kernel and=20 >>> when rebooted promptly crashed with=20 >>>=20 >>> ---<>--- >>> panic: Too many early devmap mappings >>> cpuid =3D 0 >>> time =3D 1 >>> KDB: stack backtrace: >>> (null)() at 0xffff00000011ad90 >>> pc =3D 0xffff000000760f70 lr =3D 0xffff00000011ad90 >>> sp =3D 0xffff0000011df330 fp =3D 0xffff0000011df530 >>>=20 >>> (null)() at 0xffff00000045c2d4 >>> pc =3D 0xffff00000011ad90 lr =3D 0xffff00000045c2d4 >>> sp =3D 0xffff0000011df540 fp =3D 0xffff0000011df5a0 >>>=20 >>> (null)() at 0xffff00000045c07c >>> pc =3D 0xffff00000045c2d4 lr =3D 0xffff00000045c07c >>> sp =3D 0xffff0000011df5b0 fp =3D 0xffff0000011df660 >>>=20 >>> (null)() at 0xffff0000007d8380 >>> pc =3D 0xffff00000045c07c lr =3D 0xffff0000007d8380 >>> sp =3D 0xffff0000011df670 fp =3D 0xffff0000011df670 >>>=20 >>> (null)() at 0xffff00000075dc98 >>> pc =3D 0xffff0000007d8380 lr =3D 0xffff00000075dc98 >>> sp =3D 0xffff0000011df680 fp =3D 0xffff0000011df6a0 >>>=20 >>> (null)() at 0xffff0000007710e4 >>> pc =3D 0xffff00000075dc98 lr =3D 0xffff0000007710e4 >>> sp =3D 0xffff0000011df6b0 fp =3D 0xffff0000011df6d0 >>>=20 >>> (null)() at 0xffff00000028850c >>> pc =3D 0xffff0000007710e4 lr =3D 0xffff00000028850c >>> sp =3D 0xffff0000011df6e0 fp =3D 0xffff0000011df7a0 >>>=20 >>> (null)() at 0xffff0000007c8788 >>> pc =3D 0xffff00000028850c lr =3D 0xffff0000007c8788 >>> sp =3D 0xffff0000011df7b0 fp =3D 0xffff0000011df830 >>>=20 >>> (null)() at 0xffff00000028a64c >>> pc =3D 0xffff0000007c8788 lr =3D 0xffff00000028a64c >>> sp =3D 0xffff0000011df840 fp =3D 0xffff0000011df850 >>>=20 >>> (null)() at 0xffff00000039b340 >>> pc =3D 0xffff00000028a64c lr =3D 0xffff00000039b340 >>> sp =3D 0xffff0000011df860 fp =3D 0xffff0000011df870 >>>=20 >>> (null)() at 0xffff0000004a6950 >>> pc =3D 0xffff00000039b340 lr =3D 0xffff0000004a6950 >>> sp =3D 0xffff0000011df880 fp =3D 0xffff0000011df8b0 >>>=20 >>> (null)() at 0xffff00000076d73c >>> pc =3D 0xffff0000004a6950 lr =3D 0xffff00000076d73c >>> sp =3D 0xffff0000011df8c0 fp =3D 0xffff0000011dfa00 >>>=20 >>> (null)() at 0xffff00000000089c >>> pc =3D 0xffff00000076d73c lr =3D 0xffff00000000089c >>> sp =3D 0xffff0000011dfa10 fp =3D 0x0000000000000000 >>>=20 >>> KDB: enter: panic >>> [ thread pid 0 tid 0 ] >>> Stopped at 0xffff0000004a6550 >>> db> reboot >>> cpu_reset failed >>>=20 >>> It had to be power-cycled to restart. It came back up readily with >>> kernel.old, which reports main-c255664-g4d64c7243d26 compiled Jan 9. >>>=20 >>> In particular, how does one recognize which revision fixes=20 >>> this problem, assuming it's a bug and not operator error?=20 >>> Presumably, it'll take at least several days to reach git. >>=20 >> Discovered last night on 8GiByte RPi4B's relative to this: >> Booting without a monitor changes the memory use and avoids >> the panic. WIth the 1920x1080 monitor it fails. (Only kernels >> with INVARIANTS make the check that panics, but need not >> mean that others are operating well, even if it is not >> obvious in a specific context.) >>=20 >> Quoted from part of a message list item from last night: >>=20 >> QUOTE >> Going back to my 19cca0b9613d based debug kernel build that >> has the printf's reporting the values used in the test, but >> with no monitor attached, it boots fine and reports: >>=20 >> pmap_mapdev early_boot: akva_devmap_vaddr: ffff007ffffff000 size: = 1000 >> pmap_mapdev early_boot: va: ffff007fffffe000 VM_MAX_KERNEL_ADDRESS: = ffff008000000000 L2_SIZE: 200000 >>=20 >> That compares to the previously reported failure figures from >> having the monitor attached for that debug kernel: >>=20 >> pmap_mapdev early_boot: akva_devmap_vaddr: ffff007fff816000 size: = 1000 >> pmap_mapdev early_boot: va: ffff007fff815000 VM_MAX_KERNEL_ADDRESS: = ffff008000000000 L2_SIZE: 200000 >> panic: Too many early devmap mappings >>=20 >> where the code does: >>=20 >> KASSERT(va >=3D VM_MAX_KERNEL_ADDRESS - L2_SIZE, >> ("Too many early devmap mappings")); >>=20 >>=20 >> Looks like akva_devmap_vaddr gets smaller to make room above >> for monitor related data and so va can end up being too small >> by the criteria of this test. >>=20 >> I've no clue who would be appropriate for dealing with this. >> END QUOTE >>=20 >> You may have provided a bound for a bisection >>=20 >=20 > It looks as if unplugging the HDMI monitor (1920x1200) fixed the > panic on the RPi3B+ as well.=20 >=20 > [the original subject line said "devmatch", which confused me hugely = 8-)]=20 >=20 A git bisect sequence on a 8 GiBYte RPi4B with a monitor plugged in (to make it use more high kernel RAM such that the KASSERT indicated above fails) resulted in: # git bisect good bbfa199cbc1698631a0e932848e62dd76559d4d7 is the first bad commit commit bbfa199cbc1698631a0e932848e62dd76559d4d7 Author: mhorne Date: Wed Dec 9 16:38:42 2020 -0400 arm64: gdb(4) machine-dependent bits =20 Everything required for remote kernel debugging over a serial connection. For FDT-based systems, a debug port can be specified by setting hw.fdt.dbgport to the desired device tree node in = loader.conf. For example, hw.fdt.dbgport=3D"uart1", or hw.fdt.dbgport=3D"serial@ff1a0000". =20 Looks good: emaste Tested by: rwatson MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D27727 sys/arm64/arm64/gdb_machdep.c | 112 = ++++++++++++++++++++++++++++++++++++++++ sys/arm64/conf/GENERIC | 2 +- sys/arm64/include/gdb_machdep.h | 81 +++++++++++++++++++++++++++++ sys/conf/files.arm64 | 1 + 4 files changed, 195 insertions(+), 1 deletion(-) create mode 100644 sys/arm64/arm64/gdb_machdep.c create mode 100644 sys/arm64/include/gdb_machdep.h =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)