From owner-freebsd-current@freebsd.org Sun Feb 24 17:07:06 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FA2E1504448 for ; Sun, 24 Feb 2019 17:07:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B1A275109 for ; Sun, 24 Feb 2019 17:07:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x730.google.com with SMTP id z13so4003763qki.2 for ; Sun, 24 Feb 2019 09:07:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=u6sSNMbV7AWggVafGl+9EjQ2biXDsQUnKCQVNFxppZI=; b=csoE0+jaYza9lPQIEKfu2muywhY0kn8AFTKroMgxVVfGkDqrhiZL6SLgjmNInweJdB j4ktbRArWjbMP93N+8xBjOvcx0hzW09hx8ETVu+0oKAlELJt3psrRdz08FM1L8eSVbHa KplB7sJQkJ2G4o6IG71GwWwn/rSkp4DJnLZ2pOL/UVlkED+Su2UnAp7+/9wBPTnOoLwk bfY5uhPd5i1qFi3HhpAKqHPv/6E4m4hxFuwiKS0nk0ZDbUbr7rWbD9T5SSvSfXVWZGn6 opNMzaXQinYYGaulIcXcrBQA1LgwTcwmhPnAM3tE1Pa8gK07NrFwq41WltChUjhRfPPT og+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=u6sSNMbV7AWggVafGl+9EjQ2biXDsQUnKCQVNFxppZI=; b=CyDxbl2Zxi+Wc+t7O0NdeEfNo0HNdRRq3jYSRXjqdbWF54osiDErID8TbpooHuUHNc DwWbAxF5ipp/Z5pScxx2ew9FIyj7XITewGsflOQAH4H5ShpH9jDBKG1Kglu4wBN75Adi UUGagrH37+Ghg9FlUnS4MKiScWBsCRBRIHAtcwivJp7gFTLzdIZJnlB+5Dpf8++bgHCL P6XvxssUjr5LmBKZtnkw3x3wi3c14f/GLJpirgyUdYN/iVlpV6lTWoBqQtc0jIQCKpSk I3mgceE3HGf8JBPGM5zzqPr2snv8mXsMuypeUE3/iGk+c+uyrp5Q4KvCRuWTVQooWd2E ShKw== X-Gm-Message-State: AHQUAua8jZCyLHEWmmut3EkbXa6s+1AXrCxJHHMojHV8AnSxFMY01FyX msV1XSfrGK44amQ9vtffIYuCqut5hUT9miVkDzxrGQ== X-Google-Smtp-Source: AHgI3IaGu5lPTLgnKxyLRYH7+d9aDXJUe9G+1bVd58Wiwe+CGgYoA3oftPEXGNQsHL/myYpBiHh871XHZ5UnWjp7eSE= X-Received: by 2002:a37:9201:: with SMTP id u1mr10288653qkd.258.1551028024496; Sun, 24 Feb 2019 09:07:04 -0800 (PST) MIME-Version: 1.0 References: <20190222033924.GA25285@troutmask.apl.washington.edu> <20190222060410.GA25817@troutmask.apl.washington.edu> <20190223032644.GA14058@troutmask.apl.washington.edu> <20190223091931.GE2420@kib.kiev.ua> <20190223163619.GA18805@troutmask.apl.washington.edu> <20190224012851.GA21748@troutmask.apl.washington.edu> <20190224102726.746adb9f@kalimero.tijl.coosemans.org> In-Reply-To: <20190224102726.746adb9f@kalimero.tijl.coosemans.org> From: Warner Losh Date: Sun, 24 Feb 2019 10:06:53 -0700 Message-ID: Subject: Re: r343567 aka PAE vs non-PAE merge breaks i386 freebsd To: =?UTF-8?Q?T=C4=B3l_Coosemans?= Cc: Steve Kargl , Konstantin Belousov , FreeBSD Current , Peter Holm , Mark Johnston X-Rspamd-Queue-Id: 6B1A275109 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=csoE0+ja X-Spamd-Result: default: False [-5.62 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT1.aspmx.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[0.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.85)[-0.847,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; IP_SCORE(-2.76)[ip: (-9.10), ipnet: 2607:f8b0::/32(-2.65), asn: 15169(-2.00), country: US(-0.07)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Feb 2019 17:07:06 -0000 On Sun, Feb 24, 2019 at 2:27 AM T=C4=B3l Coosemans wrote= : > On Sat, 23 Feb 2019 17:28:51 -0800 Steve Kargl > wrote: > > On Sat, Feb 23, 2019 at 12:03:58PM -0700, Warner Losh wrote: > >> On Sat, Feb 23, 2019 at 10:57 AM Steve Kargl > >> wrote: > >>> Supposely, the laptop only has 4 GB of memory. Not sure how > >>> it finds memory above 4 GB. > >> > >> Some older chipsets had a 'hole' in memory that they mapped the PCI bu= s > >> into and then remapped RAM in that range up above the 4GB boundary. > That's > >> how it can find memory above 4GB when you have only 4GB of RAM. I hit = it > >> with the PC Card stuff I did back in the day since it broke certain > >> heuristics I had in the code that turned out to be unwise for many > reasons > >> (not just this one). I don't recall all the details, since it's been s= o > >> long ago. > >> > >> So I think kib@ is right when he highlights > >>> +0x0000000100000000 - 0x000000011ffe7fff, 536772608 bytes (131048 > pages) > >> > >> as the memory, since this is indeed above the 4GB limit. It's about > 128k > >> of 4k pages (just shy of the 131072 I'd expect), which is a surprising= ly > >> round number. Also one that's easy to implement in hardware. So it > >> certainly "smells" the same... > >> > >> That's why I agree with others that hw.above4g_allow=3D0 is worth a sh= ot, > for > >> at least diagnostic purposes. This memory wasn't used before and if it= 's > >> used now by the drm drivers, and those aren't PAE safe (meaning they > cope > >> with allocations beyond 4GB), then that's quite useful to know. Or may= be > >> it's a different driver hating things and stomping on video memory due > to > >> wrap around. > > > > Thanks for the explanation. Here's an update. TL;DR: xorg is > > up and running; drm-legacy-kmod seems to be unsafe/unaware of > > PAE. > > > > Build world/kernel, drm-legacy-kmod, minimum needed ports for xorg. > > Kernel is unmodified GENERIC. > > > > Reboot without setting anything in /boot/loader.conf > > > > % sysctl -a | grep above > > % sysctl -a | grep pae > > vm.pmap.pae_mode: 1 > > % kldload /boot/modules/i915kms.ko > > > > Black screen of death. Did not even get to running xinit. > > > > Hard reset to single user mode. > > > > # fsck -y > > # mount -a > > # vi /boot/loader.conf. > > (Add hw.above4g_allow=3D0) > > # sync > > # shutdown -r now > > > > % sysctl -a | grep above > > % sysctl -a | grep pae > > vm.pmap.pae_mode: 1 > > % cat /boot/loader.conf > > if_ath_load=3D"YES" > > if_ath_pci_load=3D"YES" > > cpuctl_load=3D"YES" > > hw.above4g_allow=3D0 > > % kldload /boot/modules/i915kms.ko > > > > Switch to vt3, login as normal user. > > > > % startx -- -depth 24 >& ~/tmp/.x.out > > > > Xorg is up and running. Not sure why my first attempt at using > > hw.above4g_allow=3D0 did not work. Perhaps, mismatch between the xorg > > bits and kernel/world bits. > > > > % sysctl -a | grep mem > > vm.lowmem_period: 10 > > vm.kmem_map_free: 1669365760 > > vm.kmem_map_size: 41910272 > > vm.kmem_size_scale: 1 > > vm.kmem_size_max: 1711276032 > > vm.kmem_size_min: 12582912 > > vm.kmem_zmax: 65536 > > vm.kmem_size: 1711276032 > > hw.physmem: 3715489792 > > hw.usermem: 3592175616 > > hw.realmem: 4294963200 > > > > % dmesg | grep memory > > real memory =3D 4294967296 (4096 MB) > > avail memory =3D 3637673984 (3469 MB) > > agp0: aperture size is 256M, detected 7676k stolen memory > > > > The pre-r343567 dmesg has > > > > real memory =3D 4294967296 (4096 MB) > > avail memory =3D 3639914496 (3471 MB) > > > > I can live with 2 MB loss. > > > > Conclusion, drm-legacy-kmod is not PAE safe/aware. > > > > Probably want to put something in /usr/src about possible > > problems with new pmap.h on i386 FreeBSD. > > Now it would be interesting to do the same tests with drm-current-kmod. > Maybe I missed it, but Steve, did you run the patched in a different way tests that I suggested? Replacing the limits with 0xffffffff for testing purposes to ensure that drm isn't saying it can cope with larger addresses? That might help narrow down what the problem here one more level than "It's PAE". Warner