From owner-dev-commits-src-main@freebsd.org Fri May 21 17:53:57 2021 Return-Path: Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE34F638BE5; Fri, 21 May 2021 17:53:57 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmvP12DPkz4v8P; Fri, 21 May 2021 17:53:57 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.66.148.124]) by shaw.ca with ESMTPA id k9LNlIwztMrQqk9LOlAhas; Fri, 21 May 2021 11:53:55 -0600 X-Authority-Analysis: v=2.4 cv=Nv6yz+RJ c=1 sm=1 tr=0 ts=60a7f3b3 a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=8nJEP1OIZ-IA:10 a=5FLXtPjwQuUA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=oCJs8q-oAAAA:8 a=EkcXrb_YAAAA:8 a=ZEWdpI7IREN6c40-odoA:9 a=wPNLvfGTeEIA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 a=qUF70SbvcHBaGhGVny9j:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id F104E1E5; Fri, 21 May 2021 10:53:51 -0700 (PDT) Received: from slippy (localhost [127.0.0.1]) by slippy.cwsent.com (8.16.1/8.16.1) with ESMTP id 14LHrpAN004663; Fri, 21 May 2021 10:53:51 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <202105211753.14LHrpAN004663@slippy.cwsent.com> X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Marcin Wojtas cc: Cy Schubert , Jessica Clarke , shawn.webb@hardenedbsd.org, Marcin Wojtas , "src-committers@freebsd.org" , "dev-commits-src-all@freebsd.org" , "dev-commits-src-main@freebsd.org" , =?UTF-8?Q?Dawid_G=C3=B3recki?= Subject: Re: git: af949c590bd8 - main - Disable stack gap for ntpd during build. In-reply-to: References: <202105211334.14LDYqoa004343@gitrepo.freebsd.org> <04F25FD0-7863-4AC1-A257-EF0F1EB90659@freebsd.org> <02078965-24BE-4F23-92D5-5E8E54A0C3E7@freebsd.org> <202105211446.14LEk8kZ009266@slippy.cwsent.com> Comments: In-reply-to Marcin Wojtas message dated "Fri, 21 May 2021 18:42:31 +0200." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Fri, 21 May 2021 10:53:51 -0700 X-CMAE-Envelope: MS4xfEgYXFa9V04gso1yKnFvwL1O9dbQnSXXmT7touAA2JRX63WY996JJfiguRBuN9sbkyjMMS7kqk84SvjqEYgH87VvnD5uTDg/FdtLkZC7AVo0xivojVC8 sHkeEIp8JDBhWLVGUOsja6v+vweRGFNIzPHYOYdQpI84GhYtYM3X5F8uhr+WleSmLjSmlm2zg3q4GFkfVfXNt2zGl9uzkFPaox5zM9jQLP4Hur7vlBxP81jD /SJHn4LrYRRYcew99l9pGAXAS/DD/8Kz4AAxiZIv0rTmF/ni7li5jVzzHAgW/k/4OqUCL5SOKOhjwWAElUKd71VjyaOx8uGDgcJWloty6AILHOq4neiGB5FK pFwG44PJ9y02f/DZ7QGtNqdqfvSTHNQ/bQxk4vnDSViVLSIanu2K2M80rBo7T+h5AdAy0bC5xzqZkNJU3S1URwFwA200tg== X-Rspamd-Queue-Id: 4FmvP12DPkz4v8P X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: dev-commits-src-main@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for the main branch of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2021 17:53:57 -0000 In message , Marcin Wojtas writes: > Hi Cy, > > pt., 21 maj 2021 o 16:46 Cy Schubert napisał(a): > > > > In message <02078965-24BE-4F23-92D5-5E8E54A0C3E7@freebsd.org>, Jessica > > Clarke w > > rites: > > > > On 21 May 2021, at 15:11, Marcin Wojtas wrote: > > > > > > > > Hi Jess > > > > > > > > pt., 21 maj 2021 o 15:39 Jessica Clarke napisał(a > ): > > > >> > > > >> On 21 May 2021, at 14:34, Marcin Wojtas wrote: > > > >>> > > > >>> The branch main has been updated by mw: > > > >>> > > > >>> URL: https://cgit.FreeBSD.org/src/commit/?id=af949c590bd8a00a5973b587 > 5d7e > > > 0fa6832ea64a > > > >>> > > > >>> commit af949c590bd8a00a5973b5875d7e0fa6832ea64a > > > >>> Author: Marcin Wojtas > > > >>> AuthorDate: 2021-05-21 09:29:22 +0000 > > > >>> Commit: Marcin Wojtas > > > >>> CommitDate: 2021-05-21 13:33:06 +0000 > > > >>> > > > >>> Disable stack gap for ntpd during build. > > > >>> > > > >>> When starting, ntpd calls setrlimit(2) to limit maximum size of its > > > >>> stack. The stack limit chosen by ntpd is 200K, so when stack gap > > > >>> is enabled, the stack gap is larger than this limit, which results > > > >>> in ntpd crashing. > > > >> > > > >> Isn’t the bug that the unusable gap counts as usage? > > > >> > > > >> Jess > > > >> > > > > > > > > An alternative solution was submitted > > > > (https://reviews.freebsd.org/D29832), so that to extend the limit for > > > > ntpd, but eventually it was recommended to simple disable the stack > > > > gap for it until it's fixed upstream (see the last comment in the > > > > linked revision). > > > > > > That’s my point, there is nothing to “fix” upstream. NTPD uses less > tha > > > n 200K > > > of stack, thus it is perfectly reasonable for it to set its limit to that > . Th > > > e > > > fact that FreeBSD decides to count an arbitrary, non-deterministic amount > of > > > additional unusable virtual address space towards that limit is not its f > ault > > > , > > > but a bug in FreeBSD that needs to be fixed as it’s entirely unreasonab > le f > > > or > > > applications to have to account for that. > > > > This latest problem is not stack gap. It is PIE. > > > > I have to disagree. We are talking cross purposes. Your examples later on in your email prove my point. > ntpd does not start because of stack gap, not PIE, even though it may > seem like PIE causes this. This is due to the fact that stack gap is > disabled if PIE is disabled. Because of that value of sysctl > kern.elf64.aslr.stack_gap does not matter when kern.elf64.aslr.pie_enable > is set to 0. When pie_enabled is set to 1 and stack gap is enabled, then > ntpd fails to start, but when pie_enabled is set to 1 and stack_gap > is set to 0, then ntpd starts without any issue. We verified this on > FreeBSD-CURRENT snapshot from 2021-05-20. I verified the PIE problem on a -CURRENT as of my comments in the review. Enabling stack gap and disabling PIE resolved the issue. The reason for stack gap is not a problem is that ntpd disables stack gap at line 441 of ntpd.c. Furthermore enabling stack gap and disabling PIE circumvents the problem. I tested this myself and left that note in the review. Enable stack gap and disable PIE: It works. But look at line 441 of ntpd.c to see stack gap disabled before ntpd forks itself. > > The fact that this is a stack gap issue can be verified using following > procedure: > 1. Install FreeBSD-CURRENT snapshot from 2021-05-20 using default > configuration. > 2. On a newly installed system start ntpd. With default configuration > it should start successfully. > 3. Set sysctl kern.elf64.aslr.pie_enable=1 and start ntpd. This time ntpd > should fail. An entry indicating that ntpd was killed because of signal > 11 should be visible in /var/log/messages. > 4. Set sysctl kern.elf64.aslr.stack_gap=0 and start ntpd once again. This > time ntpd should start even though pie_enable is set to 1. > > Exact log from the boot it was tested: > root@freebsd-ntpd-test:~ # sysctl -a | grep aslr > kern.elf32.aslr.stack_gap: 3 > kern.elf32.aslr.honor_sbrk: 1 > kern.elf32.aslr.pie_enable: 0 > kern.elf32.aslr.enable: 0 > kern.elf64.aslr.stack_gap: 3 > kern.elf64.aslr.honor_sbrk: 1 > kern.elf64.aslr.pie_enable: 0 > kern.elf64.aslr.enable: 0 > vm.aslr_restarts: 0 > root@freebsd-ntpd-test:~ # ntpd > root@freebsd-ntpd-test:~ # ps aux | grep ntpd > root 826 0.0 0.2 22060 6960 - Ss 17:38 0:00.01 ntpd > root 828 0.0 0.1 12976 2416 0 S+ 17:38 0:00.00 grep ntpd > root@freebsd-ntpd-test:~ # killall ntpd > root@freebsd-ntpd-test:~ # ps aux | grep ntpd > root 831 0.0 0.1 12976 2416 0 S+ 17:38 0:00.00 grep ntpd > root@freebsd-ntpd-test:~ # sysctl kern.elf64.aslr.pie_enable=1 > kern.elf64.aslr.pie_enable: 0 -> 1 This causes the problem. > root@freebsd-ntpd-test:~ # ntpd > root@freebsd-ntpd-test:~ # ps aux | grep ntpd > root 836 0.0 0.1 14128 2452 0 S+ 17:39 0:00.00 grep ntpd > root@freebsd-ntpd-test:~ # cat /var/log/messages | tail > May 21 17:38:25 freebsd-ntpd-test ntpd[826]: ntpd exiting on signal 15 > (Terminated) > May 21 17:39:14 freebsd-ntpd-test ntpd[833]: ntpd 4.2.8p15-a (1): Starting > May 21 17:39:14 freebsd-ntpd-test ntpd[833]: Command line: ntpd > May 21 17:39:14 freebsd-ntpd-test ntpd[833]: > ---------------------------------------------------- > May 21 17:39:14 freebsd-ntpd-test ntpd[833]: ntp-4 is maintained by > Network Time Foundation, > May 21 17:39:14 freebsd-ntpd-test ntpd[833]: Inc. (NTF), a non-profit > 501(c)(3) public-benefit > May 21 17:39:14 freebsd-ntpd-test ntpd[833]: corporation. Support and > training for ntp-4 are > May 21 17:39:14 freebsd-ntpd-test ntpd[833]: available at > https://www.nwtime.org/support > May 21 17:39:14 freebsd-ntpd-test ntpd[833]: > ---------------------------------------------------- > May 21 17:39:14 freebsd-ntpd-test kernel: pid 834 (ntpd), jid 0, uid > 0: exited on signal 11 (core dumped) > root@freebsd-ntpd-test:~ # sysctl kern.elf64.aslr.stack_gap=0 > kern.elf64.aslr.stack_gap: 3 -> 0 > root@freebsd-ntpd-test:~ # sysctl -a | grep aslr > kern.elf32.aslr.stack_gap: 3 > kern.elf32.aslr.honor_sbrk: 1 > kern.elf32.aslr.pie_enable: 0 > kern.elf32.aslr.enable: 0 > kern.elf64.aslr.stack_gap: 0 > kern.elf64.aslr.honor_sbrk: 1 > kern.elf64.aslr.pie_enable: 1 This is the problem. > kern.elf64.aslr.enable: 0 > vm.aslr_restarts: 1 > root@freebsd-ntpd-test:~ # ntpd > root@freebsd-ntpd-test:~ # ps aux | grep ntpd > root 845 0.0 0.2 22060 6924 - Ss 17:40 0:00.01 ntpd > root 847 0.0 0.1 12976 2440 0 S+ 17:40 0:00.00 grep ntpd > root@freebsd-ntpd-test:~ # cat /var/log/messages | tail > May 21 17:39:14 freebsd-ntpd-test kernel: pid 834 (ntpd), jid 0, uid > 0: exited on signal 11 (core dumped) > May 21 17:40:52 freebsd-ntpd-test ntpd[844]: ntpd 4.2.8p15-a (1): Starting > May 21 17:40:52 freebsd-ntpd-test ntpd[844]: Command line: ntpd > May 21 17:40:52 freebsd-ntpd-test ntpd[844]: > ---------------------------------------------------- > May 21 17:40:52 freebsd-ntpd-test ntpd[844]: ntp-4 is maintained by > Network Time Foundation, > May 21 17:40:52 freebsd-ntpd-test ntpd[844]: Inc. (NTF), a non-profit > 501(c)(3) public-benefit > May 21 17:40:52 freebsd-ntpd-test ntpd[844]: corporation. Support and > training for ntp-4 are > May 21 17:40:52 freebsd-ntpd-test ntpd[844]: available at > https://www.nwtime.org/support > May 21 17:40:52 freebsd-ntpd-test ntpd[844]: > ---------------------------------------------------- > May 21 17:40:52 freebsd-ntpd-test ntpd[845]: leapsecond file > ('/var/db/ntpd.leap-seconds.list'): stat failed: No such file or > directory > root@freebsd-ntpd-test:~ # killall ntpd > > Best regards, > Marcin Running on my firewall, which has had this same ASLR configuration for about a year. cwfw# sysctl kern.elf64.aslr kern.elf64.aslr.stack_gap: 3 kern.elf64.aslr.honor_sbrk: 1 kern.elf64.aslr.pie_enable: 0 kern.elf64.aslr.enable: 1 cwfw# ps auxww | grep ntpd ntpd 1499 0.0 0.1 22044 5776 - Ss 09:30 0:00.28 /usr/sbin/ntpd -p /var/db/ntp/ntpd.pid -c /etc/ntp.conf -f /var/db/ntp/ntpd.drift -g root 3032 0.0 0.0 13044 2384 0 S+ 10:49 0:00.00 grep ntpd cwfw# uptime 10:49AM up 1:20, 1 user, load averages: 1.06, 1.02, 0.97 cwfw# uname -a FreeBSD cwfw 14.0-CURRENT FreeBSD 14.0-CURRENT #151 komquats-n246804-af949c590bd8-dirty: Fri May 21 07:09:32 PDT 2021 root@cwsys:/export/obj/opt/src/git-src/amd64.amd64/sys/PROD2 amd64 cwfw# My laptop: slippy# sysctl kern.elf64.aslr kern.elf64.aslr.stack_gap: 3 kern.elf64.aslr.honor_sbrk: 1 kern.elf64.aslr.pie_enable: 0 kern.elf64.aslr.enable: 1 slippy# ps auxww | grep ntpd ntpd 2100 0.0 0.1 22036 8600 - Ss 09:35 0:00.27 /usr/sbin/ntpd -p /var/db/ntp/ntpd.pid -c /etc/ntp.conf -f /var/db/ntp/ntpd.drift -g root 4632 0.0 0.0 13040 2724 1 S+ 10:51 0:00.00 grep ntpd slippy# uptime 10:51AM up 1:17, 0 users, load averages: 0.11, 0.16, 0.16 slippy# uname -a FreeBSD slippy 14.0-CURRENT FreeBSD 14.0-CURRENT #155 komquats-n246804-af949c590bd8-dirty: Fri May 21 07:07:22 PDT 2021 root@cwsys:/export/obj/opt/src/git-src/amd64.amd64/sys/BREAK amd64 slippy# One of my poudriere machines: cwsys# sysctl kern.elf64.aslr kern.elf64.aslr.stack_gap: 3 kern.elf64.aslr.honor_sbrk: 1 kern.elf64.aslr.pie_enable: 0 kern.elf64.aslr.enable: 1 cwsys# ps auxww | grep ntpd ntpd 4039 0.0 0.1 22040 7340 - Ss 09:34 0:00.46 /usr/sbin/ntpd -p /var/db/ntp/ntpd.pid -c /etc/ntp.conf -f /var/db/ntp/ntpd.drift -g root 6385 0.0 0.0 13044 2712 2 S+ 10:52 0:00.01 grep ntpd cwsys# uptime 10:52AM up 1:19, 2 users, load averages: 0.26, 0.25, 0.24 cwsys# uname -a FreeBSD cwsys 14.0-CURRENT FreeBSD 14.0-CURRENT #155 komquats-n246804-af949c590bd8-dirty: Fri May 21 07:07:22 PDT 2021 root@cwsys:/export/obj/opt/src/git-src/amd64.amd64/sys/BREAK amd64 cwsys# Three examples of stack gap enabled and PIE disabled. When I enable PIE, ntpd fails. -- Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org The need of the many outweighs the greed of the few.