From owner-dev-commits-src-all@freebsd.org Fri May 21 16:42:46 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 527F163760B for ; Fri, 21 May 2021 16:42:46 +0000 (UTC) (envelope-from mw@semihalf.com) Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fmsps0yk1z3rGB for ; Fri, 21 May 2021 16:42:45 +0000 (UTC) (envelope-from mw@semihalf.com) Received: by mail-qt1-x835.google.com with SMTP id c10so15715290qtx.10 for ; Fri, 21 May 2021 09:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=xL4+f1eDR9mvoxSgmtHNM8ZxvRYze4yry2BPswOFyFo=; b=avFkGPt6IoclB8Cn3+IZWCEEpJOnNXazjQd72+/mIwH2SZ/93zqeVxyLq+TAl3B3Im Qu8S/DHTU5bj9DQVKUr0ztnG8F6uieIJQ2Kto6bD+2T5cH+bcu6ZtthsRQK4nR9Rcb1d vJZdBnqbNytPewOA4BbAOWiotRvda78vnhexFlsi6www6zLMGXVd09aW9mD3u4BT5FrH Xt+fYvvp5z/YSrCKNlwzq+JUWRKqdx43cYNoM5PHg3AEyw9bEOK5/is2BBt0z1qDh2hz rrRbK+dzlA+dgO4Ju3ONOvGvNci3RRmIYghrx6xT1SsKY+hT7ZhUyn7x3mDVECG+dAyB 9GbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=xL4+f1eDR9mvoxSgmtHNM8ZxvRYze4yry2BPswOFyFo=; b=fZMXggG9t0ea3ejPs/6dis+92ktDXK6CgN2W7kJG16NF3heybVdLEZ5pOHmI8W8wyM Ie6dJ87ZCW03YGT4dIxSRbxiicOwPodRn37fa+cFjycMsApYoHBviQuSx3gfrQCn4JgJ c9tEyMBRdCXdM3ML3ykQW1n+E3ky6ZBSgMcYvo6fxyza1LkAz6bf+hBPTBYLq4dV+EUi 17Nj4QFFY0fTxAYeikqmlImXu5khtG5jcoLO04QhASsKM8ix3BkKmoX6Ogk4D4dddtHg w5yFyLiv3GhRwRjxt46nVexKMY2qMkcXieAKRpSSqg7KrQqaETvkWJ67dhJHZdL5VOWP gLew== X-Gm-Message-State: AOAM532ZXH+afXeksgV2icZ03bUSefjEy05pxCFFvL2Zdj1vYtKVRQNA zPVDPXsHU17UYzVHA9lbjSY8Ea8NpCobN0WwQtGDwQ== X-Google-Smtp-Source: ABdhPJxFPOxWW9DA7BniEnhfPdPIEu0Ms8IaF/0vnYHxx4y+mBo/qlV0p1rrzlqPKUVCiKnWkq3n+NYzqs3BRVAx2G4= X-Received: by 2002:a05:622a:202:: with SMTP id b2mr11988511qtx.343.1621615363785; Fri, 21 May 2021 09:42:43 -0700 (PDT) MIME-Version: 1.0 References: <202105211334.14LDYqoa004343@gitrepo.freebsd.org> <04F25FD0-7863-4AC1-A257-EF0F1EB90659@freebsd.org> <02078965-24BE-4F23-92D5-5E8E54A0C3E7@freebsd.org> <202105211446.14LEk8kZ009266@slippy.cwsent.com> In-Reply-To: <202105211446.14LEk8kZ009266@slippy.cwsent.com> From: Marcin Wojtas Date: Fri, 21 May 2021 18:42:31 +0200 Message-ID: Subject: Re: git: af949c590bd8 - main - Disable stack gap for ntpd during build. To: Cy Schubert Cc: Jessica Clarke , shawn.webb@hardenedbsd.org, Marcin Wojtas , "src-committers@freebsd.org" , "dev-commits-src-all@freebsd.org" , "dev-commits-src-main@freebsd.org" , =?UTF-8?Q?Dawid_G=C3=B3recki?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Fmsps0yk1z3rGB X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=semihalf-com.20150623.gappssmtp.com header.s=20150623 header.b=avFkGPt6; dmarc=none; spf=none (mx1.freebsd.org: domain of mw@semihalf.com has no SPF policy when checking 2607:f8b0:4864:20::835) smtp.mailfrom=mw@semihalf.com X-Spamd-Result: default: False [-3.26 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[semihalf-com.20150623.gappssmtp.com:s=20150623]; FREEFALL_USER(0.00)[mw]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.963]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[dev-commits-src-all@freebsd.org]; DMARC_NA(0.00)[semihalf.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::835:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[semihalf-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::835:from]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::835:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[dev-commits-src-all] X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2021 16:42:46 -0000 Hi Cy, pt., 21 maj 2021 o 16:46 Cy Schubert napisa=C5= =82(a): > > In message <02078965-24BE-4F23-92D5-5E8E54A0C3E7@freebsd.org>, Jessica > Clarke w > rites: > > > On 21 May 2021, at 15:11, Marcin Wojtas wrote: > > > > > > Hi Jess > > > > > > pt., 21 maj 2021 o 15:39 Jessica Clarke napisa= =C5=82(a): > > >> > > >> On 21 May 2021, at 14:34, Marcin Wojtas wrote: > > >>> > > >>> The branch main has been updated by mw: > > >>> > > >>> URL: https://cgit.FreeBSD.org/src/commit/?id=3Daf949c590bd8a00a5973= b5875d7e > > 0fa6832ea64a > > >>> > > >>> commit af949c590bd8a00a5973b5875d7e0fa6832ea64a > > >>> Author: Marcin Wojtas > > >>> AuthorDate: 2021-05-21 09:29:22 +0000 > > >>> Commit: Marcin Wojtas > > >>> CommitDate: 2021-05-21 13:33:06 +0000 > > >>> > > >>> Disable stack gap for ntpd during build. > > >>> > > >>> When starting, ntpd calls setrlimit(2) to limit maximum size of i= ts > > >>> stack. The stack limit chosen by ntpd is 200K, so when stack gap > > >>> is enabled, the stack gap is larger than this limit, which result= s > > >>> in ntpd crashing. > > >> > > >> Isn=E2=80=99t the bug that the unusable gap counts as usage? > > >> > > >> Jess > > >> > > > > > > An alternative solution was submitted > > > (https://reviews.freebsd.org/D29832), so that to extend the limit for > > > ntpd, but eventually it was recommended to simple disable the stack > > > gap for it until it's fixed upstream (see the last comment in the > > > linked revision). > > > > That=E2=80=99s my point, there is nothing to =E2=80=9Cfix=E2=80=9D upst= ream. NTPD uses less tha > > n 200K > > of stack, thus it is perfectly reasonable for it to set its limit to th= at. Th > > e > > fact that FreeBSD decides to count an arbitrary, non-deterministic amou= nt of > > additional unusable virtual address space towards that limit is not its= fault > > , > > but a bug in FreeBSD that needs to be fixed as it=E2=80=99s entirely un= reasonable f > > or > > applications to have to account for that. > > This latest problem is not stack gap. It is PIE. > I have to disagree. ntpd does not start because of stack gap, not PIE, even though it may seem like PIE causes this. This is due to the fact that stack gap is disabled if PIE is disabled. Because of that value of sysctl kern.elf64.aslr.stack_gap does not matter when kern.elf64.aslr.pie_enable is set to 0. When pie_enabled is set to 1 and stack gap is enabled, then ntpd fails to start, but when pie_enabled is set to 1 and stack_gap is set to 0, then ntpd starts without any issue. We verified this on FreeBSD-CURRENT snapshot from 2021-05-20. The fact that this is a stack gap issue can be verified using following procedure: 1. Install FreeBSD-CURRENT snapshot from 2021-05-20 using default configuration. 2. On a newly installed system start ntpd. With default configuration it should start successfully. 3. Set sysctl kern.elf64.aslr.pie_enable=3D1 and start ntpd. This time ntpd should fail. An entry indicating that ntpd was killed because of signal 11 should be visible in /var/log/messages. 4. Set sysctl kern.elf64.aslr.stack_gap=3D0 and start ntpd once again. This time ntpd should start even though pie_enable is set to 1. Exact log from the boot it was tested: root@freebsd-ntpd-test:~ # sysctl -a | grep aslr kern.elf32.aslr.stack_gap: 3 kern.elf32.aslr.honor_sbrk: 1 kern.elf32.aslr.pie_enable: 0 kern.elf32.aslr.enable: 0 kern.elf64.aslr.stack_gap: 3 kern.elf64.aslr.honor_sbrk: 1 kern.elf64.aslr.pie_enable: 0 kern.elf64.aslr.enable: 0 vm.aslr_restarts: 0 root@freebsd-ntpd-test:~ # ntpd root@freebsd-ntpd-test:~ # ps aux | grep ntpd root 826 0.0 0.2 22060 6960 - Ss 17:38 0:00.01 ntpd root 828 0.0 0.1 12976 2416 0 S+ 17:38 0:00.00 grep ntpd root@freebsd-ntpd-test:~ # killall ntpd root@freebsd-ntpd-test:~ # ps aux | grep ntpd root 831 0.0 0.1 12976 2416 0 S+ 17:38 0:00.00 grep ntpd root@freebsd-ntpd-test:~ # sysctl kern.elf64.aslr.pie_enable=3D1 kern.elf64.aslr.pie_enable: 0 -> 1 root@freebsd-ntpd-test:~ # ntpd root@freebsd-ntpd-test:~ # ps aux | grep ntpd root 836 0.0 0.1 14128 2452 0 S+ 17:39 0:00.00 grep ntpd root@freebsd-ntpd-test:~ # cat /var/log/messages | tail May 21 17:38:25 freebsd-ntpd-test ntpd[826]: ntpd exiting on signal 15 (Terminated) May 21 17:39:14 freebsd-ntpd-test ntpd[833]: ntpd 4.2.8p15-a (1): Starting May 21 17:39:14 freebsd-ntpd-test ntpd[833]: Command line: ntpd May 21 17:39:14 freebsd-ntpd-test ntpd[833]: ---------------------------------------------------- May 21 17:39:14 freebsd-ntpd-test ntpd[833]: ntp-4 is maintained by Network Time Foundation, May 21 17:39:14 freebsd-ntpd-test ntpd[833]: Inc. (NTF), a non-profit 501(c)(3) public-benefit May 21 17:39:14 freebsd-ntpd-test ntpd[833]: corporation. Support and training for ntp-4 are May 21 17:39:14 freebsd-ntpd-test ntpd[833]: available at https://www.nwtime.org/support May 21 17:39:14 freebsd-ntpd-test ntpd[833]: ---------------------------------------------------- May 21 17:39:14 freebsd-ntpd-test kernel: pid 834 (ntpd), jid 0, uid 0: exited on signal 11 (core dumped) root@freebsd-ntpd-test:~ # sysctl kern.elf64.aslr.stack_gap=3D0 kern.elf64.aslr.stack_gap: 3 -> 0 root@freebsd-ntpd-test:~ # sysctl -a | grep aslr kern.elf32.aslr.stack_gap: 3 kern.elf32.aslr.honor_sbrk: 1 kern.elf32.aslr.pie_enable: 0 kern.elf32.aslr.enable: 0 kern.elf64.aslr.stack_gap: 0 kern.elf64.aslr.honor_sbrk: 1 kern.elf64.aslr.pie_enable: 1 kern.elf64.aslr.enable: 0 vm.aslr_restarts: 1 root@freebsd-ntpd-test:~ # ntpd root@freebsd-ntpd-test:~ # ps aux | grep ntpd root 845 0.0 0.2 22060 6924 - Ss 17:40 0:00.01 ntpd root 847 0.0 0.1 12976 2440 0 S+ 17:40 0:00.00 grep ntpd root@freebsd-ntpd-test:~ # cat /var/log/messages | tail May 21 17:39:14 freebsd-ntpd-test kernel: pid 834 (ntpd), jid 0, uid 0: exited on signal 11 (core dumped) May 21 17:40:52 freebsd-ntpd-test ntpd[844]: ntpd 4.2.8p15-a (1): Starting May 21 17:40:52 freebsd-ntpd-test ntpd[844]: Command line: ntpd May 21 17:40:52 freebsd-ntpd-test ntpd[844]: ---------------------------------------------------- May 21 17:40:52 freebsd-ntpd-test ntpd[844]: ntp-4 is maintained by Network Time Foundation, May 21 17:40:52 freebsd-ntpd-test ntpd[844]: Inc. (NTF), a non-profit 501(c)(3) public-benefit May 21 17:40:52 freebsd-ntpd-test ntpd[844]: corporation. Support and training for ntp-4 are May 21 17:40:52 freebsd-ntpd-test ntpd[844]: available at https://www.nwtime.org/support May 21 17:40:52 freebsd-ntpd-test ntpd[844]: ---------------------------------------------------- May 21 17:40:52 freebsd-ntpd-test ntpd[845]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): stat failed: No such file or directory root@freebsd-ntpd-test:~ # killall ntpd Best regards, Marcin