From nobody Sun Oct 16 15:08:30 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Mr3RR5x7Xz4fQPf for ; Sun, 16 Oct 2022 15:08:35 +0000 (UTC) (envelope-from paulf2718@gmail.com) Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Mr3RR1bLxz3VDL for ; Sun, 16 Oct 2022 15:08:35 +0000 (UTC) (envelope-from paulf2718@gmail.com) Received: by mail-wm1-x335.google.com with SMTP id y10so6865998wma.0 for ; Sun, 16 Oct 2022 08:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=JwBfmb0/ZrNi0xcX3o5A2tTBEGJBzjL6+rGeICu4pTg=; b=Oz3HhpQjfpSloM3oTkuhB6J229cKw01brM9qMnNFKIy5sH4iTySijBp8oDWg7Vck0a /Dad7rpSFSu4UHPB/2lVBb4yE7GOf2aDv6CxpNNtS5xMOXmxgI8Z9tv76JL+w/O6sDVX zfB1O7a9fyaLNlPhg2wPc3seHkNm3yJaXxdnZu+TtcpfudYGYe5P8VqJWOKTL9EN9cbk AoBJzoaDo8oqH1agOVp3CpNXoiAuA/nwx2/mXUJHjXblVedhgautO17ofFp0bNec9MPO O1NKp19/tybpPr81m6dpkKz8Z583yjP2jfwBtBoKoXg+b1hNDycf8BdDBDaVUSYgQrkD MhLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=JwBfmb0/ZrNi0xcX3o5A2tTBEGJBzjL6+rGeICu4pTg=; b=5dIyUZG91wtVIUQeGaaYcttYuMmfJLFDggyf5QYadgxA0iJ1EsiKd3z0sGcwchqeYU PXY7MghsEl0WOxKHrX7rbOZEVnBNp6MiaQeinCFDJs1Qt+fbR9nrVRqTh4+XkjnQBKy7 /3Gu9aLMngqv8460I2oLzo1UaG+juY6FtAINlBqeZqPitwDyr9zUxE0mFfTloIS+uifq lUSr0LBX2cSAL3LrgTHZIL6wrdZOrdJnkKeWal3qkehriz5sjz7p2ytrUHqQP+DNp+Lm nC0rt5caSDX9JERkVXGFh9zTswwLPlQ9farQGXxFwFrvn1yIl8NExl22bbnL3PDcaDVv JW0A== X-Gm-Message-State: ACrzQf3pT/W/WCWtRa7Uxvz0hqHwQ1EPVgojRUdWCcpFMXQ6cmnQXuL3 wmjkMZxEk2/DLmfB1O/7QMxJBX7h2dw= X-Google-Smtp-Source: AMsMyM6Y4ljGvS9pHbQ6MpX+HBrwiei7KFRqg59t3xwDJm8vrQmnd4/W3En0NYry3r8A+vhKT6nsTQ== X-Received: by 2002:a05:600c:1e17:b0:3c6:bc31:1f3d with SMTP id ay23-20020a05600c1e1700b003c6bc311f3dmr16608524wmb.52.1665932912119; Sun, 16 Oct 2022 08:08:32 -0700 (PDT) Received: from [192.168.1.28] (lfbn-lyo-1-263-217.w2-7.abo.wanadoo.fr. [2.7.103.217]) by smtp.gmail.com with ESMTPSA id t9-20020a05600c198900b003b4fe03c881sm13227644wmq.48.2022.10.16.08.08.31 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 16 Oct 2022 08:08:31 -0700 (PDT) Message-ID: Date: Sun, 16 Oct 2022 17:08:30 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.3.2 To: freebsd-hackers Content-Language: en-US From: Paul Floyd Subject: AMD64 14.0-CURRENT memory layout changes Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Mr3RR1bLxz3VDL X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=Oz3HhpQj; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of paulf2718@gmail.com designates 2a00:1450:4864:20::335 as permitted sender) smtp.mailfrom=paulf2718@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::335:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N Hi I just noticed that the memory layout has changed for elf binaries running on amd64 (my last attempt to setup an i386 VM failed so I can't confirm if that also changed, and I'm not yet concerned by other platforms). Here's a procstat -v for ksh93 on 13.1 on the host machine > paulf> procstat -v 1456 > PID START END PRT RES PRES REF SHD FLAG TP PATH > 1456 0x200000 0x273000 r-- 70 343 30 16 CN--- vn /usr/local/bin/ksh93 > 1456 0x273000 0x3d1000 r-x 257 343 30 16 CN--- vn /usr/local/bin/ksh93 > 1456 0x3d1000 0x3dd000 r-- 11 0 1 0 CN--- vn /usr/local/bin/ksh93 > 1456 0x3dd000 0x3de000 rw- 1 0 1 0 CN--- vn /usr/local/bin/ksh93 > 1456 0x3de000 0x3e4000 rw- 6 0 1 0 C---- vn /usr/local/bin/ksh93 > 1456 0x3e4000 0x3ec000 rw- 5 5 1 0 C---- df > 1456 0x8003de000 0x8003e4000 r-- 6 28 364 120 CN--- vn /libexec/ld-elf.so.1 > 1456 0x8003e4000 0x8003fb000 r-x 23 28 364 120 CN--- vn /libexec/ld-elf.so.1 > 1456 0x8003fb000 0x8003fc000 r-- 1 0 1 0 CN--- vn /libexec/ld-elf.so.1 > 1456 0x8003fc000 0x800411000 rw- 19 19 1 0 CN--- df > 1456 0x800420000 0x800432000 r-- 13 33 352 176 CN--- vn /lib/libm.so.5 > 1456 0x800432000 0x800459000 r-x 20 33 352 176 CN--- vn /lib/libm.so.5 > 1456 0x800459000 0x80045a000 rw- 1 0 1 0 CN--- vn /lib/libm.so.5 > 1456 0x80045a000 0x80045b000 rw- 1 0 1 0 CN--- vn /lib/libm.so.5 > 1456 0x80045b000 0x8004df000 r-- 132 380 528 284 CN--- vn /lib/libc.so.7 > 1456 0x8004df000 0x80062b000 r-x 234 380 528 284 CN--- vn /lib/libc.so.7 > 1456 0x80062b000 0x800633000 r-- 8 0 1 0 CN--- vn /lib/libc.so.7 > 1456 0x800633000 0x800634000 rw- 1 0 1 0 CN--- vn /lib/libc.so.7 > 1456 0x800634000 0x80063b000 rw- 7 0 1 0 C---- vn /lib/libc.so.7 > 1456 0x80063b000 0x8008f5000 rw- 84 84 1 0 C---- df > 1456 0x800a00000 0x801200000 rw- 13 13 1 0 CN--- df > 1456 0x7fffdffff000 0x7ffffffdf000 --- 0 0 0 0 ----- gd > 1456 0x7ffffffdf000 0x7ffffffff000 rw- 11 11 1 0 C--D- df > 1456 0x7ffffffff000 0x800000000000 r-x 1 1 125 0 ----- ph Here the stack starts at 0x7ffffffdf000 And the same on 14.0 running on a 4Gbyte VirtualBox VM > paulf@freebsd:~/valgrind $ procstat -v 62770 > PID START END PRT RES PRES REF SHD FLAG TP PATH > 62770 0x200000 0x273000 r-- 115 488 4 2 CN--- vn /usr/local/bin/ksh93 > 62770 0x273000 0x3c7000 r-x 340 488 4 2 CN--- vn /usr/local/bin/ksh93 > 62770 0x3c7000 0x3d4000 r-- 13 0 2 0 C---- vn /usr/local/bin/ksh93 > 62770 0x3d4000 0x3d5000 rw- 1 0 2 0 C---- vn /usr/local/bin/ksh93 > 62770 0x3d5000 0x3da000 rw- 5 0 1 0 C---- vn /usr/local/bin/ksh93 > 62770 0x3da000 0x3e2000 rw- 5 5 1 0 ----- sw > 62770 0x80075d000 0x82073d000 --- 0 0 0 0 ----- gd > 62770 0x82073d000 0x82075d000 rw- 14 14 1 0 ---D- sw > 62770 0x8209c8000 0x8209c9000 r-x 1 1 28 0 ----- ph > 62770 0x8217b0000 0x8217c2000 rw- 16 16 1 0 ----- sw > 62770 0x822186000 0x822210000 r-- 138 496 104 54 CN--- vn /lib/libc.so.7 > 62770 0x822210000 0x82235e000 r-x 334 496 104 54 CN--- vn /lib/libc.so.7 > 62770 0x82235e000 0x822367000 r-- 9 0 2 0 C---- vn /lib/libc.so.7 > 62770 0x822367000 0x822368000 rw- 1 0 2 0 C---- vn /lib/libc.so.7 > 62770 0x822368000 0x82236f000 rw- 7 0 1 0 C---- vn /lib/libc.so.7 > 62770 0x82236f000 0x82259e000 rw- 20 20 1 0 ----- sw > 62770 0x823434000 0x823447000 r-- 19 59 4 2 CN--- vn /lib/libm.so.5 > 62770 0x823447000 0x82346f000 r-x 40 59 4 2 CN--- vn /lib/libm.so.5 > 62770 0x82346f000 0x823470000 rw- 1 0 1 0 C---- vn /lib/libm.so.5 > 62770 0x823470000 0x823471000 rw- 1 0 1 0 C---- vn /lib/libm.so.5 > 62770 0x823e0e000 0x823e3e000 rw- 16 16 1 0 ----- sw > 62770 0x824600000 0x824800000 rw- 11 11 1 0 ----- sw > 62770 0x8251a1000 0x8253a1000 rw- 1 1 1 0 ----- sw > 62770 0x825e00000 0x826200000 rw- 3 3 1 0 ----- sw > 62770 0x826a49000 0x826a61000 rw- 8 8 1 0 ----- sw > 62770 0x826e5c000 0x826e74000 rw- 14 14 1 0 ----- sw > 62770 0x827d6e000 0x827d86000 rw- 9 9 1 0 ----- sw > 62770 0x8288ba000 0x8288d2000 rw- 5 5 1 0 ----- sw > 62770 0x8296db000 0x8296f3000 rw- 3 3 1 0 ----- sw > 62770 0xeeeecc15000 0xeeeecc1b000 r-- 6 29 71 21 CN--- vn /libexec/ld-elf.so.1 > 62770 0xeeeecc1b000 0xeeeecc32000 r-x 23 29 71 21 CN--- vn /libexec/ld-elf.so.1 > 62770 0xeeeecc32000 0xeeeecc33000 r-- 1 0 1 0 C---- vn /libexec/ld-elf.so.1 > 62770 0xeeeecc33000 0xeeeecc35000 rw- 2 2 1 0 ----- sw > 62770 0x7ffffffff000 0x800000000000 --- 0 0 0 0 ----- gd ldrt is now mapped up at 0xeeeecc15000 and the user stack looks like it starts at 0x82073d000. This is causing me problems with Valgrind, which creates the guest stack at 0x7ffffffdf000. I haven't yet done any debugging of the problem but this causes Fatal error 'Cannot allocate red zone for initial thread' at line 395 in file /usr/src/lib/libthr/thread/thr_init.c (errno = 22) for elf binaries linked with libthr.so Can anyone point me to more information on this change? Phabricator for instance. Are there any syscalls that control where rtld gets loaded and/or where the stack base is located? Also is there a sysctl to disable this changed mapping, as a temporary workaround? A+ Paul