Date: Sun, 16 Oct 2022 17:08:30 +0200 From: Paul Floyd <paulf2718@gmail.com> To: freebsd-hackers <freebsd-hackers@freebsd.org> Subject: AMD64 14.0-CURRENT memory layout changes Message-ID: <bcfa260b-7a94-1414-a5ae-c281e96c76ec@gmail.com>
next in thread | raw e-mail | index | archive | help
Hi I just noticed that the memory layout has changed for elf binaries running on amd64 (my last attempt to setup an i386 VM failed so I can't confirm if that also changed, and I'm not yet concerned by other platforms). Here's a procstat -v for ksh93 on 13.1 on the host machine > paulf> procstat -v 1456 > PID START END PRT RES PRES REF SHD FLAG TP PATH > 1456 0x200000 0x273000 r-- 70 343 30 16 CN--- vn /usr/local/bin/ksh93 > 1456 0x273000 0x3d1000 r-x 257 343 30 16 CN--- vn /usr/local/bin/ksh93 > 1456 0x3d1000 0x3dd000 r-- 11 0 1 0 CN--- vn /usr/local/bin/ksh93 > 1456 0x3dd000 0x3de000 rw- 1 0 1 0 CN--- vn /usr/local/bin/ksh93 > 1456 0x3de000 0x3e4000 rw- 6 0 1 0 C---- vn /usr/local/bin/ksh93 > 1456 0x3e4000 0x3ec000 rw- 5 5 1 0 C---- df > 1456 0x8003de000 0x8003e4000 r-- 6 28 364 120 CN--- vn /libexec/ld-elf.so.1 > 1456 0x8003e4000 0x8003fb000 r-x 23 28 364 120 CN--- vn /libexec/ld-elf.so.1 > 1456 0x8003fb000 0x8003fc000 r-- 1 0 1 0 CN--- vn /libexec/ld-elf.so.1 > 1456 0x8003fc000 0x800411000 rw- 19 19 1 0 CN--- df > 1456 0x800420000 0x800432000 r-- 13 33 352 176 CN--- vn /lib/libm.so.5 > 1456 0x800432000 0x800459000 r-x 20 33 352 176 CN--- vn /lib/libm.so.5 > 1456 0x800459000 0x80045a000 rw- 1 0 1 0 CN--- vn /lib/libm.so.5 > 1456 0x80045a000 0x80045b000 rw- 1 0 1 0 CN--- vn /lib/libm.so.5 > 1456 0x80045b000 0x8004df000 r-- 132 380 528 284 CN--- vn /lib/libc.so.7 > 1456 0x8004df000 0x80062b000 r-x 234 380 528 284 CN--- vn /lib/libc.so.7 > 1456 0x80062b000 0x800633000 r-- 8 0 1 0 CN--- vn /lib/libc.so.7 > 1456 0x800633000 0x800634000 rw- 1 0 1 0 CN--- vn /lib/libc.so.7 > 1456 0x800634000 0x80063b000 rw- 7 0 1 0 C---- vn /lib/libc.so.7 > 1456 0x80063b000 0x8008f5000 rw- 84 84 1 0 C---- df > 1456 0x800a00000 0x801200000 rw- 13 13 1 0 CN--- df > 1456 0x7fffdffff000 0x7ffffffdf000 --- 0 0 0 0 ----- gd > 1456 0x7ffffffdf000 0x7ffffffff000 rw- 11 11 1 0 C--D- df > 1456 0x7ffffffff000 0x800000000000 r-x 1 1 125 0 ----- ph Here the stack starts at 0x7ffffffdf000 And the same on 14.0 running on a 4Gbyte VirtualBox VM > paulf@freebsd:~/valgrind $ procstat -v 62770 > PID START END PRT RES PRES REF SHD FLAG TP PATH > 62770 0x200000 0x273000 r-- 115 488 4 2 CN--- vn /usr/local/bin/ksh93 > 62770 0x273000 0x3c7000 r-x 340 488 4 2 CN--- vn /usr/local/bin/ksh93 > 62770 0x3c7000 0x3d4000 r-- 13 0 2 0 C---- vn /usr/local/bin/ksh93 > 62770 0x3d4000 0x3d5000 rw- 1 0 2 0 C---- vn /usr/local/bin/ksh93 > 62770 0x3d5000 0x3da000 rw- 5 0 1 0 C---- vn /usr/local/bin/ksh93 > 62770 0x3da000 0x3e2000 rw- 5 5 1 0 ----- sw > 62770 0x80075d000 0x82073d000 --- 0 0 0 0 ----- gd > 62770 0x82073d000 0x82075d000 rw- 14 14 1 0 ---D- sw > 62770 0x8209c8000 0x8209c9000 r-x 1 1 28 0 ----- ph > 62770 0x8217b0000 0x8217c2000 rw- 16 16 1 0 ----- sw > 62770 0x822186000 0x822210000 r-- 138 496 104 54 CN--- vn /lib/libc.so.7 > 62770 0x822210000 0x82235e000 r-x 334 496 104 54 CN--- vn /lib/libc.so.7 > 62770 0x82235e000 0x822367000 r-- 9 0 2 0 C---- vn /lib/libc.so.7 > 62770 0x822367000 0x822368000 rw- 1 0 2 0 C---- vn /lib/libc.so.7 > 62770 0x822368000 0x82236f000 rw- 7 0 1 0 C---- vn /lib/libc.so.7 > 62770 0x82236f000 0x82259e000 rw- 20 20 1 0 ----- sw > 62770 0x823434000 0x823447000 r-- 19 59 4 2 CN--- vn /lib/libm.so.5 > 62770 0x823447000 0x82346f000 r-x 40 59 4 2 CN--- vn /lib/libm.so.5 > 62770 0x82346f000 0x823470000 rw- 1 0 1 0 C---- vn /lib/libm.so.5 > 62770 0x823470000 0x823471000 rw- 1 0 1 0 C---- vn /lib/libm.so.5 > 62770 0x823e0e000 0x823e3e000 rw- 16 16 1 0 ----- sw > 62770 0x824600000 0x824800000 rw- 11 11 1 0 ----- sw > 62770 0x8251a1000 0x8253a1000 rw- 1 1 1 0 ----- sw > 62770 0x825e00000 0x826200000 rw- 3 3 1 0 ----- sw > 62770 0x826a49000 0x826a61000 rw- 8 8 1 0 ----- sw > 62770 0x826e5c000 0x826e74000 rw- 14 14 1 0 ----- sw > 62770 0x827d6e000 0x827d86000 rw- 9 9 1 0 ----- sw > 62770 0x8288ba000 0x8288d2000 rw- 5 5 1 0 ----- sw > 62770 0x8296db000 0x8296f3000 rw- 3 3 1 0 ----- sw > 62770 0xeeeecc15000 0xeeeecc1b000 r-- 6 29 71 21 CN--- vn /libexec/ld-elf.so.1 > 62770 0xeeeecc1b000 0xeeeecc32000 r-x 23 29 71 21 CN--- vn /libexec/ld-elf.so.1 > 62770 0xeeeecc32000 0xeeeecc33000 r-- 1 0 1 0 C---- vn /libexec/ld-elf.so.1 > 62770 0xeeeecc33000 0xeeeecc35000 rw- 2 2 1 0 ----- sw > 62770 0x7ffffffff000 0x800000000000 --- 0 0 0 0 ----- gd ldrt is now mapped up at 0xeeeecc15000 and the user stack looks like it starts at 0x82073d000. This is causing me problems with Valgrind, which creates the guest stack at 0x7ffffffdf000. I haven't yet done any debugging of the problem but this causes Fatal error 'Cannot allocate red zone for initial thread' at line 395 in file /usr/src/lib/libthr/thread/thr_init.c (errno = 22) for elf binaries linked with libthr.so Can anyone point me to more information on this change? Phabricator for instance. Are there any syscalls that control where rtld gets loaded and/or where the stack base is located? Also is there a sysctl to disable this changed mapping, as a temporary workaround? A+ Paul
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bcfa260b-7a94-1414-a5ae-c281e96c76ec>