Date: Wed, 15 Jun 2022 23:15:26 +0200 (CEST) From: Sysadmin Lists <sysadmin.lists@mailfence.com> To: freebsd-emulation@freebsd.org Cc: dchagin@freebsd.org, Martin Simmons <martin@lispworks.com> Subject: Re: libc6 Compatibility Message-ID: <15732251.1658703.1655327726772@ichabod.co-bxl> In-Reply-To: <202206151944.25FJinlg024188@higson.cam.lispworks.com> References: <1236409007.1125724.1655169963971@ichabod.co-bxl> <Yqg3tUtdazdM1uvK@heemeyer.club> <2033861928.1354940.1655236558393@ichabod.co-bxl> <202206151704.25FH4m4d019069@higson.cam.lispworks.com> <606426103.1627955.1655317573324@ichabod.co-bxl> <202206151944.25FJinlg024188@higson.cam.lispworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> ----------------------------------------
> From: Martin Simmons <martin@lispworks.com>
> Sent: Wed Jun 15 21:44:49 CEST 2022
> To: Sysadmin Lists <sysadmin.lists@mailfence.com>
> Cc: <freebsd-emulation@freebsd.org>, <dchagin@freebsd.org>
> Subject: Re: libc6 Compatibility
>
>
> >>>>> On Wed, 15 Jun 2022 20:26:13 +0200 (CEST), Sysadmin Lists said:
> >
> > > ----------------------------------------
> > > From: Martin Simmons <martin@lispworks.com>
> > > Sent: Wed Jun 15 19:04:48 CEST 2022
> > > To: Sysadmin Lists <sysadmin.lists@mailfence.com>
> > > Cc: <freebsd-emulation@freebsd.org>, <dchagin@freebsd.org>
> > > Subject: Re: libc6 Compatibility
> > >
> > >
> > > >>>>> On Tue, 14 Jun 2022 21:55:58 +0200 (CEST), Sysadmin Lists said:
> > > >
> > > > > ----------------------------------------
> > > > > From: Dmitry Chagin <dchagin@freebsd.org>
> > > > > Sent: Tue Jun 14 09:24:37 CEST 2022
> > > > > To: Sysadmin Lists <sysadmin.lists@mailfence.com>
> > > > > Cc: Freebsd Emulation <freebsd-emulation@freebsd.org>
> > > > > Subject: Re: libc6 Compatibility
> > > > >
> > > > >
> > > > > On Tue, Jun 14, 2022 at 03:26:03AM +0200, Sysadmin Lists wrote:
> > > > > > Does Linux compat not work with Ubuntu's newest libc6 update?
> > > > > > I used to run Brave browser from it, but now I get this error message:
> > > > > >
> > > > > > $ /compat/ubuntu/opt/brave.com/brave/brave
> > > > > > ELF interpreter /lib64/ld-linux-x86-64.so.2 not found, error 2
> > > > > > Abort trap
> > > > > >
> > > > > > The shared object is loaded in memory:
> > > > > > $ ldd /opt/brave.com/brave/brave | grep ld-
> > > > > > /lib64/ld-linux-x86-64.so.2 (0x0000000001021000)
> > > > > >
> > > > > > And it exists on the filesystem:
> > > > > > $ find /compat/ubuntu/lib**/ -name ld-\* -exec ls -lh '{}' +
> > > > > > -rwxr-xr-x 1 root wheel 187K Dec 16 2020 /compat/ubuntu/lib/x86_64-linux-gnu/ld-2.31.so
> > > > > > lrwxr-xr-x 1 root wheel 10B Dec 16 2020 /compat/ubuntu/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 -> ld-2.31.so
> > > > > > lrwxr-xr-x 1 root root 34B Mar 23 2021 /compat/ubuntu/lib64/ld-linux-x86-64.so.2 -> ../lib/x86_64-linux-gnu/ld-2.31.so
> > > > > >
> > > > > > This is the update that broke it:
> > > > > > $ apt-get install libc6
> > > > > > Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libc6 amd64 2.31-0ubuntu9.9 [2722 kB]
> > > > > >
> > > > >
> > > > > try ktrace it, aka ktrace -di /compat/ubuntu/opt/brave.com/brave/brave
> > > > > then kdump -HAR -m 128 > brave.log
> > > > >
> > > > >
> > > >
> > > > Thanks, Dmitry. That helped me find the problem. Here's the full log:
> > > >
> > > > $ cat brave.log
> > > > 8184 101222 ktrace 0.000000 RET F64 ktrace 0
> > > > 8184 101222 ktrace 0.000014 CALL F64 execve(0x7fffffffeda3,0x7fffffffeac0,0x7fffffffead0)
> > > > 8184 101222 ktrace 0.000003 NAMI F64 "/compat/ubuntu/opt/brave.com/brave/brave"
> > > > 8184 101222 ktrace 0.055726 NAMI F64 "/compat/ubuntu/lib64/ld-linux-x86-64.so.2"
> > > > 8184 101222 ktrace 0.000032 NAMI F64 "/lib64/ld-linux-x86-64.so.2"
> > > >
> > > > And here's an updated `ls' printout after upgrading libc6:
> > > > $ find /compat/ubuntu/lib**/ -name ld-\* -exec ls -lh '{}' +
> > > > -rwxr-xr-x 1 root wheel 187K Apr 6 18:24 /compat/ubuntu/lib/x86_64-linux-gnu/ld-2.31.so
> > > > lrwxr-xr-x 1 root wheel 10B Apr 6 18:24 /compat/ubuntu/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 -> ld-2.31.so
> > > > lrwxr-xr-x 1 root wheel 32B Apr 6 18:24 /compat/ubuntu/lib64/ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.31.so
> > > >
> > > > The new .so is missing 'previous directory' dots in the symlink path.
> > >
> > > It looks like the link with the dots was made by hand in 2021, presumably to
> > > fix this.
> >
> > That might be the case. I remember having to make changes to the shared object
> > before. The question then becomes, why only this update to libc6 and not
> > previous updates?
>
> Based on the file dates, I suspect there were no previous updates to libc6
> installed since Dec 16 2020.
You must be right, but I could have sworn I've updated libc6 at least a couple times.
Something caused me to recreate the symlink in Mar 2021.
> > > > I thought Linux compat was designed to search compat.linux.emul_path
> > > > for shared libs first, but apparently it works differently than that.
> > >
> > > What does linux compat do in general with absolute symlinks? Does it check
> > > the target with compat.linux.emul_path prepended?
> >
> > That's what the documentation says:
> > "Linux mode dynamically reroots lookups. This is, in effect, equivalent to
> > union to file system mounts. First, an attempt is made to look up the file in
> > /compat/linux/original-path. If that fails, the lookup is done in
> > /original-path. This makes sure that binaries that require other binaries can
> > run."
> > https://docs.freebsd.org/en/books/handbook/linuxemu/#linuxemu-advanced
> >
> > Looking at the installer script, it's clear this is a known issue:
> > $ sed -n '/fix_ld_path(/,/^$/p' /usr/local/share/linux-browser-installer/linux-browser-installer
> > fix_ld_path()
> > {
> > (cd ${chroot_path}/lib64 && \
> > (unlink ./ld-linux-x86-64.so.2; \
> > ln -s ../lib/x86_64-linux-gnu/ld-${ld_version}.so \
> > ld-linux-x86-64.so.2))
> > }
> >
> > So, either:
> > 1. The documentation is wrong.
> > 2. It's a bug.
>
> Looking at how it is implemented, the rerooting applies to the argument to the
> Linux system call before it calls the code that implements the FreeBSD system
> call to do the actual work. I think the problem here is that symlinks are
> resolved by FreeBSD system call code (the NAMI operation), but that doesn't do
> any rerooting.
>
> I don't know if that is a bug or a necessary limitation.
Interesting. Thanks for the added details and insight.
> __Martin
--
Sent with https://mailfence.com
Secure and private email
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15732251.1658703.1655327726772>