Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 1 Dec 2017 16:55:31 -0500
From:      Shawn Webb <shawn.webb@hardenedbsd.org>
To:        Warner Losh <imp@bsdimp.com>
Cc:        "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org>, FreeBSD Current <freebsd-current@freebsd.org>
Subject:   Re: Booting UEFI ZFS is broken on arm64
Message-ID:  <20171201215531.u4f2c4eno7z5irxj@mutt-hbsd>
In-Reply-To: <CANCZdfp6CdTCTQ-wDbHQO2v4w1HC8=iY12F%2BNVofu=kq1%2BVvkg@mail.gmail.com>
References:  <20171130002135.q7p27hh6qkog4slr@mutt-hbsd> <CANCZdfrjKkR-Lr4FKVyCBBnYy8k-n6rgz6ES6=_K6SeXtFOtMg@mail.gmail.com> <20171130003458.bnltotlgfdbke5ue@mutt-hbsd> <CANCZdfqgco6gp-ccph3hSgJg81=%2BEfW2DJp0DOx2xgCdgH7UkQ@mail.gmail.com> <20171130004358.usia2jrvman4cvvz@mutt-hbsd> <CANCZdfrH%2ByhkP3RK9aRvgXXnOJ2cw%2B9xU4G9Ge82vZFPERCCCg@mail.gmail.com> <CANCZdfqzHYRt-99MuzWJzYW6bn2YYkQwHE=0APfQM_iOY6RkoQ@mail.gmail.com> <20171201214955.zopa6v2f2mx3rkt2@mutt-hbsd> <CANCZdfp6CdTCTQ-wDbHQO2v4w1HC8=iY12F%2BNVofu=kq1%2BVvkg@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help

--yxqbpbqi3jogdpsy
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Dec 01, 2017 at 02:53:53PM -0700, Warner Losh wrote:
> On Fri, Dec 1, 2017 at 2:49 PM, Shawn Webb <shawn.webb@hardenedbsd.org>
> wrote:
>=20
> > On Wed, Nov 29, 2017 at 07:31:17PM -0700, Warner Losh wrote:
> > > On Wed, Nov 29, 2017 at 5:54 PM, Warner Losh <imp@bsdimp.com> wrote:
> > >
> > > >
> > > >
> > > > On Wed, Nov 29, 2017 at 5:43 PM, Shawn Webb <
> > shawn.webb@hardenedbsd.org>
> > > > wrote:
> > > >
> > > >> On Wed, Nov 29, 2017 at 05:42:52PM -0700, Warner Losh wrote:
> > > >> > On Wed, Nov 29, 2017 at 5:34 PM, Shawn Webb <
> > shawn.webb@hardenedbsd.org
> > > >> >
> > > >> > wrote:
> > > >> >
> > > >> > > On Wed, Nov 29, 2017 at 05:33:46PM -0700, Warner Losh wrote:
> > > >> > > > On Wed, Nov 29, 2017 at 5:21 PM, Shawn Webb <
> > > >> shawn.webb@hardenedbsd.org>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > It appears that in the latest FreeBSD 12-CURRENT/arm64
> > snapshot,
> > > >> > > > > booting UEFI GPT ZFS on my OverDrive 1000 is broken. It bo=
ots
> > up
> > > >> to
> > > >> > > > > this line:
> > > >> > > > >
> > > >> > > > > Using DTB provided by EFI at 0x801fe00000.
> > > >> > > >
> > > >> > > >
> > > >> > > > Which snapshot is that? Boot1 was broken until recently.
> > > >> > >
> > > >> > > FreeBSD-12.0-CURRENT-arm64-aarch64-20171121-r326056-memstick.i=
mg
> > > >> > >
> > > >> > > It also happens on latest HEAD, so it would appear to still be
> > broken.
> > > >> >
> > > >> >
> > > >> > Is this boot1.efi producing the output, or loader.efi? I'm guess=
ing
> > the
> > > >> > latter, but wanted to make sure. If so, then we're past the point
> > where
> > > >> > boot1.efi would have failed (besides, it was fixed before that
> > > >> snapshot).
> > > >>
> > > >> With DEBUG turned on for stand/fdt:
> > > >>
> > > >> Booting [/boot/kernel/kernel]...
> > > >> fdt_copy(): fdt_copy va 0x01208000
> > > >> fdt_setup_fdtp(): fdt_setup_fdtp()
> > > >> fdt_load_dtb_addr(): fdt_load_dtb_addr(0x801fe00000)
> > > >> Using DTB provided by EFI at 0x801fe00000.
> > > >> Loaded the platform dtb: 0x81f56f1630.
> > > >> fdt_fixup(): fdt_fixup()
> > > >>
> > > >> ^ hangs after that message
> > > >
> > > >
> > > > That doesn't sound like anything I've changed, but it could well be=
=2E..
> > I
> > > > think to find this breakage, you may need to bisect backwards along
> > stand /
> > > > sys/boot until we find the spot where it broke.
> > > >
> > >
> > > There's been several conversations on IRC about how others are hittin=
g a
> > > scheduler bug, at least on x86. hps' fix seems to do the trick for th=
eir
> > > issues.
> > >
> > > Author: hselasky <hselasky@ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f>
> > > Date:   Wed Nov 29 23:28:40 2017 +0000
> > >
> > >     The sched_add() function is not only used when the thread is
> > initially
> > >     started, but also by the turnstiles to mark a thread as runnable =
for
> > >     all locks, for instance sleepqueues do:
> > >     setrunnable()->sched_wakeup()->sched_add()
> > >
> > >     In r326218 code was added to allow booting from non-zero CPU numb=
ers
> > >     by setting the ts_cpu field inside the ULE scheduler's sched_add()
> > >     function. This had an undesired side-effect that prior sched_pin()
> > and
> > >     sched_bind() calls got disregarded. This patch fixes the
> > >     initialization of the ts_cpu field for the ULE scheduler to only
> > >     happen once when the initial thread is constructed during system
> > >     init. Forking will then later on ensure that a valid ts_cpu value
> > gets
> > >     copied to all children.
> > >
> > >     Reviewed by:    jhb, kib
> > >     Discussed with: nwhitehorn
> > >     MFC after:      1 month
> > >     Differential revision:  https://reviews.freebsd.org/D13298
> > >     Sponsored by:   Mellanox Technologies
> > >
> > >
> > >     git-svn-id: svn+ssh://svn.freebsd.org/base/head@326376
> > > ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
> > >
> > > is the fix.... But the bug it fixes post-dates the snapshot, so maybe
> > this
> > > isn't the same thing...
> >
> > Definitely is not the same thing. I've so far got it printf'd to where
> > the uefi loader jumps into the kernel's entry point. So the loader
> > itself might be fine. Something in the kernel, then, is going funky.
> >
> > Booting in verbose mode does not provide any additional input.
> >
> > Here's the output I get (some of the output is from printf's I've
> > done):
> >
> > FreeBSD/arm64 EFI loader, Revision 1.1
> > (Wed Nov 29 21:51:14 EST 2017 shawn@hbsd-dev-laptop)
> > EFI boot environment
> > Loading /boot/defaults/loader.conf
> > /boot/kernel/kernel text=3D0x7e0a78 data=3D0xaad80+0x443f62
> > syms=3D[0x8+0x10ec78+0x8+0x1021d4]
> > /boot/entropy size=3D0x1000
> > /boot/kernel/zfs.ko text=3D0x99070 text=3D0x130390 data=3D0x21ff8+0x9ef=
98
> > syms=3D[0x8+0x22c68+0x8+0x1b99b]
> > /boot/kernel/opensolaris.ko text=3D0x1330 text=3D0xd00 data=3D0x10160+0=
x125d0
> > syms=3D[0x8+0xff0+0x8+0x8d8]
> >
> > Hit [Enter] to boot immediately, or any other key for command prompt.
> > Booting [/boot/kernel/kernel]...
> > Using DTB provided by EFI at 0x801fe00000.
> > fdt_copy returned. dtb_size is 9060.
> > bi_load finished. err: 0
> > dev_cleanup finished
> > About to call into the entry point at 0x81ee601000
> >
>=20
> You might try booting the same kernel off a small UFS partition. There's a
> tiny chance that the loader didn't load it right, but more likely the
> kernel is borked. Maybe DTB issues? Maybe something else... A quick test
> like that would remove ZFS from the equation, even if it's just a USB
> stick...

UFS works fine and dandy. It's ZFS that's b0rked.

Thanks,

--=20
Shawn Webb
Cofounder and Security Engineer
HardenedBSD

GPG Key ID:          0x6A84658F52456EEE
GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89  3D9E 6A84 658F 5245 6EEE

--yxqbpbqi3jogdpsy
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEKrq2ve9q9Ia+iT2eaoRlj1JFbu4FAlohz9IACgkQaoRlj1JF
bu4oXw/9EOPJzkBWeiYIsnLFVg05wSQVIIuMyhYE/oEmnY0lK5H+muPK6t9ETqM+
AVEGxjEpmgeOCkzt3tOOrC5jul/PSR/XGpmVi68ho5KEbuOFD+UPjtN50cZJraSD
Uusjl8jtFWBDAybQ78O43XIrr2710NQ0PTbeZYJouQnTvLz+MkVhMgKF37YglZOK
4CLqEMzEqwNIxj/tKRvJt+G6NB12xpN3mp8w9Svq/G10v7GjxZC7ikqsd5FLH6iP
hWpyqqRtoucbCpn66Ahc98c32EHpMvwx2XffsddWb8g1Z/dSjxAD1j7mBLLHBVbV
1e/CB5S7Fyexei66/UZ47q0ZIqTtVGwhYE1LqMVdvLUimhtd4t06nWJeRGUDrr0C
NZuLRBOUiY8suaI5oYizyg84m4hjfCoEK27Hye5aYizF6gAh/eodllfm7YHde+oT
Nos1YTb2/axrS60ibUdhom3a1gD1TvUqVVkrdRCf1+ds0RfIgAHiAAsZ9qogEoIP
PBeumYz6+4Waq44VUjD1QKoEy9AG6yj0plLp/n/Dt6AoViM0nOT4puYMuhbgps+i
RjBIjn8JiToBYsNppn3Gv7GTn7vwEiqU2OyfFs4CH8SZ6MIvhhLOrQ16B8BuBEGm
7erXmEi8ZtfOZHtJ6MZlJTXj48OYP1TwUuvVwuvEmPAO9LF4Mqk=
=+uXh
-----END PGP SIGNATURE-----

--yxqbpbqi3jogdpsy--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20171201215531.u4f2c4eno7z5irxj>