From owner-freebsd-arm@freebsd.org Thu Nov 30 02:31:19 2017 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA64FDF2605 for ; Thu, 30 Nov 2017 02:31:19 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B1BC6F907 for ; Thu, 30 Nov 2017 02:31:19 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x236.google.com with SMTP id d21so5917681ioe.7 for ; Wed, 29 Nov 2017 18:31:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=t0zM1Ly+b6FMoO2miBpGpDkALNhPf+yrrjYzzUbUG6k=; b=YT/TsXwXIq7TlzDPI9ijhEzHHYqL1ZI52YmOgT5zEZxehhWIsulaFZYzQ2A/TPSZMX kGwpBAFWv4nb0BB4NI+zbEt2tJBQwIz8WxafSVR/8yc6tzlHRFOylNGS3MqLulP+Id+s iB1Qx2GACWn9z+OBjrN01yka0RjZvQVYzB9tXcGzXyApd0fTIWzaCtXhCgp4oDtp+QTu uRNtaWkTug5q+dWT1BNjOF28k4mSUe6ERmXxOAOTaRoowicKDoF7TCnNZ3TJ9B1njjng fEubI+3TWJ/ebxZjUTWw/wQxZmOCiqFfnk94gVyNuSSLh9CwL87KwCktz1Q2mZ3Sp0MN Vgrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=t0zM1Ly+b6FMoO2miBpGpDkALNhPf+yrrjYzzUbUG6k=; b=GNHTXDDV3pzvaMOo/ebrNzwBG45dA9/sWEFf6UOTuLOqTa3VzBpPlqT6+xu80z9r35 ijbTq0y362qC6JwYG3iiZU0h6RxhLDZHHU1uSxjOJJOiFcFFYjBfAYnb557O9MbTvfI6 RCIIwlFdKid45QZEgfoBiKMmoAg3DQZng0f/XGsoSo/2hs9vnHWevvb1Uv9ki0aJHklj Oxa07SLr3jZjck6uIETvEMoxKuZGA/BTFhs1xFJ/prOn5PVCBfKEEcq6q3BCwKVYGtfw PAUVms6Mav9GNg9q3gBt7BUbbseSHWZAmbZ3ficl2J/F9FS2oDpeGuRWLZnC6Zg2SzMI /nsA== X-Gm-Message-State: AJaThX6hwa3jb/cPWqFzdrW8LCv+tviyhqhuNut613L3yuT5zyd82h+P dJSsva1TGsrLObw7StbKiJSLbnOojiSRdGUUKB9jpwS3 X-Google-Smtp-Source: AGs4zMbJm0gF17ersF1ewY9u3bIF+fktM/SYHVG8/E6bl9ABxxfjyIkcwjk0SlhX3GkJaVfTpUCTT/c2r3g3Hfevmeo= X-Received: by 10.107.81.24 with SMTP id f24mr5841036iob.63.1512009078633; Wed, 29 Nov 2017 18:31:18 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.108.204 with HTTP; Wed, 29 Nov 2017 18:31:17 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: References: <20171130002135.q7p27hh6qkog4slr@mutt-hbsd> <20171130003458.bnltotlgfdbke5ue@mutt-hbsd> <20171130004358.usia2jrvman4cvvz@mutt-hbsd> From: Warner Losh Date: Wed, 29 Nov 2017 19:31:17 -0700 X-Google-Sender-Auth: DycRMVjGxUxf7RZ6dh0eK5QDm0U Message-ID: Subject: Re: Booting UEFI ZFS is broken on arm64 To: Shawn Webb Cc: "freebsd-arm@freebsd.org" , FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2017 02:31:19 -0000 On Wed, Nov 29, 2017 at 5:54 PM, Warner Losh wrote: > > > On Wed, Nov 29, 2017 at 5:43 PM, Shawn Webb > wrote: > >> On Wed, Nov 29, 2017 at 05:42:52PM -0700, Warner Losh wrote: >> > On Wed, Nov 29, 2017 at 5:34 PM, Shawn Webb > > >> > wrote: >> > >> > > On Wed, Nov 29, 2017 at 05:33:46PM -0700, Warner Losh wrote: >> > > > On Wed, Nov 29, 2017 at 5:21 PM, Shawn Webb < >> shawn.webb@hardenedbsd.org> >> > > > wrote: >> > > > >> > > > > It appears that in the latest FreeBSD 12-CURRENT/arm64 snapshot, >> > > > > booting UEFI GPT ZFS on my OverDrive 1000 is broken. It boots up >> to >> > > > > this line: >> > > > > >> > > > > Using DTB provided by EFI at 0x801fe00000. >> > > > >> > > > >> > > > Which snapshot is that? Boot1 was broken until recently. >> > > >> > > FreeBSD-12.0-CURRENT-arm64-aarch64-20171121-r326056-memstick.img >> > > >> > > It also happens on latest HEAD, so it would appear to still be broken. >> > >> > >> > Is this boot1.efi producing the output, or loader.efi? I'm guessing the >> > latter, but wanted to make sure. If so, then we're past the point where >> > boot1.efi would have failed (besides, it was fixed before that >> snapshot). >> >> With DEBUG turned on for stand/fdt: >> >> Booting [/boot/kernel/kernel]... >> fdt_copy(): fdt_copy va 0x01208000 >> fdt_setup_fdtp(): fdt_setup_fdtp() >> fdt_load_dtb_addr(): fdt_load_dtb_addr(0x801fe00000) >> Using DTB provided by EFI at 0x801fe00000. >> Loaded the platform dtb: 0x81f56f1630. >> fdt_fixup(): fdt_fixup() >> >> ^ hangs after that message > > > That doesn't sound like anything I've changed, but it could well be... I > think to find this breakage, you may need to bisect backwards along stand / > sys/boot until we find the spot where it broke. > There's been several conversations on IRC about how others are hitting a scheduler bug, at least on x86. hps' fix seems to do the trick for their issues. Author: hselasky Date: Wed Nov 29 23:28:40 2017 +0000 The sched_add() function is not only used when the thread is initially started, but also by the turnstiles to mark a thread as runnable for all locks, for instance sleepqueues do: setrunnable()->sched_wakeup()->sched_add() In r326218 code was added to allow booting from non-zero CPU numbers by setting the ts_cpu field inside the ULE scheduler's sched_add() function. This had an undesired side-effect that prior sched_pin() and sched_bind() calls got disregarded. This patch fixes the initialization of the ts_cpu field for the ULE scheduler to only happen once when the initial thread is constructed during system init. Forking will then later on ensure that a valid ts_cpu value gets copied to all children. Reviewed by: jhb, kib Discussed with: nwhitehorn MFC after: 1 month Differential revision: https://reviews.freebsd.org/D13298 Sponsored by: Mellanox Technologies git-svn-id: svn+ssh://svn.freebsd.org/base/head@326376 ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f is the fix.... But the bug it fixes post-dates the snapshot, so maybe this isn't the same thing... Warner