From owner-freebsd-current@freebsd.org Tue Sep 6 16:25:45 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA830BC715D for ; Tue, 6 Sep 2016 16:25:45 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qk0-x229.google.com (mail-qk0-x229.google.com [IPv6:2607:f8b0:400d:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FB57B69 for ; Tue, 6 Sep 2016 16:25:45 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qk0-x229.google.com with SMTP id l2so223426886qkf.3 for ; Tue, 06 Sep 2016 09:25:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=l8X1Cly//KBB/fEQLaDMtMa5oEAvCewYhv3h6TrCJzw=; b=nGLdPpnIFE9ZpLf1uc7rl7zmLeDAfEtIla4rDKSf5mZy+l7WzKdpY3r6hr/r7q93Gu auzp8fHSWpejzFw9GMx01keAHtYxsSntn1ReWQWZyELmnoeg2wM/ABf3O6q5qLom7AHU 4j/CY9eGMzxU5B9vSA9RLHAnJJ6H6oeb92NSgvol98NHW8B9XOoedOhSho2ZxiHf1J1p JY7F9JSgzFyEIWrT7XNybyb8RbwQdyv7f1kNA16KQI0Jkra85aABuu/QxCbiy9rIwovL DpGVajWAe7lRkTlYoAz0ND8p2EAD4KloKM+2O3YW9SNCb1tNXCvWzCBrUzJqhu3mnyRg 3PCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=l8X1Cly//KBB/fEQLaDMtMa5oEAvCewYhv3h6TrCJzw=; b=hYhF345W1MsK+ODtpqrSaUOjRdneZB5G1pNj+a8ovcW0FQ2svItaSCS1ko0PxSbPHp 2eMCFoVZZitUiBbiAnMRg3/SkRCS3KCKbHRJ7Zqy359sI8yQQytcevNMfWLUz6dN6+WP Ck796eJXQ8GMV2lzbb9V66fiXSsZUNxNmxtEQ4bg1xC0S+qrDb90ZygB2C4iyLxnscVx D+ot6KOLYQiYPGF06L4FvzLczHqzt6IQTaDBw4qNoXyw9bxvLfDHRrQvYemFYUzgEPa5 D6f+ZZJSrxTiKKCeegt9Y+gJk1q4hwtmKFNNI0Yh2ECtUdZwGlxy49dl6+F7SfcjHb/s 3zfA== X-Gm-Message-State: AE9vXwMgNx8Mrw/YgZezxLKiK1cVWi7c9xR+wHsHXHDDLIkRtLFUEAF9mx8RxEYzsvsubX+9 X-Received: by 10.55.191.67 with SMTP id p64mr43895533qkf.44.1473179144639; Tue, 06 Sep 2016 09:25:44 -0700 (PDT) Received: from mutt-hardenedbsd ([63.88.83.66]) by smtp.gmail.com with ESMTPSA id p139sm18328200qke.45.2016.09.06.09.25.43 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 06 Sep 2016 09:25:43 -0700 (PDT) Date: Tue, 6 Sep 2016 12:25:41 -0400 From: Shawn Webb To: Mark Johnston Cc: freebsd-current@freebsd.org Subject: Re: taskqgroup_adjust kernel panic Message-ID: <20160906162541.GA7722@mutt-hardenedbsd> References: <20160905175538.GA81799@mutt-hardenedbsd> <20160905215454.GE70066@wkstn-mjohnston.west.isilon.com> <20160905235102.GA42492@mutt-hardenedbsd> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="tThc/1wpZn/ma/RB" Content-Disposition: inline In-Reply-To: <20160905235102.GA42492@mutt-hardenedbsd> X-Operating-System: FreeBSD mutt-hardenedbsd 12.0-CURRENT-HBSD FreeBSD 12.0-CURRENT-HBSD X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Sep 2016 16:25:45 -0000 --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 05, 2016 at 07:51:02PM -0400, Shawn Webb wrote: > On Mon, Sep 05, 2016 at 02:54:54PM -0700, Mark Johnston wrote: > > On Mon, Sep 05, 2016 at 01:55:38PM -0400, Shawn Webb wrote: > > > Hey all, > > >=20 > > > I'm at revision 3872750 of the hardened/current/drm-next-4.7 branch in > > > the HardenedBSD/hardenedBSD-playground repo. I've gotten this kernel > > > panic a couple times when booting. I'm using full-disk encryption with > > > ZFS and encrypted swap. The hardware is a Purism 15 2K laptop. > > >=20 > > > The panic doesn't happen often nor is there a way I can reproduce it > > > 100%. > > >=20 > > > Here's my `uname -a` output: > > >=20 > > > FreeBSD hbsd-dev-laptop 12.0-CURRENT-HBSD FreeBSD 12.0-CURRENT-HBSD #= 0 3872750(hardened/current/drm-next-4.7): Tue Aug 30 17:41:53 EDT 2016 = shawn@hbsd-dev-laptop:/usr/obj/usr/src/sys/LATT-SEC amd64 > > >=20 > > > Here's a couple pictures of the panic I took: > > >=20 > > > https://goo.gl/photos/P5kiwabPYjwQX7Kr8 > > > https://goo.gl/photos/BWtvBnq7QLnwgRP28 > >=20 > > Based on the faulting instruction, the panic probably happened because > > qid is uninitialized in the loop that starts with > >=20 > > while ((gtask =3D LIST_FIRST(>ask_head))) { > >=20 > > I don't know this code very well, so I'm not sure how that can happen. I > > suspect iflib_irq_alloc_generic() is buggy: it calls > >=20 > > taskqgroup_attach_cpu(... CPU_FFS(&cpus) ...); > >=20 > > and CPU_FFS returns 1-indexed IDs, but taskqgroup_attach_cpu() pretty > > clearly expects 0-indexed CPU IDs. There's a similar bug in find_nth() > > in iflib.c. >=20 > I think you hit the nail right on the head. Attached is a patch that > doesn't fix the underlying issue, but at least detects improperly > setting qid. It'll throw a KASSERT if qid isn't set properly. >=20 > I'll study this code a bit more within the next couple days and I hope > to have a full patch to address the underlying issue. I've now verified, using that patch, that qid is always uninitialized in my case. That KASSERT is hit 100% of the time when the laptop is booting up. I've filed a bug report. The problematic code exists in 11-STABLE and 11.0-RELENG as well. Link to bug report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212418 Thanks, --=20 Shawn Webb Cofounder and Security Engineer HardenedBSD GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --tThc/1wpZn/ma/RB Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJXzu4DAAoJEGqEZY9SRW7uiTgQAL6cZbHN4S/E2E1RyuGOCfpk dFp5O895kyYNRWgQgDtCRfHSBQmlv+fbT2LKlW1IY3KL3/nHNHewPSFAZQ1fs71D hewKX+6v/J+qEQFdBmyUXoSzt62JhBzrTQRTr2dmtBi4jsTGhyfp4KOCbyll7w/7 vuM+63ZWUq5IZ4VQFg0SQrF/7hMEevDiHGcX/6lFfESo5D1y1ZTB1vE0JSRKyHEm q0gtmn+2SPLw+dsXgsVpswmDE8OfiKkAKIcDHg2PPMXX4j/g9Fv0Ako3r1DXL00h AooITS9gdt1ydr7+kD9DaSADa7Ms3caT4VMesGxd8Kah3cM5t3SIUkv8m0d8aQ0K 0zg2k99ZGMMdgEOJ9lWXyFwQLRDcEcz3btHKxswJjOVU/aQ2pCNvATJ7Kgis1wde 64fYb/vDDnH5KX/XSzgeKUE4eTIdWWcTa9ZeiA9G1hq+ajFRa/b5z6Q403WUJase MxkzXnpaSsqVVRIm7jnnC0KefMG/PbYs9Xy14WfPnFRqmvI6rnA2sRWs3LQxS7fu rG4nV8YVGobyMXnSvfa1m+In4zQCwf64xgAZ3ePhocVc9LdJ1Xfzvb2zEl5eHTOZ pZMKabOpc4xWfXsrygCk5vC1vCim/l1qPySomCkiW+aOt2KjV2IjzqmzRDtK+JS5 69O35nuLp6QtVcY7aPaZ =OQeh -----END PGP SIGNATURE----- --tThc/1wpZn/ma/RB--