From owner-freebsd-current@freebsd.org Mon Oct 22 03:29:03 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39912FE598B for ; Mon, 22 Oct 2018 03:29:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-vs1-xe2d.google.com (mail-vs1-xe2d.google.com [IPv6:2607:f8b0:4864:20::e2d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B12A68514D for ; Mon, 22 Oct 2018 03:29:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-vs1-xe2d.google.com with SMTP id e126so28676499vsc.9 for ; Sun, 21 Oct 2018 20:29:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=9pm7BX5piXUzbLjtx4i7rZTJvhkWQAPzMzflVHKzHzA=; b=r7Bja/HtGDUwvRQk7RMVscP2SBVmbtinMhtyIWn/BAI/QFdiGW34DwakBZvOBO3TXh fwjggsbyS1kGGdfWzPU6V4DmKjfZ66rwbwnLA5AAa4XBs35+nJXdKnMcYaZlrwzsxXZF vdwZj7tmrlZgsKzA/czD4HGapxyankcBmqkfGtCGkNu4ixA7nbHBWdxkQh/aofUnTEA0 ZcVqx7gKukXfBNEXWHx5pNWdiPlfoDY7e4FWPR/0j3JIGLH67OD8xoSLqSLZ56uDPyYK XdoSJQ/aUZ1Hqg+YbDvH1EDOoeZyUcLKRTTBp5KwjOtB8/KypPqevZ0E0w5+3K3ShdCD 7XAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=9pm7BX5piXUzbLjtx4i7rZTJvhkWQAPzMzflVHKzHzA=; b=RJxrLqfZ/xzZ/H04vTgieHeKrD+eCD71pZKwRNguE1fk5yUvofq2KGBO2RifbupeSf PImVGf/ajG7QhnFOs0YWLsEGVXQ7Pig0CZyDZ2wCX4raVyn7+4xVERMKnMKs7FrvcVxe RLZe70m00FHdrAQjeql3sVTEyfRDsgk5v7T5LfuR7IXt7uIdYr9dKFU86DKoXzG7wFyb kvpAzq0aflAvXcqRGbwXsuQGOkSCfF3R4QC7O7NOoHEqJtFgtplem9s9MN/EZ8Pc1Y7w bMPTHMxznzzaVJTqXpWo5kt0dSnsUxaoBH+iVyxR+s8aCkI+DgoImFzn0XjtufbyPqYI 6sgA== X-Gm-Message-State: ABuFfogTxCPZhbABCULCGd6BZiBk+n+pRlLBq04UJmKyJVXCYkbibocJ hXldrOQO4OjLM+03R55bt2CSgrZCv8psdv0CC/eV+A== X-Google-Smtp-Source: ACcGV606LRAE5fUoQyP7pGPBP3ltaPXRBVSTKjYaqkORID371gRbPcsm5s8Tsfn2UVtrtu1/eafnbFdO/EK8mE6qfUE= X-Received: by 2002:a67:4dc7:: with SMTP id i68mr18183109vsg.46.1540178942064; Sun, 21 Oct 2018 20:29:02 -0700 (PDT) MIME-Version: 1.0 References: <79973E2B-F5C4-4E7C-B92B-1C8D4441C7D1@yahoo.com> In-Reply-To: From: Warner Losh Date: Sun, 21 Oct 2018 21:28:51 -0600 Message-ID: Subject: Re: head -r338804 boots threadripper 1950X fine; head -r338810+ do not; -r338807 seems implicated To: Mark Millard Cc: Konstantin Belousov , FreeBSD Current , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 03:29:03 -0000 On Sun, Oct 21, 2018 at 8:57 PM Mark Millard via freebsd-stable < freebsd-stable@freebsd.org> wrote: > [I built based on WITHOUT_ZFS= for other reasons. But, > after installing the build, Hyper-V based boots are > working.] > > On 2018-Oct-20, at 2:09 AM, Mark Millard wrote: > > > On 2018-Oct-20, at 1:39 AM, Mark Millard wrote: > > > >> I attempted to jump from head -r334014 to -r339076 > >> on a threadripper 1950X board and the boot fails. > >> This is both native booting and under Hyper-V, > >> same machine and root file system in both cases. > > > > I did my investigation under Hyper-V after seeing > > a boot failure native. > > > > Looks like the native failure is even earlier, > > before db> is even possible, possibly during > > early loader activity. > > > > So this report is really for running under > > Hyper-V: -r338804 boots and -r338810 does > > not. By contrast -r334804 does not boot native. > > (But I've little information for that context.) > > > > Sorry for the confusion. I rushed the report > > in hopes of getting to sleep. It was not to be. > > > >> It fails just after the FreeBSD/SMP lines, > >> reporting "kernel trap 9 with interrupts disabled". > >> > >> It fails in pmap_force_invaldiate_cache_range at > >> a clflusl (%rax) instruction that produces a > >> "Fatal trap 9: general protection fault while > >> in kernel mode". cpudid=0 apic id= 00 > >> > >> I used kernel.txz files from: > >> > >> https://artifact.ci.freebsd.org/snapshot/head/r*/amd64/amd64/ > >> > >> to narrow the range of kernel builds for working -> failing > >> and got: > >> > >> -r338804 boots fine > >> (no amd64 kernel builds between to try) > >> -r338810+ fails (any that I tried, anyway) > >> > >> In that range is -r338807 : > >> > >> QUOTE > >> Author: kib > >> Date: Wed Sep 19 19:35:02 2018 > >> New Revision: 338807 > >> URL: > >> https://svnweb.freebsd.org/changeset/base/338807 > >> > >> > >> Log: > >> Convert x86 cache invalidation functions to ifuncs. > >> > >> This simplifies the runtime logic and reduces the number of > >> runtime-constant branches. > >> > >> Reviewed by: alc, markj > >> Sponsored by: The FreeBSD Foundation > >> Approved by: re (gjb) > >> Differential revision: > >> https://reviews.freebsd.org/D16736 > >> > >> Modified: > >> head/sys/amd64/amd64/pmap.c > >> head/sys/amd64/include/pmap.h > >> head/sys/dev/drm2/drm_os_freebsd.c > >> head/sys/dev/drm2/i915/intel_ringbuffer.c > >> head/sys/i386/i386/pmap.c > >> head/sys/i386/i386/vm_machdep.c > >> head/sys/i386/include/pmap.h > >> head/sys/x86/iommu/intel_utils.c > >> END QUOTE > >> > >> There do seem to be changes associated with > >> clflush(...) use. Looking at: > >> > >> > https://svnweb.freebsd.org/base/head/sys/amd64/amd64/pmap.c?annotate=339432 > >> > >> it appears that pmap_force_invalidate_cache_range has not > >> changed since -r338807. > >> > >> It seems that -r338806 and -r3388810 would be unlikely > >> contributors. > > > > I went after my native-boot loader problem first because I > could switch kernels via the loader for booting FreeBSD under > Hyper-V. Switching loaders is more of a problem. > > In order to avoid the loader-time crash I switched to building > installing based on WITHOUT_ZFS= . I've had no active use of > ZFS in years. (The old official-build loaders that worked were > non-ZFS ones.) > > This took care of the native-boot loader-crash --and, to my > surprise, also the Hyper-V-boot kernel-time crash. > > My private builds now boot the 1950X in both contexts just > fine. > > During my early investigation I did pick up specific changes > from after -r339076 that seemed to be tied to Ryzen and such. > (They made no difference to the boot problems at the time > but I saw no reason to remove them.) > > # uname -apKU > FreeBSD FBSDFSSD 12.0-ALPHA8 FreeBSD 12.0-ALPHA8 #5 r339076:339432M: Sun > Oct 21 16:44:25 PDT 2018 markmi@FBSDFSSD:/usr/obj/amd64_clang/amd64.amd64/usr/src/amd64.amd64/sys/GENERIC-NODBG > amd64 amd64 1200084 1200084 > The phrase "no active use" bothers me. What does that mean? Are there any Warner