From owner-freebsd-stable@freebsd.org Tue Jun 11 05:15:15 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E271615AD99C for ; Tue, 11 Jun 2019 05:15:14 +0000 (UTC) (envelope-from koobs.freebsd@gmail.com) Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6464474685 for ; Tue, 11 Jun 2019 05:15:13 +0000 (UTC) (envelope-from koobs.freebsd@gmail.com) Received: by mail-pf1-x441.google.com with SMTP id q10so6626191pff.9 for ; Mon, 10 Jun 2019 22:15:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:reply-to:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=RVO8V7YY6wQVE468YUrBeMWrmdyVn20hY5g2ZvC1hxk=; b=GeIGolnG3MdIbNmR1vNyTx8S+k54srx2g9oRc3MC/sOg3QPYY8oFp/QjA5dicGG7v0 8duWOl9xk6qcTvUsUlc25rZv1hHgpgiNEHNDjwtUW5oPs4+MKpkov81j/coGjuIVFMhA 8c4CLrLJQbyzPYGSPy59ZjSDX1CKfXudgtBCKL9U0qKRzFpk0GgJkONdaD80iE58nKv3 noOUBm2+Is+50wUJYtfiKXlJWRopVME9Mu4zzBUCysN00SCr4Wz9CCehh1Bn/RnClmnr /TwmSFR6wncZUHeOA1Q0Lv6uevlaw7kM9RqTDUk6/2u/XUFIkwxwwzMsfD/px49RdDx7 /u9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:reply-to:subject:to:references:from :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=RVO8V7YY6wQVE468YUrBeMWrmdyVn20hY5g2ZvC1hxk=; b=fjzEwaZ/vY1v+zjrTa18FtgYytP2e2KgrdtaNVeJgYeCUDrKM895ZVcn18vDCUEqcc pNWilGLkeA3/cbComTkuveehxQaIXYwuxSoJyO4MLhCsUBWVxKDoNQWvJKP7fBx0fOsU l/yJlcJ1D0jEgJMWzT+Leyv7LDeIw+WNiAg5vdPXLmr2BwBmCfwWoIa0Lk8/ip7iDVSC jzKQQGE7nlpJxEtboBNJw6DEdVKgR6Wo5a6FLfgPbgbsgMEPUP4CanXtyPq8rTxVxD6C 8Um2Y6uoMEP2R1IpquaoIpmTiKNR6S/uSw85ZJrjNs50AFuUMBN0uqrl25pk/IZ/+UNp N9Ng== X-Gm-Message-State: APjAAAV1xUhunhaKrNbmPYDX4HEX3bp0+3tSQRFYUXmeR8qoLh+19hh1 G/qyQcKowaRjIGwldJ0bLdkLXmwN X-Google-Smtp-Source: APXvYqyudzHkhgeyfYz47iohcmK/XZlILi63RuKHiVkB06uoovZ0X/TvRYiktUFI3tdmI3vwHsH1iQ== X-Received: by 2002:a65:5787:: with SMTP id b7mr18982204pgr.148.1560230112063; Mon, 10 Jun 2019 22:15:12 -0700 (PDT) Received: from [192.168.1.100] (180-150-68-130.b49644.syd.nbn.aussiebb.net. [180.150.68.130]) by smtp.gmail.com with ESMTPSA id 131sm7939935pfx.57.2019.06.10.22.15.10 (version=TLS1_3 cipher=AEAD-AES128-GCM-SHA256 bits=128/128); Mon, 10 Jun 2019 22:15:11 -0700 (PDT) Sender: Kubilay Kocak Reply-To: koobs@FreeBSD.org Subject: Re: Kernel panic on 12-STABLE-r348203 amd64 To: Mark Saad , FreeBSD-Stable ML References: From: Kubilay Kocak Message-ID: Date: Tue, 11 Jun 2019 15:15:08 +1000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Thunderbird/67.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 6464474685 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=GeIGolnG; spf=pass (mx1.freebsd.org: domain of koobsfreebsd@gmail.com designates 2607:f8b0:4864:20::441 as permitted sender) smtp.mailfrom=koobsfreebsd@gmail.com X-Spamd-Result: default: False [-4.16 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[koobs@FreeBSD.org]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FORGED_SENDER(0.30)[koobs@FreeBSD.org,koobsfreebsd@gmail.com]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FROM_NEQ_ENVFROM(0.00)[koobs@FreeBSD.org,koobsfreebsd@gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; TAGGED_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.92)[-0.923,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[1.4.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.03)[ip: (0.40), ipnet: 2607:f8b0::/32(-3.18), asn: 15169(-2.29), country: US(-0.06)] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Jun 2019 05:15:15 -0000 On 6/06/2019 5:04 am, Mark Saad wrote: > On Wed, Jun 5, 2019 at 2:42 PM Mark Saad wrote: >> >> On Wed, Jun 5, 2019 at 12:29 PM Mark Saad wrote: >>> >>> All >>> I was wondering if anyone could shed some light on this boot panic I >>> saw yesterday. This is on a Dell R630 with Bios 2.9.1 booting >>> 12.0-STABLE-r348203 amd64. >>> I reverted this back to 12.0-RELEASE-p4 and its fine . >>> >>> The only custom options I had were in loader.conf >>> >>> kern.geom.label.gptid.enable="0" >>> ipmi_load="YES" >>> boot_multicons="YES" >>> boot_serial="YES" >>> console="comconsole,vidconsole" >>> net.inet.tcp.tso="0" >>> cc_htcp_load="YES" >>> autoboot_delay="5" >>> hw.mfi.mrsas_enable="1" >>> hw.usb.no_pf="1" # Disable USB packet filtering >>> hw.usb.no_shutdown_wait="1" >>> hw.vga.textmode="1" # Text mode >>> machdep.hyperthreading_allowed="0" >>> >>> Any ideas ? >>> >>> Screen shot here >>> https://imgur.com/a/nGvHtIs >>> >>> -- >>> mark saad | nonesuch@longcount.org >> >> Plain text version of the crash >> >> Loading kernel... >> /boot/kernel/kernel text=0x168d811 data=0x1cf968+0x768c80 >> syms=[0x8+0x1778e8+0x8 / >> +0x194f1d] >> Loading configured modules... >> /boot/kernel/ipmi.ko size 0x11e10 at 0x2645000 >> loading required module 'smbus' >> /boot/kernel/smbus.ko size 0x2ef0 at 0x2657000 >> /boot/entropy size=0x1000 >> /boot/kernel/cc_httcp.ko size 0x2330 at 0x265b000 >> ---<>---c_hmodule 'smbus' >> Copyright (c) 1992-2019 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >> The Regents of the University of California. All rights reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 12.0-STABLE r348693 GENERIC amd64 >> FreeBSD clang version 8.0.0 (tags/RELEASE_800/final 356365) (based on >> LLVM 8.0.0) >> panic: UMA zone "UMA Zones": Increase vm.boot_pages >> cpuid = 0 >> time = 1 >> KDB: stack backtrace: >> #0 0xffffffff80c16df7 at ??+0 >> #1 0xffffffff80bcaccd at ??+0 >> #2 0xffffffff80bcab23 at ??+0 >> #3 0xffffffff80f0b03c at ??+0 >> #4 0xffffffff80f08d8d at ??+0 >> #5 0xffffffff80f0bb3d at ??+0 >> #6 0xffffffff80f0b301 at ??+0 >> #7 0xffffffff80f0b3d1 at ??+0 >> #8 0xffffffff80f066c4 at ??+0 >> #9 0xffffffff80f0543f at ??+0 >> #10 0xffffffff80f23aef at ??+0 >> #11 0xffffffff80f1133b at ??+0 >> #12 0xffffffff80b619c8 at ??+0 >> #13 0xffffffff8036a02c at ??+0 >> Uptime: 1s >> >> >> Also increasing the vm.boot_pages to 128 in the loader works. Anyone >> know why ? This box has 64G ram. >> >> -- >> mark saad | nonesuch@longcount.org > > So after some poking in the bios this has to do with how the Dell NUMA > options are set. If the system is set Cluster On Die mode, you get a > kernel panic > Home Snoop or Early Snoop no issue. > > Hi Mark, Could you report this bug (Bugzilla) if you haven't already, providing: - exact freebsd version(s) reproducible with - panic/backtrace output as an attachment. Ideally with a debug kernel - /var/run/dmesg.boot output (as an attachment) in a verbose boot - if you can test a current snapshot, that would be great - any other system information you believe might be helpful in isolating root cause(s) or potential fixes Thanks! Feel free to CC me on it