From owner-freebsd-stable@freebsd.org Sun Jan 31 09:03:33 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4BA9E4E8C62 for ; Sun, 31 Jan 2021 09:03:33 +0000 (UTC) (envelope-from szymluk86@gmail.com) Received: from mail-oo1-xc2d.google.com (mail-oo1-xc2d.google.com [IPv6:2607:f8b0:4864:20::c2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DT4ql59Q9z4mp9 for ; Sun, 31 Jan 2021 09:03:31 +0000 (UTC) (envelope-from szymluk86@gmail.com) Received: by mail-oo1-xc2d.google.com with SMTP id z36so3475689ooi.6 for ; Sun, 31 Jan 2021 01:03:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=aLjIa9JkbVKEl6ULhaBwdUFwXbwcnC+PH+r94shCG2M=; b=TrJsqThD9Nk/BA4qkniBIist+2NyvcAxhQZ+QiRjo8FttAFkPqo4t4Ycu0FyfT/SjX K99Vz77UYU/z4K3coRyKGWGyDbD0pl+UxtOHrVG7+yKC3yYoKl4aNPXC6YeqFzyvs5y+ XwF6sLIR7n1f1I9yxbPkUOIDOJ+2FO5xfkBIX0/x2HNqWhwo07C/FJGz/HsMbbLV8Nlq dTBI0q7X948VvpOSpzkNJAP3Eizuo5Udd5cNJxQqKhrxZ5bSJ9QIP9VPlsc3Jx/DWvYM aeVrDfGHsZG1PMAwR3srYooJdBOLWpydP5KDJ+XgCxSzrKjnEZ8Ep/AGF/CXUqPJG2kj Ht0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=aLjIa9JkbVKEl6ULhaBwdUFwXbwcnC+PH+r94shCG2M=; b=PK78rwsVOUfAnizikUF5ss2X7tDxTNTtuTeJ0DlQ9DVp6md9y0piG1OzLr3unyURK5 lom1yu+w+s4kUijkeazKXUyiofFp4MLbUlfi7ckQqlQ/d4dkToA7fA7fHqy5ciz5NmIr YCHYfkW0vn9TyEHDXaeja2P1+kvgPbHpebmDy+puJ0/uBfGNcWmOCg4RTtL3s/oMTbmg u2ZVAKK12Ybi40aJ893kxdulLpInyXzcwJEuBytsSJ4VWhBk5vneYjaOx7Q9CDupSFTI ApgSc6fv4jjihSI1+OSeuv7orra68iwB40e8lXtzhuRVpYwf6vXFLHtanHnkS2EeKVez DXsg== X-Gm-Message-State: AOAM530nUO1y3fpeDe7jf7U3xm9qGS/78gPXT9nF8oN2INlJOe1XbvoU ZQ5uHTMi3ONm3ClaY+Kw74NOKAqmcTYkgeXl9RK8GTsPAjRjAA== X-Google-Smtp-Source: ABdhPJzsjDXoG7qLno2QBShjemGSRaAyxAL/vvrN4w+uLaxnxdGQeJ1OaSQYrRcWVDRsUnrAGaT4x6KKvhYaG2575jQ= X-Received: by 2002:a4a:d891:: with SMTP id b17mr8426792oov.28.1612083810326; Sun, 31 Jan 2021 01:03:30 -0800 (PST) MIME-Version: 1.0 From: =?UTF-8?B?xZtydWJva3LEmXQ=?= Date: Sun, 31 Jan 2021 10:03:19 +0100 Message-ID: Subject: devel/libphonenumber - problem with updating port To: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4DT4ql59Q9z4mp9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=TrJsqThD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of szymluk86@gmail.com designates 2607:f8b0:4864:20::c2d as permitted sender) smtp.mailfrom=szymluk86@gmail.com X-Spamd-Result: default: False [-2.72 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.97)[-0.974]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_MIXED_CHARSET(1.25)[subject]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::c2d:from]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::c2d:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::c2d:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2021 09:03:33 -0000 Hi, When I try update devel/libphonenumber port from version 8.12.16 to 8.12.17, errors appears: In file included from /usr/local/include/phonenumbers/asyoutypeformatter.h:43: /usr/local/include/phonenumbers/phonemetadata.pb.h:17:2: error: This file was generated by an o lder version of protoc which is #error This file was generated by an older version of protoc which is ^ /usr/local/include/phonenumbers/phonemetadata.pb.h:18:2: error: incompatible with your Protocol Buffer headers. Please #error incompatible with your Protocol Buffer headers. Please ^ /usr/local/include/phonenumbers/phonemetadata.pb.h:19:2: error: regenerate this file with a new er version of protoc. #error regenerate this file with a newer version of protoc. ^ /usr/local/include/phonenumbers/phonemetadata.pb.h:28:10: fatal error: 'google/protobuf/inlined _string_field.h' file not found #include ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 4 errors generated. *** Error code 1 Stop. make[3]: stopped in /usr/ports/devel/libphonenumber/work/.build *** Error code 1 Stop. make[2]: stopped in /usr/ports/devel/libphonenumber/work/.build *** Error code 1 Stop. make[1]: stopped in /usr/ports/devel/libphonenumber/work/.build *** Error code 1 Stop. make: stopped in /usr/ports/devel/libphonenumber ===>>> make build failed for devel/libphonenumber ===>>> Aborting update ===>>> You can restart from the point of failure with this command line: portmaster devel/libphonenumber This command has been saved to ~/portmasterfail.txt I re-built devel/protobuf port, but this action didn't solve my problem. I use FreeBSD 13.0-ALPHA2 branch. What to do? Thanks for help :) From owner-freebsd-stable@freebsd.org Sun Jan 31 11:04:59 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EDC384EFB19; Sun, 31 Jan 2021 11:04:59 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DT7Wt0nddz4vCM; Sun, 31 Jan 2021 11:04:57 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1612091096; bh=GJMSRb/re5esuZhopGt/XqISTiQ0iTxdRpikeTepAA4=; h=X-UI-Sender-Class:From:To:Subject:Date; b=UG8QS6JXTm00/Vkp+DMsCMcwX3gz869ZMEgu9Vtzlj3KuOiqLKGXuv8Z/mADOAZGB 63RUAoP/mbEgbHrRhD/hhXBHjxmgsbPlioBW2VxF82gq0Z5yrYJSvec6E1yerSd7lO en3y3I5RuBGzNZDe/8mXsuV9r50prKdHQuWbrqYI= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [46.142.71.68] ([46.142.71.68]) by web-mail.gmx.net (3c-app-gmx-bap35.server.lan [172.19.172.105]) (via HTTP); Sun, 31 Jan 2021 12:04:55 +0100 MIME-Version: 1.0 Message-ID: From: mj-mailinglist@gmx.de To: freebsd-stable@freebsd.org, freebsd-pkgbase@freebsd.org Subject: pkgbase: Missing /etc/master.passwd in FreeBSD-runtime 12-stable pkg Content-Type: text/plain; charset=UTF-8 Date: Sun, 31 Jan 2021 12:04:55 +0100 Importance: normal Sensitivity: Normal X-Priority: 3 X-Provags-ID: V03:K1:uYnQTeZgihyxropNTYoW8Wg+wC79C9VDbRPuE57Cmz4NrysW1VRpR8uuiyAZ2ULuv6tum T/B5Y3vIwt3mYHncR6rj1bMmKzE96EFMIjSJljTkxlfiETNAp7rNWO4Mxm4Yt8xMVc0vWtlrtKzO xTzramPvi03G9UqlKw32O/irpq545jSIsK+uCeb3daAY6w6oYlxr9NcL6xNGK5LPV1Qp2DlQ4ucw cwVQKfQ5HoqUkPAjOZ8vXx/gQsjP/SlnZTBehSHMNG4ZbyEA0J5FUmYhi5yM+KCHNVGYoxhiA32C fc= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:atdtl4VysZ4=:6M8788ZxC3kyw9wtOvXMxD 6FDyMXziX1bZcXvR6uy2mcijPATeJzBPSBbSIo5DTPBb6cq0ZAXxjP2U/oElFCjXXYtPKDKxz gZXU+83Kd/SFbxtHOVkTpzqkFrZIF4dkxYBdjYdnJf0l1H9MqNtflEAyYUiT0kHjYYjE3W6+K 6CUDL2zRLmY2bSxkiLSRUkpqpKWdsqFw/5MwUvWFoec8kSFuoZhoG1zPb5yDexpvMuDwO79ee gBCwJlf+/jdgt5/BJLtEBzfdhk0Pw/+fwn9ZDEHxZGHYhUCG95O+b24jF7LjDE+i4UpmxYH9u 1I+qk1KhBA4YxnkGefKWaIxZS+Glpxqkxw/uCKbhaQaFWog88z93O/TTAI1/1usorzZaZb044 swbx7jWS0gprZOU7YDOzz5w5f2n3wQJDKPXcWdeI3petzGaY1GL0IwKJzWes1alO2427qNbZi 6nC45O6GuSNpOOYYKPqhMKFEsYgHXp+6frIP+eJfMHoYXOD/SuMcks7XFO75cRPYhbQo46T5k kYGbK5V3wr1dHkR45bTWC4eWQp0TvnGX86+OrnSwwI3Fo+bhehaE+bxfcmmsl0bOkTbQQi9Lv F113w1tjrWHSM= X-Rspamd-Queue-Id: 4DT7Wt0nddz4vCM X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=UG8QS6JX; dmarc=pass (policy=none) header.from=gmx.de; spf=pass (mx1.freebsd.org: domain of mj-mailinglist@gmx.de designates 212.227.17.22 as permitted sender) smtp.mailfrom=mj-mailinglist@gmx.de X-Spamd-Result: default: False [-3.60 / 15.00]; RWL_MAILSPIKE_GOOD(0.00)[212.227.17.22:from]; FREEMAIL_FROM(0.00)[gmx.de]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; RCPT_COUNT_TWO(0.00)[2]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmx.de,none]; RECEIVED_SPAMHAUS_PBL(0.00)[46.142.71.68:received]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; RBL_DBL_DONT_QUERY_IPS(0.00)[212.227.17.22:from]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.17.22:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[212.227.17.22:from:127.0.2.255]; FROM_NO_DN(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable,freebsd-pkgbase] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2021 11:05:00 -0000 I noticed that my jails, build from a recent pkg base don't start: ... Starting jails: cannot start jail "j6": 7 jail: j6: getpwnam root: No such file or directory jail: j6: /bin/sh /etc/rc: failed . Looking at the installation process, i see this: ... [2/79] Installing FreeBSD-runtime-stable12-c263-gf56d7f838... [2/79] Extracting FreeBSD-runtime-stable12-c263-gf56d7f838: .......... done pwd_mkdb: /jails/j6/etc/master.passwd: No such file or directory [3/79] Installing FreeBSD-libcasper-stable12-c263-gf56d7f838... [3/79] Extracting FreeBSD-libcasper-stable12-c263-gf56d7f838: .......... done ... Comparing the FreeBSD-runtime pkg content of "r368786M" with "stable/12-c263-gf56d7f838" shows, that the following files are not in the newer package: termcap.small, rc.sendmail, rc.firewall, rc.bsdextended, master.passwd, login.access, group Looking in METALOG on fbsd12 from yesterday (stable/12-c263-gf56d7f838): fbsd12> grep "master.passwd" /usr/obj/usr/src/amd64.amd64/worldstage/METALOG ./etc/master.passwd type=file uname=root gname=wheel mode=0600 size=1797 ./usr/tests/usr.sbin/pw/master.passwd type=file uname=root gname=wheel mode=0444 size=101 tags=package=tests ./usr/share/man/man5/master.passwd.5.gz type=file uname=root gname=wheel mode=0444 size=4337 tags=package=runtime-manuals fbsd12> grep "master.passwd" /usr/obj/usr/src/amd64.amd64/worldstage/*.plist /usr/obj/usr/src/amd64.amd64/worldstage/runtime-manuals.plist:@(root,wheel,0444,) /usr/share/man/man5/master.passwd.5.gz /usr/obj/usr/src/amd64.amd64/worldstage/tests.plist:@(root,wheel,0444,) /usr/tests/usr.sbin/pw/master.passwd so, in fbsd12 the package information for the file is missing. in fbsd13 the information is there: fbsd13> grep "master.passwd" /usr/obj/usr/src/amd64.amd64/worldstage/METALOG ./etc/master.passwd type=file uname=root gname=wheel mode=0600 size=1797 tags=package=runtime,config ./usr/share/man/man5/master.passwd.5.gz type=file uname=root gname=wheel mode=0444 size=4337 tags=package=utilities ./usr/tests/usr.sbin/pw/master.passwd type=file uname=root gname=wheel mode=0444 size=101 tags=package=tests fbsd13> grep "master.passwd" /usr/obj/usr/src/amd64.amd64/worldstage/*.plist /usr/obj/usr/src/amd64.amd64/worldstage/runtime.plist:@config(root,wheel,0600,) /etc/master.passwd /usr/obj/usr/src/amd64.amd64/worldstage/tests.plist:@(root,wheel,0444,) /usr/tests/usr.sbin/pw/master.passwd /usr/obj/usr/src/amd64.amd64/worldstage/utilities.plist:@(root,wheel,0444,) /usr/share/man/man5/master.passwd.5.gz I don't have the old METALOG files but in the package of r368786M the files are there. In my next pkgbase build from 31.12.2020 (stable/12-c58-g53bc32215) they are missing. Is this a regression, or did i miss something? -- Martin From owner-freebsd-stable@freebsd.org Sun Jan 31 17:12:45 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D3F9C4FD473 for ; Sun, 31 Jan 2021 17:12:45 +0000 (UTC) (envelope-from peo@nethead.se) Received: from ns1.nethead.se (ns1.nethead.se [5.150.237.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "ns1.nethead.se", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DTHhD66Tyz3qjd for ; Sun, 31 Jan 2021 17:12:44 +0000 (UTC) (envelope-from peo@nethead.se) X-Virus-Scanned: amavisd-new at Nethead AB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nethead.se; s=NETHEADSE; t=1612113153; bh=lFSuJyDD46F82Vnr/j4cLsMxzz4rGUQeTNzGiFRJY8Y=; h=Subject:To:References:From:Date:In-Reply-To; b=LWnsLn2tCfQlJ8tTv2Fe1O5UZO6eAXCbj5g54GA4M9tC/dHojmQ4vmgUkGi0slWnP q6MNmVGUL4yL0Ae2UCDKxlqk2oUSXKbrexTDMKuE+egiZNfYypqAc5IVEH9wVTDLEH O4cbuSnUlUtEASNTCRKUaXuyBmLWV1sYULQB//RA= Subject: Re: pkgbase: Missing /etc/master.passwd in FreeBSD-runtime 12-stable pkg To: freebsd-stable@freebsd.org References: From: Per olof Ljungmark Message-ID: Date: Sun, 31 Jan 2021 18:12:31 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DTHhD66Tyz3qjd X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=nethead.se header.s=NETHEADSE header.b=LWnsLn2t; dmarc=pass (policy=none) header.from=nethead.se; spf=pass (mx1.freebsd.org: domain of peo@nethead.se designates 5.150.237.139 as permitted sender) smtp.mailfrom=peo@nethead.se X-Spamd-Result: default: False [-4.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[nethead.se:s=NETHEADSE]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.150.237.139]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[5.150.237.139:from:127.0.2.255]; DKIM_TRACE(0.00)[nethead.se:+]; DMARC_POLICY_ALLOW(-0.50)[nethead.se,none]; NEURAL_HAM_SHORT(-1.00)[-0.998]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[5.150.237.139:from]; ASN(0.00)[asn:8473, ipnet:5.150.192.0/18, country:SE]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2021 17:12:45 -0000 On 1/31/21 12:04 PM, mj-mailinglist@gmx.de wrote: > I noticed that my jails, build from a recent pkg base don't start: > ... > Starting jails: cannot start jail "j6": > 7 > jail: j6: getpwnam root: No such file or directory > jail: j6: /bin/sh /etc/rc: failed > . > [snip] Just a thought, did you run make distribution DESTDIR= Per From owner-freebsd-stable@freebsd.org Sun Jan 31 17:18:55 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 065A34FDC19 for ; Sun, 31 Jan 2021 17:18:55 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DTHqL6nNhz3rNq for ; Sun, 31 Jan 2021 17:18:54 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id D1F5B4CB2 for ; Sun, 31 Jan 2021 17:18:54 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qt1-f175.google.com with SMTP id d15so10548360qtw.12 for ; Sun, 31 Jan 2021 09:18:54 -0800 (PST) X-Gm-Message-State: AOAM5318f/9QeJtVBmpz6l+IX7oSJu9aUJTbr0Wx1xKdN3KIej/f5kCD tK8QG/UdHBsjRG4RqB2ZTG47NJPuzBcBmgrjkKc= X-Google-Smtp-Source: ABdhPJydHuW1Ap7/N17Q6BUiT6Wlv33qq0sjYeq1JpaRkvSglXRLUzYUV0E7fMcxS1nh9C8ovCOcsqQgABFRwdOvLlU= X-Received: by 2002:ac8:100b:: with SMTP id z11mr12064310qti.60.1612113534450; Sun, 31 Jan 2021 09:18:54 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Kyle Evans Date: Sun, 31 Jan 2021 11:18:41 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: pkgbase: Missing /etc/master.passwd in FreeBSD-runtime 12-stable pkg To: Per olof Ljungmark Cc: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2021 17:18:55 -0000 On Sun, Jan 31, 2021 at 11:12 AM Per olof Ljungmark wrote: > > On 1/31/21 12:04 PM, mj-mailinglist@gmx.de wrote: > > I noticed that my jails, build from a recent pkg base don't start: > > ... > > Starting jails: cannot start jail "j6": > > 7 > > jail: j6: getpwnam root: No such file or directory > > jail: j6: /bin/sh /etc/rc: failed > > . > > > > [snip] > > Just a thought, did you run > > make distribution DESTDIR= This shouldn't be necessary with pkgbase, but I haven't looked at pkgbase in 12 on a long time -- it sounds like we managed to regress it somewhere. From owner-freebsd-stable@freebsd.org Sun Jan 31 22:15:21 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BC1EE4EA119 for ; Sun, 31 Jan 2021 22:15:21 +0000 (UTC) (envelope-from juraj@lutter.sk) Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DTQPN1qK3z4mR2 for ; Sun, 31 Jan 2021 22:15:19 +0000 (UTC) (envelope-from juraj@lutter.sk) Received: from chemex.owhome.lan (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 0850B45D0B3 for ; Sun, 31 Jan 2021 23:15:12 +0100 (CET) From: Juraj Lutter Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Weird loader behavior Message-Id: <6059C0E7-20CC-4B43-86DF-5EE2DD66818C@lutter.sk> Date: Sun, 31 Jan 2021 23:15:10 +0100 To: freebsd-stable@freebsd.org X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Rspamd-Queue-Id: 4DTQPN1qK3z4mR2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of juraj@lutter.sk designates 92.60.51.55 as permitted sender) smtp.mailfrom=juraj@lutter.sk X-Spamd-Result: default: False [-2.79 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[92.60.51.55:from]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[92.60.51.55:from:127.0.2.255]; ARC_NA(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:92.60.51.55]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.986]; DMARC_NA(0.00)[lutter.sk]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:44185, ipnet:92.60.48.0/22, country:SK]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2021 22:15:21 -0000 Hi, after a routine buildworld/kernel/installworld/kernel/etcupdate on a = stable/12 bhyve guest, loader started to spit: Consoles: EFI console Reading loader env vars from /efi/freebsd/loader.env Setting currdev to disk0p1: FreeBSD/amd64 EFI loader, Revision 1.1 Command line arguments: loader.efi EFI version: 2.40 EFI Firmware: BHYVE (rev 1.00) Console: efi (0x20001000) Load Path: \EFI\BOOT\BOOTX64.EFI Load Device: = PciRoot(0x0)/Pci(0x3,0x0)/HD(1,GPT,A7A0422F-6F85-11EA-8A55-00A09858FAF5,0x= 28,0x64000) BootCurrent: 0000 BootOrder: 0000[*] 0001 0002 0003 BootInfo Path: PciRoot(0x0)/Pci(0x3,0x0) Ignoring Boot0000: Only one DP found Trying ESP: = PciRoot(0x0)/Pci(0x3,0x0)/HD(1,GPT,A7A0422F-6F85-11EA-8A55-00A09858FAF5,0x= 28,0x64000) Setting currdev to disk0p1: Trying: = PciRoot(0x0)/Pci(0x3,0x0)/HD(2,GPT,A7C612E0-6F85-11EA-8A55-00A09858FAF5,0x= 64028,0x400) Setting currdev to disk0p2: Trying: = PciRoot(0x0)/Pci(0x3,0x0)/HD(3,GPT,A7D27D9F-6F85-11EA-8A55-00A09858FAF5,0x= 64800,0x800000) Setting currdev to disk0p3: Trying: = PciRoot(0x0)/Pci(0x3,0x0)/HD(4,GPT,A7D7E55E-6F85-11EA-8A55-00A09858FAF5,0x= 864800,0x479B000) Setting currdev to zfs:zroot/ROOT/default: ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable ERROR: /boot/lua/loader.lua:1: unexpected symbol near '`'. Type '?' for a list of commands, 'help' for more detailed help. OK - KERNCONF is stock GENERIC - problematic version is stable/12-c243335-ge817c8f77fe - Booting off 12.2-STABLE snapshot ISO 20210128-r369150 and using -a = flag to ask for a root mount and specifying zfs:zroot/ROOT/default makes the system = run normally (apropriate modules have to be loaded from within the loadet, = ofc). - using in-tree zfs - zpool scrub did not show any anomalies - gpart bootcode has already been run (after first unsusccessful boot). - Another bhyve guest (13.0-ALPHA3) on the same host works OK Any suggestions what to check, please? Thanks otis =E2=80=94 Juraj Lutter XMPP: juraj (at) lutter.sk GSM: +421907986576 From owner-freebsd-stable@freebsd.org Mon Feb 1 01:02:51 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 92D754EDC37 for ; Mon, 1 Feb 2021 01:02:51 +0000 (UTC) (envelope-from parv.0zero9@gmail.com) Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DTV6f6SbKz4tym for ; Mon, 1 Feb 2021 01:02:50 +0000 (UTC) (envelope-from parv.0zero9@gmail.com) Received: by mail-wm1-x335.google.com with SMTP id u14so11726644wmq.4 for ; Sun, 31 Jan 2021 17:02:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=hicdepRy+9dpMiyhjCyxFNbcLz5C8P8mlsT/ss4Vtf4=; b=ug0lCv6L1cV/723KlKvNsZ4WpP8cjQc5rg1gQ6vpGmKwX8iIh/OU4kRplPy9Fa5Q7K v6yxvd4+cP9ynIoJKhvHZjqbFkaEtOUzhGUd3Eo+/zLcYomObvCZNCtGzcGGE8GB4cdb SGVoFv7aI4HPYy16qFeCX3WPrjNNX6pG4m/6Os30YWxvGEA/P11l1kpC3QqMdJJ2rGZg SIQ4dlhZ4ftTcIDwo2qOk2j/J0/x/wY+c5tdtiOfPrqiL2kXnn2AvJFAPR9EDZ3YYTk+ xMdX/4rRhuD+FFb/b6sYWiSnd/S3ytUR0RYpuCHHVQBJthLWtRo3YUcfbvFmxxSLfd6u ZqDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=hicdepRy+9dpMiyhjCyxFNbcLz5C8P8mlsT/ss4Vtf4=; b=dNoTsX9Qi7KhgGt9c46w/RjbqZBelEIhODXNGsVDPdzw3zSYZXVPq7q3vhoEh5sbOU 7w3tlIHNkSLp6F2hdM9jQxu4ToBHO0l3MSxXypyLEFlqepMGYq2ncsD3iG6f39p3Xarj 63vw2aXJjnbAjlM5PUi4J7U8wfnmL2eie+feUNW0neeF1NCQWMf89RzRhdcoy8Ze6KX6 /wMccGGVAM0992lzcuTJ6YlKp+53m8O5g22nkk9LET5ehaMifY+9fvzyDcFu1EuiOmC2 afL7SVlqmnBbq6W+2/BZ6n4vLGDwVb52Xo9b9Q38DV/bM75i04KGabFAYHtY8wPb+r98 34BA== X-Gm-Message-State: AOAM533QlGgWm5y9tWiKBX6RAXBOstgvYkJbDze3DEuneXiZs+i/4vGN ARk0yPFf40GnegGVCjdr4l1Yxx69XS2TOa7enePl9/5m X-Google-Smtp-Source: ABdhPJwgmhFmrAMdrLg4q/7idaj5J0RaJC8id2ya7bkjvMN1/TnLeuZE8d0trVMtmem4ey01h/T0O53898zG7i8msEA= X-Received: by 2002:a1c:318a:: with SMTP id x132mr3112439wmx.6.1612141369464; Sun, 31 Jan 2021 17:02:49 -0800 (PST) MIME-Version: 1.0 References: <6059C0E7-20CC-4B43-86DF-5EE2DD66818C@lutter.sk> In-Reply-To: From: "parv/freebsd" Date: Sun, 31 Jan 2021 15:01:46 -1000 Message-ID: Subject: Fwd: Re: Weird loader behavior To: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4DTV6f6SbKz4tym X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=ug0lCv6L; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of parv0zero9@gmail.com designates 2a00:1450:4864:20::335 as permitted sender) smtp.mailfrom=parv0zero9@gmail.com X-Spamd-Result: default: False [-3.14 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.14)[-0.143]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::335:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[freebsd]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::335:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::335:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2021 01:02:51 -0000 Forwarding to the list from the address I have actually subscribed to (in "basic HTML mode" Google Mail uses the actual account address not the alias already set up) ... ---------- Forwarded message --------- Date: Sun, Jan 31, 2021 at 2:37 PM Subject: Re: Weird loader behavior To: Juraj Lutter Cc: Juraj L wrote on Jan 31, 2021 at 12:15 PM ... > > Reading loader env vars from /efi/freebsd/loader.env > Setting currdev to disk0p1: ... > Command line arguments: loader.efi > EFI version: 2.40 > EFI Firmware: BHYVE (rev 1.00) > Console: efi (0x20001000) > Load Path: \EFI\BOOT\BOOTX64.EFI > Load Device: PciRoot(0x0)/Pci(0x3,0x0)/HD(1 ,GPT,A7A0422F-6F85-11EA-8A55-00A09858FAF5,0x28,0x64000) ... > Setting currdev to zfs:zroot/ROOT/default: > ZFS: i/o error - all block copies unavailable > ZFS: i/o error - all block copies unavailable > ERROR: /boot/lua/loader.lua:1: unexpected symbol near '`'. ... > - problematic version is stable/12-c243335-ge817c8f77fe Could you show your /boot/loader/loader.conf if you have one? Do you mean yo can boot with git commit 31c234974, one earlier than e817c8f77fe? If not, are you able to boot with even earlier commit of c48240fa6f8? > - Booting off 12.2-STABLE snapshot ISO 20210128-r369150 and using -a flag to ask > for a root mount and specifying zfs:zroot/ROOT/default makes the system run > normally (apropriate modules have to be loaded from within the loadet, ofc). > - using in-tree zfs > - zpool scrub did not show any anomalies > - gpart bootcode has already been run (after first unsusccessful boot). > - Another bhyve guest (13.0-ALPHA3) on the same host works OK What is git commit hash for 13.0-ALPHA3? - parv -- From owner-freebsd-stable@freebsd.org Mon Feb 1 02:42:53 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7196C4F0365; Mon, 1 Feb 2021 02:42:53 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DTXL43HXZz3Flg; Mon, 1 Feb 2021 02:42:52 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id 97D4510197; Mon, 1 Feb 2021 11:42:48 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1612147368; bh=d2NbRRV78s4R4loiNOda/4bCU02srbSRBaJSvqM1RJk=; h=Date:To:Subject:From; b=PADmIsINWPN9vKmjwvdaKStUa2ApkYD8yde4qd2b+gipg97IZTV/3unzjH/ILfqNS GAt4ZtvVAiq0eJa0cR+M4puDeyJfJZ8IOu6eNdgHmKKyugJlT6rq+IcqkQGUoKkeKY S3GUqv7z5+XOX7Y9KprUgBY3muJjpbdl3jBnBe5IrMgDe9RkERHbEwJQlqGSjIwLGE qrnv2UBnSPY2vHCc67CmvvpZpKsF2fXlAESDi4mFCaD3Kkt4wy5IQGqFqWLE/mDJcC 9wpcoCZ6D3P/E6rcJLHGYzLMcjgolC4DrRCT2T6QGEM+2KoqiTB79+lDkw08N10rvB 9YA5q19mGzrRQ== Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id 4F5A0181EA; Mon, 1 Feb 2021 11:42:47 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.0 at eastasia.home.utahime.org Date: Mon, 01 Feb 2021 11:41:35 +0900 (JST) Message-Id: <20210201.114135.1319455396807129550.yasu@utahime.org> To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Setting for displaying utf8 characters on all vt consoles results in panic on 14-CURRENT and 13.0-ALPHA3 From: Yasuhiro Kimura X-Mailer: Mew version 6.8 on Emacs 27.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DTXL43HXZz3Flg X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=PADmIsIN; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [-0.69 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.991]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2021 02:42:53 -0000 To display utf8 characters on all vt console I did following settings. 1. Download GNU Unifont BDF file (http://unifoundry.com/pub/unifont/unifont-13.0.05/font-builds/unifont-13.0.05.bdf.gz) 2. gunzip unifont-13.0.05.bdf.gz 3. vtfontcvt unifont-13.0.05.bdf unifont.fnt 4. cp unifont.fnt /usr/share/vt/fonts 5. Add 'allscreens_flags="-f 8x16 unifont.fnt"' to /etc/rc.conf 6. Add 'hw.vga.textmode=0' to /boot/loader.conf.local 7. shutdown -r now On 12.2-RELEASE and 11.4-RELEASE it works as is expected. But on 14-CURRENT(man) and 13.0-ALPHA3(stable/13) it result in kernel panic. Screen shot of 14-CURRENT. https://www.utahime.org/FreeBSD/panic.20210201.14-CURRENT.png 14-CURRENT(main): yasu@rolling-vm-freebsd1[1006]% uname -a FreeBSD rolling-vm-freebsd1.home.utahime.org 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n244517-f17fc5439f5: Mon Feb 1 10:55:51 JST 2021 rootz@rolling-vm-freebsd1.home.utahime.org:/usr0/freebsd/src/obj/usr0/freebsd/src/git/amd64.amd64/sys/GENERIC amd64 13.0-ALPHA3(stable/13): yasu@rolling-vm-freebsd5[1005]% uname -a FreeBSD rolling-vm-freebsd5.home.utahime.org 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #0 stable/13-c256214-g40cb0344eb2: Mon Feb 1 11:30:28 JST 2021 rootz@rolling-vm-freebsd5.home.utahime.org:/usr0/freebsd/src/obj/usr0/freebsd/src/git/amd64.amd64/sys/GENERIC amd64 --- Yasuhiro Kimura From owner-freebsd-stable@freebsd.org Mon Feb 1 03:35:37 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7A09E4F390F; Mon, 1 Feb 2021 03:35:37 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DTYVw2t8Sz3KZt; Mon, 1 Feb 2021 03:35:35 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id D8EB11017E; Mon, 1 Feb 2021 12:35:31 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1612150531; bh=EjkQKJfWi/kgTY8KQAybwFPde1q/zZj8jSuq2Z6ep2I=; h=Date:To:Subject:From:In-Reply-To:References; b=R6ZcSrn/hsVLx2aLmvi+EaVrLXQDrnXw+1jjKPMJqiIpVFjgpdZ5RApGkIY6ImnqH HJzk3gviLO8eg99/x4odd71oDYkwzQ+IU6HWbCvu5jYOIyBZeKBi7aTVN8pf60Vr+p l/ETwbCAqV8UHtkjUHIR7qj50MxuZey98FKVUXmN64RlhyymDHU0u6sy2EKvYwhyoZ MbRbAtZiUfHCJ7aMX15+Z/1VzuS26/r9pLMKZJqoGNy9JlfwL/2Oajt/gwgW/6VDOE JiIrta+5ASjT5wSO2Eo7A4U9DMydwHB3JWmSnPHGzQrUk2z0BXTZTQtSwPL55peYnl 8otvNAgKhcN+w== Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id D690018383; Mon, 1 Feb 2021 12:35:29 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.0 at eastasia.home.utahime.org Date: Mon, 01 Feb 2021 12:35:12 +0900 (JST) Message-Id: <20210201.123512.2031600289272155094.yasu@utahime.org> To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Setting for displaying utf8 characters on all vt consoles results in panic on 14-CURRENT and 13.0-ALPHA3 From: Yasuhiro Kimura In-Reply-To: <20210201.114135.1319455396807129550.yasu@utahime.org> References: <20210201.114135.1319455396807129550.yasu@utahime.org> X-Mailer: Mew version 6.8 on Emacs 27.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DTYVw2t8Sz3KZt X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=R6ZcSrn/; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [-0.69 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org:c]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.989]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2021 03:35:37 -0000 From: Yasuhiro Kimura Subject: Setting for displaying utf8 characters on all vt consoles results in panic on 14-CURRENT and 13.0-ALPHA3 Date: Mon, 01 Feb 2021 11:41:35 +0900 (JST) > To display utf8 characters on all vt console I did following settings. > > 1. Download GNU Unifont BDF file > (http://unifoundry.com/pub/unifont/unifont-13.0.05/font-builds/unifont-13.0.05.bdf.gz) > 2. gunzip unifont-13.0.05.bdf.gz > 3. vtfontcvt unifont-13.0.05.bdf unifont.fnt > 4. cp unifont.fnt /usr/share/vt/fonts > 5. Add 'allscreens_flags="-f 8x16 unifont.fnt"' to /etc/rc.conf > 6. Add 'hw.vga.textmode=0' to /boot/loader.conf.local > 7. shutdown -r now > > On 12.2-RELEASE and 11.4-RELEASE it works as is expected. But on > 14-CURRENT(man) and 13.0-ALPHA3(stable/13) it result in kernel panic. > > Screen shot of 14-CURRENT. > https://www.utahime.org/FreeBSD/panic.20210201.14-CURRENT.png > > 14-CURRENT(main): > yasu@rolling-vm-freebsd1[1006]% uname -a > FreeBSD rolling-vm-freebsd1.home.utahime.org 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n244517-f17fc5439f5: Mon Feb 1 10:55:51 JST 2021 rootz@rolling-vm-freebsd1.home.utahime.org:/usr0/freebsd/src/obj/usr0/freebsd/src/git/amd64.amd64/sys/GENERIC amd64 > > 13.0-ALPHA3(stable/13): > yasu@rolling-vm-freebsd5[1005]% uname -a > FreeBSD rolling-vm-freebsd5.home.utahime.org 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #0 stable/13-c256214-g40cb0344eb2: Mon Feb 1 11:30:28 JST 2021 rootz@rolling-vm-freebsd5.home.utahime.org:/usr0/freebsd/src/obj/usr0/freebsd/src/git/amd64.amd64/sys/GENERIC amd64 I submitted this problem to Bugzilla. Bug 253147 - Setting for displaying utf8 characters on all vt consoles results in panic on 14-CURRENT and 13.0-ALPHA3 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253147 --- Yasuhiro Kimura From owner-freebsd-stable@freebsd.org Mon Feb 1 21:48:52 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 064AB537FCB for ; Mon, 1 Feb 2021 21:48:52 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DV1mM22STz3HSN for ; Mon, 1 Feb 2021 21:48:50 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: by mail-ej1-x62e.google.com with SMTP id w1so26688530ejf.11 for ; Mon, 01 Feb 2021 13:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=Bklj8j8u0nqyeHCXxV+VKAOL6Wr0y8DkVckVmypNpow=; b=bHxwJIN7V2syJgQ50lpwBNl2nS1EsC1CLa5xlpc90nG2tcGDMEOQnK2mYWWd/wWDw5 9dgF4VvyllMK7AFazw0hsaxAnmdP0rR4swjsTLSa5bYoY/I46TX01eAI/K4rB+gujlA/ dXwkiRGBl4BiV1gPpi4v3l2oTsM0CfgGWHeVn4mAv5HbNP0DpoyU22bSJbC/tUwf/a7K rLLdfTR8fXNsnr/LrQtGP/2H9SCPFudIS40VLlzwNX7esQkeKHogDZD26CaPWWF59Yx4 r0w2MsPLyloo7K4ypK+Eay7jRp1uD2W0f82nwczIY1tc2O+r8bzOebwisktM9hK0+uXj 0Syg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding:content-language; bh=Bklj8j8u0nqyeHCXxV+VKAOL6Wr0y8DkVckVmypNpow=; b=n6Bf21XieRCE5FN5hSWlg/R2W1TGucmacPv5y0B/2hd8hpvWVhCga5y7p4zpajvzxP r9YR5xvHl5vxevx/zsOqwwE2OpKgvRLxVHpcPJnrS3algnjTM1WwS16dZMcXWSrHBBh2 IREYJWhI4rufRkB42ke6vny/xTHEXfcmNIEO9Z3h+hv6Setft4L9e7cort1wy3SOY7W9 0sXy3Qqh1z1KtSGRWpoeSaTTXjhmfjHWftEeu+gxjKYj9sNDj7owGPp9RTqcM0yERNbu 9M7Fkyb/wM8n78deo2kWbWDTMzbQMmoQ9g/GzH0zDgYelyO65ODtVI05pwfcuyuG1Cb9 CUJg== X-Gm-Message-State: AOAM533M1yEuFB8nHVclTqpA9bxCZl+ovzNefEPBosi9NRdaPJxNy7e0 r5dLI/fYEAECyW4v4CsYwZ0IdkySncw= X-Google-Smtp-Source: ABdhPJzZgxnVlqozJOrCBLoTZAHpZwAYVsChhZcTk3NuTtCVZKRzjaTc0HYbEx8lMw2ozZTuUMBedg== X-Received: by 2002:a17:906:c954:: with SMTP id fw20mr19479267ejb.342.1612216129138; Mon, 01 Feb 2021 13:48:49 -0800 (PST) Received: from MacBook-Pro-van-Johan.local (85-147-130-226.cable.dynamic.v4.ziggo.nl. [85.147.130.226]) by smtp.gmail.com with ESMTPSA id h16sm9243813edw.34.2021.02.01.13.48.48 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 01 Feb 2021 13:48:48 -0800 (PST) To: freebsd-stable@freebsd.org From: Johan Hendriks Subject: latest stable13.0-ALPHA3 can not start varnish anymore. Message-ID: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> Date: Mon, 1 Feb 2021 22:48:42 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4DV1mM22STz3HSN X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=bHxwJIN7; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of johhendriks@gmail.com designates 2a00:1450:4864:20::62e as permitted sender) smtp.mailfrom=johhendriks@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::62e:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::62e:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62e:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2021 21:48:52 -0000 I just updated my FreeBSD 13.0-APLPHA3 to the latest revision and now i can not start varnish anymore. This is on two machines. if i start varnish it errors out as the startup script does a config file check. If i try to do a test of the config file i get the following error. root@jhost002:~ # varnishd -C -f /usr/local/etc/varnish/default.vcl Error: Cannot create working directory '/tmp/varnishd_C_dwbl7mn/': Is a directory Error: Cannot create working directory (/tmp/varnishd_C_dwbl7mn/): No error: 0 (-? gives usage) root@jhost002:~ # This is on: FreeBSD jhost002.mydomain.nl 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #35 stable/13-c256281-gc415d0df47fc: Mon Feb  1 17:04:49 CET 2021 root@srv-01.home.local:/usr/obj/usr/src/amd64.amd64/sys/KRNL amd64 I did not update the package or installed any other software besides the buildworld. regards, Johan From owner-freebsd-stable@freebsd.org Tue Feb 2 01:24:24 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7AD484F90D9; Tue, 2 Feb 2021 01:24:24 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DV6Y33mNrz3p5n; Tue, 2 Feb 2021 01:24:23 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id 1BF3810705; Tue, 2 Feb 2021 10:24:14 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1612229054; bh=4lvLwX+aKwobBtrfjn0j2FNiywyz2mD6jcWj41nR534=; h=Date:To:Cc:Subject:From:In-Reply-To:References; b=clHOp3rBWydlOEZ0gw6Sq3fjkZJT81uhoLSntGFMF3DDMSwWUpz2+xrtPvbnftVVH Io5r8aRcW61jL11vz5AYxFh4E4sx7pIMQUVpoxWr2dRMj7y6ojqbjdgFiIOyD9WDvL iNXTpQrLm+EwmpwUWCRnNtUNAyU7JnxUP0LbPZaSZW7dFUi0NRLAY92mN7MTFJq27G oeYoKYCk+GICyreuojmTDAax1Dm1CrJkk5zjp4635mkTCf8pSAmCS6VfrV93OkH5rj LaY5wPcrdl+leAyRvM60J74zVcE6e0wyNtzxErwFk8Z2Y+euHi8aBshQZDdGFn+FWr e5kTZepaM562g== Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id 2A5F818E34; Tue, 2 Feb 2021 10:24:13 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.0 at eastasia.home.utahime.org Date: Tue, 02 Feb 2021 10:23:44 +0900 (JST) Message-Id: <20210202.102344.1749940138848745203.yasu@utahime.org> To: freebsd-current@freebsd.org Cc: freebsd-stable@freebsd.org Subject: Re: Setting for displaying utf8 characters on all vt consoles results in panic on 14-CURRENT and 13.0-ALPHA3 From: Yasuhiro Kimura In-Reply-To: <74FB4A6C-E2DC-4125-879E-2AE66E83C1E7@me.com> References: <20210201.114135.1319455396807129550.yasu@utahime.org> <20210201.123512.2031600289272155094.yasu@utahime.org> <74FB4A6C-E2DC-4125-879E-2AE66E83C1E7@me.com> X-Mailer: Mew version 6.8 on Emacs 27.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DV6Y33mNrz3p5n X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=clHOp3rB; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [-0.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 01:24:24 -0000 From: Toomas Soome via freebsd-current Subject: Re: Setting for displaying utf8 characters on all vt consoles results in panic on 14-CURRENT and 13.0-ALPHA3 Date: Tue, 2 Feb 2021 00:35:49 +0200 > Should be fixed on current now. Confirmed. Would you please MFC to stable/13? Best Regards. --- Yasuhiro Kimura From owner-freebsd-stable@freebsd.org Tue Feb 2 09:42:07 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DB78F52B57A for ; Tue, 2 Feb 2021 09:42:07 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DVKbL6jKHz4klk for ; Tue, 2 Feb 2021 09:42:06 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: by mail-ej1-x636.google.com with SMTP id p20so9559517ejb.6 for ; Tue, 02 Feb 2021 01:42:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:references:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=hOtQr7ElxXWTjOb9zBVXXotxMfGNJiypicPH4wi0nZ4=; b=rpvJ5aJVzn5eoay6CQjcqiEDTfFt4XSQBb3w/d1cnJxQsF0RMN8CUcq15gwN52jxbY EAyrFyMwaN+xxWqDTJ92MgjPDed4PKJxPtdwgGNf7drO82idwGpc9da12wRhr6MwYZ0a 4GpSSip6epnBc7fopXZggxKKkV5kGPiQ/KCZ9bBKoFWob63b/t3UDH67F0b4IBWWBr6H VLVwNIq0hH9tQL51xEaiVoAjvQB1R83APPln22c0YhTLnDBFFA/ccrxevCZ0aBUd0+aq cJKfeTvBILo/F93gZzJua9wcy/ue8ENzbH/pla6g1JqZ3vXPsZdl0mZSuE+9awHbClQf s2mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:references:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=hOtQr7ElxXWTjOb9zBVXXotxMfGNJiypicPH4wi0nZ4=; b=cUXuBP1Bok88l0Kod7Vck6b61zW9ronlxgvSxE5MKEPlQZ5Rk1+getp3oYr4HTz2jO zbwul4aLbPw3tL5Mdh1ghd4LGoG5cB8WD+uJT7EB2WlEtOzIX4HiC2almE4Kba3A0gFA ox4OgE9PiuKIqL6ZG0AGlndyZkAXLVimHU4A0fg2p02rJJGTQa5O6FmCZsDmdBbF8AVL qCCB3X7cNH+8pfWGhLX+FZbKojMSexNjXXlnk7v3il0UDHonfjqS7pQgz17B5FNjlTfo pFHJjU9WNxhzW8dS+Wd3YyXrrU0h5Lswb4iAYJw2pzsYLW7zI0dGjZV47sca84gUrJDA XmrA== X-Gm-Message-State: AOAM531ECUd0bFA6JI0pU3MUyRrZjjnXF6ZGvsoHk2x+VwLdzRYpDBAA Zz31vBmBBcLYzYk48Him7AU2onTlwRE= X-Google-Smtp-Source: ABdhPJwIZbFEpc1AELOndBlVUNg0W6kpkXoAGM00fN8HfEWHDAddDAdjhdRskPFJlGgJlws0yQ5YDw== X-Received: by 2002:a17:906:ce49:: with SMTP id se9mr21834488ejb.341.1612258924591; Tue, 02 Feb 2021 01:42:04 -0800 (PST) Received: from MacBook-Pro-van-Johan.local (85-147-130-226.cable.dynamic.v4.ziggo.nl. [85.147.130.226]) by smtp.gmail.com with ESMTPSA id q26sm9173794ejr.97.2021.02.02.01.42.03 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 02 Feb 2021 01:42:03 -0800 (PST) Subject: Re: latest stable13.0-ALPHA3 can not start varnish anymore. From: Johan Hendriks To: freebsd-stable@freebsd.org References: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> Message-ID: Date: Tue, 2 Feb 2021 10:41:59 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4DVKbL6jKHz4klk X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rpvJ5aJV; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of johhendriks@gmail.com designates 2a00:1450:4864:20::636 as permitted sender) smtp.mailfrom=johhendriks@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::636:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::636:from:127.0.2.255]; NEURAL_SPAM_SHORT(1.00)[0.996]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::636:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 09:42:07 -0000 On 01/02/2021 22:48, Johan Hendriks wrote: > I just updated my FreeBSD 13.0-APLPHA3 to the latest revision and now > i can not start varnish anymore. > This is on two machines. > > if i start varnish it errors out as the startup script does a config > file check. > If i try to do a test of the config file i get the following error. > > root@jhost002:~ # varnishd -C -f /usr/local/etc/varnish/default.vcl > Error: Cannot create working directory '/tmp/varnishd_C_dwbl7mn/': Is > a directory > Error: Cannot create working directory (/tmp/varnishd_C_dwbl7mn/): No > error: 0 > (-? gives usage) > root@jhost002:~ # > > This is on: > FreeBSD jhost002.mydomain.nl 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #35 > stable/13-c256281-gc415d0df47fc: Mon Feb  1 17:04:49 CET 2021 > root@srv-01.home.local:/usr/obj/usr/src/amd64.amd64/sys/KRNL amd64 > > I did not update the package or installed any other software besides > the buildworld. > > regards, > Johan > > Without the check, varnish fails to start all together. root@jhost002:/usr/local/etc/varnish # /usr/local/etc/rc.d/varnishd start Performing sanity check on varnishd configuration: Starting varnishd. Warnings: VCL compiled. Debug: Version: varnish-6.5.1 revision NOGIT Debug: Platform: FreeBSD,13.0-ALPHA3,amd64,-junix,-smalloc,-sdefault,-hcritbit Debug: Child (78795) Started eval: 000: not found /usr/local/etc/rc.d/varnishd: WARNING: failed to start varnishd I have tested this on a new machine updated to the same revision, same result. From owner-freebsd-stable@freebsd.org Tue Feb 2 13:05:29 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6E17E531B37 for ; Tue, 2 Feb 2021 13:05:29 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DVQ603S6Xz3GCY for ; Tue, 2 Feb 2021 13:05:28 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: by mail-ej1-x630.google.com with SMTP id i8so13379658ejc.7 for ; Tue, 02 Feb 2021 05:05:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:references:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=SptJNbWo+mDOh2iLFDsPEOlaA+a9tEXduApF62XyTd8=; b=O4TH0lJ8ucbBfArDQzmEnlq2Rm5bGHks6pDTwW2xTmkKqiXUH1sJDdEnf7bI/Cwnsu cje7zDzbvlXwqkYMYij+uKiQAEH0zVMTn0whZCU+v56phE48yTwURsrGbm9fPDfTC63U p4p6Ip6vdi22dNxfXJ4YhseMfrdtxVgiZpLZd0k+1RROSGsoZY4KR4nAloryywbtsOpY Ql3rO8FiUjJ7UbVSWdh61Mch6XMlzybdL+RTvJIVR9axWxjLwv6fCkYKTZNTWnK5QKP1 JBrUF2i+gftiYF3AkMUjAp9Z6kIY2svSg14Pc658LG6Y9nNxavodKfcuY5MmT2ydjbqW JMgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:references:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=SptJNbWo+mDOh2iLFDsPEOlaA+a9tEXduApF62XyTd8=; b=Hcr8/CEHjPImeeiO0Sp0QGvwACiPcLxTmE8OCDJszIC4b0d924tjKMioOYvbIZcrhq pbDHEDOtkIXQkxOguKAKfst8U+bcXDcZpene++6GfuBLLNSIGi/Q8C0+ESwEa+yLKVM1 YI2t9e9EkbwO5v/fN6UT9jG+MiVeuHCE2unHSJpzAzz//TrZJoD0A9mMUvoWFfzvsQzy m9k5wtNuj1dlTOw3+9ZOXgoLdpjQdJDq9jaNhyZWiDE3+IwHMNSiSRndFguCGfql3+Xf Gr91ghT1EwJsshOkaUrxavecY9yrZ4iwcn5L/r+osNcVukweiw/YbCjYaWm67sFI6DPC EiVw== X-Gm-Message-State: AOAM533/nfOmAoLL1q7CvEi8+639Bo/WH9e0uLwj4GPAHmASEHC8vm8L Bm7uNiWZUDPtzoK+3Z5ogGFhIQEGnv8= X-Google-Smtp-Source: ABdhPJxDpwi9XSDRtqVpH0frm+1yqUxRkA0grsBbQJmH6RNgNVkyQrPvrUTvwWGFm9sOg4DUsKiGrQ== X-Received: by 2002:a17:906:35d9:: with SMTP id p25mr10253355ejb.398.1612271126460; Tue, 02 Feb 2021 05:05:26 -0800 (PST) Received: from MacBook-Pro-van-Johan.local (85-147-130-226.cable.dynamic.v4.ziggo.nl. [85.147.130.226]) by smtp.gmail.com with ESMTPSA id y1sm4323642edq.26.2021.02.02.05.05.25 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 02 Feb 2021 05:05:25 -0800 (PST) Subject: Re: latest stable13.0-ALPHA3 can not start varnish anymore. From: Johan Hendriks To: freebsd-stable@freebsd.org References: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> Message-ID: <9821a854-9c86-060b-607b-01a2fbbc076e@gmail.com> Date: Tue, 2 Feb 2021 14:05:21 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4DVQ603S6Xz3GCY X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=O4TH0lJ8; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of johhendriks@gmail.com designates 2a00:1450:4864:20::630 as permitted sender) smtp.mailfrom=johhendriks@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::630:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::630:from:127.0.2.255]; NEURAL_SPAM_SHORT(1.00)[1.000]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::630:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 13:05:29 -0000 On 01/02/2021 22:48, Johan Hendriks wrote: > I just updated my FreeBSD 13.0-APLPHA3 to the latest revision and now > i can not start varnish anymore. > This is on two machines. > > if i start varnish it errors out as the startup script does a config > file check. > If i try to do a test of the config file i get the following error. > > root@jhost002:~ # varnishd -C -f /usr/local/etc/varnish/default.vcl > Error: Cannot create working directory '/tmp/varnishd_C_dwbl7mn/': Is > a directory > Error: Cannot create working directory (/tmp/varnishd_C_dwbl7mn/): No > error: 0 > (-? gives usage) > root@jhost002:~ # > > This is on: > FreeBSD jhost002.mydomain.nl 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #35 > stable/13-c256281-gc415d0df47fc: Mon Feb  1 17:04:49 CET 2021 > root@srv-01.home.local:/usr/obj/usr/src/amd64.amd64/sys/KRNL amd64 > > I did not update the package or installed any other software besides > the buildworld. > > regards, > Johan > > I have tried some bisecting as far as my understanding of git goes. I do not know which one is the latest, but on these revisions varnish works. FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #8 c256261-g9375a93b6c22: Tue Feb  2 13:33:05 CET 2021 root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64 uname -a FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #7 c256260-g247f652e622d: Tue Feb  2 13:07:37 CET 2021 root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64 From owner-freebsd-stable@freebsd.org Tue Feb 2 16:42:10 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 123E65367E7 for ; Tue, 2 Feb 2021 16:42:10 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DVVw16ncNz3l5t; Tue, 2 Feb 2021 16:42:09 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id C6D072AE2B; Tue, 2 Feb 2021 16:42:09 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id 6A98B35813; Tue, 2 Feb 2021 17:42:08 +0100 (CET) From: "Kristof Provost" To: "Johan Hendriks" Cc: freebsd-stable@freebsd.org, mjg@FreeBSD.org Subject: Re: latest stable13.0-ALPHA3 can not start varnish anymore. Date: Tue, 02 Feb 2021 17:42:07 +0100 X-Mailer: MailMate (1.13.2r5673) Message-ID: <44315237-2B01-4C1B-A992-E235A5A911B0@FreeBSD.org> In-Reply-To: <9821a854-9c86-060b-607b-01a2fbbc076e@gmail.com> References: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> <9821a854-9c86-060b-607b-01a2fbbc076e@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 16:42:10 -0000 On 2 Feb 2021, at 14:05, Johan Hendriks wrote: > On 01/02/2021 22:48, Johan Hendriks wrote: >> I just updated my FreeBSD 13.0-APLPHA3 to the latest revision and now >> i can not start varnish anymore. >> This is on two machines. >> >> if i start varnish it errors out as the startup script does a config >> file check. >> If i try to do a test of the config file i get the following error. >> >> root@jhost002:~ # varnishd -C -f /usr/local/etc/varnish/default.vcl >> Error: Cannot create working directory '/tmp/varnishd_C_dwbl7mn/': Is >> a directory >> Error: Cannot create working directory (/tmp/varnishd_C_dwbl7mn/): No >> error: 0 >> (-? gives usage) >> root@jhost002:~ # >> >> This is on: >> FreeBSD jhost002.mydomain.nl 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #35 >> stable/13-c256281-gc415d0df47fc: Mon Feb  1 17:04:49 CET 2021 >> root@srv-01.home.local:/usr/obj/usr/src/amd64.amd64/sys/KRNL amd64 >> >> I did not update the package or installed any other software besides >> the buildworld. >> >> regards, >> Johan >> >> > I have tried some bisecting as far as my understanding of git goes. I > do not know which one is the latest, but on these revisions varnish > works. > > FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #8 > c256261-g9375a93b6c22: Tue Feb  2 13:33:05 CET 2021 > root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64 > > uname -a > FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #7 > c256260-g247f652e622d: Tue Feb  2 13:07:37 CET 2021 > root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64 > Can you try setting `sysctl vfs.cache_fast_lookup=0` ? (As suggested by Mateusz elsewhere.) Best regards, Kristof From owner-freebsd-stable@freebsd.org Tue Feb 2 16:44:39 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2E600536AE6 for ; Tue, 2 Feb 2021 16:44:39 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DVVyv0stgz3lTw; Tue, 2 Feb 2021 16:44:39 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 0C1DD2B19B; Tue, 2 Feb 2021 16:44:39 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qk1-f172.google.com with SMTP id u20so20405855qku.7; Tue, 02 Feb 2021 08:44:39 -0800 (PST) X-Gm-Message-State: AOAM530m79nRXyHRJ1rEBSioKq1Mp15XzhiUJtktmZCQdNPSuKe/hAtY bw/glHNxkQcsP6d3YGlWKi8sd2WgXPGMtX+73NU= X-Google-Smtp-Source: ABdhPJz+nVRYRA636hNckrLinB48fjJhr9J2tMpE8WkWcQWSCs7cp2nbY9cZECTtbu5sLQpqkwUZ/dCW0/FuejH3hKo= X-Received: by 2002:a37:745:: with SMTP id 66mr23310931qkh.120.1612284278526; Tue, 02 Feb 2021 08:44:38 -0800 (PST) MIME-Version: 1.0 References: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> <9821a854-9c86-060b-607b-01a2fbbc076e@gmail.com> <44315237-2B01-4C1B-A992-E235A5A911B0@FreeBSD.org> In-Reply-To: <44315237-2B01-4C1B-A992-E235A5A911B0@FreeBSD.org> From: Kyle Evans Date: Tue, 2 Feb 2021 10:44:25 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: latest stable13.0-ALPHA3 can not start varnish anymore. To: Kristof Provost Cc: Johan Hendriks , FreeBSD-STABLE Mailing List , Mateusz Guzik Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 16:44:39 -0000 On Tue, Feb 2, 2021 at 10:42 AM Kristof Provost wrote: > > On 2 Feb 2021, at 14:05, Johan Hendriks wrote: > > On 01/02/2021 22:48, Johan Hendriks wrote: > >> I just updated my FreeBSD 13.0-APLPHA3 to the latest revision and now > >> i can not start varnish anymore. > >> This is on two machines. > >> > >> if i start varnish it errors out as the startup script does a config > >> file check. > >> If i try to do a test of the config file i get the following error. > >> > >> root@jhost002:~ # varnishd -C -f /usr/local/etc/varnish/default.vcl > >> Error: Cannot create working directory '/tmp/varnishd_C_dwbl7mn/': Is > >> a directory > >> Error: Cannot create working directory (/tmp/varnishd_C_dwbl7mn/): No > >> error: 0 > >> (-? gives usage) > >> root@jhost002:~ # > >> > >> This is on: > >> FreeBSD jhost002.mydomain.nl 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #35 > >> stable/13-c256281-gc415d0df47fc: Mon Feb 1 17:04:49 CET 2021 > >> root@srv-01.home.local:/usr/obj/usr/src/amd64.amd64/sys/KRNL amd64 > >> > >> I did not update the package or installed any other software besides > >> the buildworld. > >> > >> regards, > >> Johan > >> > >> > > I have tried some bisecting as far as my understanding of git goes. I > > do not know which one is the latest, but on these revisions varnish > > works. > > > > FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #8 > > c256261-g9375a93b6c22: Tue Feb 2 13:33:05 CET 2021 > > root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 > > > > uname -a > > FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #7 > > c256260-g247f652e622d: Tue Feb 2 13:07:37 CET 2021 > > root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 > > > Can you try setting `sysctl vfs.cache_fast_lookup=0` ? > > (As suggested by Mateusz elsewhere.) > We confirmed that reverting 006ec2ed fixes it, I'm still trying to get an idea of what's happening. My initial analysis suggests that fplookup hits the trailing slash and aborts, leaving us at the degenerate state back up in the slow-path. I have a small reproducer that I'm working with: https://people.freebsd.org/~kevans/namereg.c From owner-freebsd-stable@freebsd.org Mon Feb 1 22:35:55 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A77F34E98EE for ; Mon, 1 Feb 2021 22:35:55 +0000 (UTC) (envelope-from tsoome@me.com) Received: from pv50p00im-ztdg10022001.me.com (pv50p00im-ztdg10022001.me.com [17.58.6.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DV2pf6rCdz3LmZ for ; Mon, 1 Feb 2021 22:35:54 +0000 (UTC) (envelope-from tsoome@me.com) Received: from nazgul.lan (148-52-235-80.sta.estpak.ee [80.235.52.148]) by pv50p00im-ztdg10022001.me.com (Postfix) with ESMTPSA id 282B2A026D; Mon, 1 Feb 2021 22:35:51 +0000 (UTC) From: Toomas Soome Message-Id: <74FB4A6C-E2DC-4125-879E-2AE66E83C1E7@me.com> Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.40.0.2.32\)) Subject: Re: Setting for displaying utf8 characters on all vt consoles results in panic on 14-CURRENT and 13.0-ALPHA3 Date: Tue, 2 Feb 2021 00:35:49 +0200 In-Reply-To: <20210201.123512.2031600289272155094.yasu@utahime.org> Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org To: Yasuhiro Kimura References: <20210201.114135.1319455396807129550.yasu@utahime.org> <20210201.123512.2031600289272155094.yasu@utahime.org> X-Mailer: Apple Mail (2.3654.40.0.2.32) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.737 definitions=2021-02-01_11:2021-01-29, 2021-02-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-2006250000 definitions=main-2102010123 X-Rspamd-Queue-Id: 4DV2pf6rCdz3LmZ X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[me.com]; MV_CASE(0.50)[]; RWL_MAILSPIKE_GOOD(0.00)[17.58.6.58:from]; R_SPF_ALLOW(-0.20)[+ip4:17.58.0.0/16]; DKIM_TRACE(0.00)[me.com:+]; DMARC_POLICY_ALLOW(-0.50)[me.com,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[me.com]; ASN(0.00)[asn:714, ipnet:17.58.0.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[17.58.6.58:from]; RECEIVED_SPAMHAUS_PBL(0.00)[80.235.52.148:received]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[me.com:s=1a1hai]; FREEFALL_USER(0.00)[tsoome]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DWL_DNSWL_NONE(0.00)[me.com:dkim]; SPAMHAUS_ZRD(0.00)[17.58.6.58:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[17.58.6.58:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-Mailman-Approved-At: Tue, 02 Feb 2021 17:09:26 +0000 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2021 22:35:55 -0000 > On 1. Feb 2021, at 05:35, Yasuhiro Kimura wrote: >=20 > From: Yasuhiro Kimura > > Subject: Setting for displaying utf8 characters on all vt consoles = results in panic on 14-CURRENT and 13.0-ALPHA3 > Date: Mon, 01 Feb 2021 11:41:35 +0900 (JST) >=20 >> To display utf8 characters on all vt console I did following = settings. >>=20 >> 1. Download GNU Unifont BDF file >> = (http://unifoundry.com/pub/unifont/unifont-13.0.05/font-builds/unifont-13.= 0.05.bdf.gz) >> 2. gunzip unifont-13.0.05.bdf.gz >> 3. vtfontcvt unifont-13.0.05.bdf unifont.fnt >> 4. cp unifont.fnt /usr/share/vt/fonts >> 5. Add 'allscreens_flags=3D"-f 8x16 unifont.fnt"' to /etc/rc.conf >> 6. Add 'hw.vga.textmode=3D0' to /boot/loader.conf.local >> 7. shutdown -r now >>=20 >> On 12.2-RELEASE and 11.4-RELEASE it works as is expected. But on >> 14-CURRENT(man) and 13.0-ALPHA3(stable/13) it result in kernel panic. >>=20 >> Screen shot of 14-CURRENT. >> https://www.utahime.org/FreeBSD/panic.20210201.14-CURRENT.png >>=20 >> 14-CURRENT(main): >> yasu@rolling-vm-freebsd1[1006]% uname -a >> FreeBSD rolling-vm-freebsd1.home.utahime.org 14.0-CURRENT FreeBSD = 14.0-CURRENT #0 main-n244517-f17fc5439f5: Mon Feb 1 10:55:51 JST 2021 = = rootz@rolling-vm-freebsd1.home.utahime.org:/usr0/freebsd/src/obj/usr0/free= bsd/src/git/amd64.amd64/sys/GENERIC amd64 >>=20 >> 13.0-ALPHA3(stable/13): >> yasu@rolling-vm-freebsd5[1005]% uname -a >> FreeBSD rolling-vm-freebsd5.home.utahime.org 13.0-ALPHA3 FreeBSD = 13.0-ALPHA3 #0 stable/13-c256214-g40cb0344eb2: Mon Feb 1 11:30:28 JST = 2021 = rootz@rolling-vm-freebsd5.home.utahime.org:/usr0/freebsd/src/obj/usr0/free= bsd/src/git/amd64.amd64/sys/GENERIC amd64 >=20 > I submitted this problem to Bugzilla. >=20 > Bug 253147 - Setting for displaying utf8 characters on all vt consoles > results in panic on 14-CURRENT and 13.0-ALPHA3 > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D253147 = >=20 > --- > Yasuhiro Kimura Should be fixed on current now. thanks, toomas From owner-freebsd-stable@freebsd.org Tue Feb 2 19:36:16 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B81A053B80F; Tue, 2 Feb 2021 19:36:16 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DVZmv3Rnzz4TKQ; Tue, 2 Feb 2021 19:36:15 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1612294573; bh=ZRV5hC0b3oReDvV/xTTaRELkVgqYGZIfj+u90cYHzCc=; h=X-UI-Sender-Class:From:To:Subject:Date; b=Eww+GqfeItP1Te2Br9Nk9YBl4FYol6McZWsYFA5/ZdfQh7KOrqp4MBhot6NFbPH/+ E83u66ZZmrg2FRnIVyXQ+BR0vS0mkf5cfrzwOv/ZZ3pNowWCjLFKW0obTHrhUnniaW xs1RK0NuMrdDcGqJk8m9iiALqSsZunLvofPd/mQc= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [89.244.179.162] ([89.244.179.162]) by web-mail.gmx.net (3c-app-gmx-bap27.server.lan [172.19.172.97]) (via HTTP); Tue, 2 Feb 2021 20:36:13 +0100 MIME-Version: 1.0 Message-ID: From: mj-mailinglist@gmx.de To: freebsd-stable@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: pkgbase: Missing /etc/master.passwd in FreeBSD-runtime 12-stable pkg Content-Type: text/plain; charset=UTF-8 Date: Tue, 2 Feb 2021 20:36:13 +0100 Importance: normal Sensitivity: Normal X-Priority: 3 X-Provags-ID: V03:K1:23wZxc1zCDCKwcVAz/+S/XMAPSwV2wcLAIfQ2/nmIqNnm0D3/9eydpPH7eBq5T/Y85gql Qa1N9zX1dBLdWudpsfHc7F6r7SpjCE1T0rjsY/JCNX2vB46VuY0SB69IVVWJRZ2ZCTGAw4fHoueK 18EBdMwvnboIN1eVZ318hFQb8NlTpI5EQkb1xCMStqQjl4umT/rXHx/YLGxpq5Is9dF9V/xzuxt0 dbTQv3yIj4DEuP+178pjhmDyl1xAX5T2qtpvu6xN7yNRdH8OX6Wul1z7AH0zZWcWeOBJpfE46QHJ OA= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:+pEyjMx24U0=:q7J6x07fI8vpgqzctz+NFW H1u0f1ThawaAHAAb9t4d+a1bwo6nX+8zvKB3AmaVyDMJdItD/MKOxvz20idqQmSCtHT+mgY73 8JwynHiJRcxDO5XSxq8K9Yeq3/6UP/ELCm0L7sMdRv9qkLZSOxjPwMlugXJQluscMMW2elZ1J vkRtBACw+gRwka9vjPtK/31pTVkJBnJOxScKrzxqDJgSMbiiGOaaWrzC5bke3nobrLpX7+5HB a1qQQ2pgmGMoGKKUwDPb4XyEogX/7MwICvo4MPmMaGvNNPXaPKC4sEC/fQ0daDoF+z5G00Rbo LmsWMcMRKsTNBW5nWUL7d4z9cMgUSUhMIkQlJ7Em4xqkWGrIpaMvBf3lmjdIshvx9eX8wtTzx Wx1Qgo8BVCddtcMVoPIeSQ9qTSNBzFJ7yMnfGqa+uIhAZeick7jUCwaAQs4se6Yc2UDkckr7Y R09fU42FmxbbfLiEMYmoge/w1CXHQDpo0+PFXLzmoFVgKukzllQ2GrJzfYcWlusmE3QCX8aMV eSYADqpackb2J7oXYn2cRT0eOkE328dz9EmhS6Ss9VmpftdlGN3qnudBgBKnuXChugTL9VnXm jVThDTyTmBJ6I= Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4DVZmv3Rnzz4TKQ X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=Eww+Gqfe; dmarc=pass (policy=none) header.from=gmx.de; spf=pass (mx1.freebsd.org: domain of mj-mailinglist@gmx.de designates 212.227.17.21 as permitted sender) smtp.mailfrom=mj-mailinglist@gmx.de X-Spamd-Result: default: False [-2.60 / 15.00]; FREEMAIL_FROM(0.00)[gmx.de]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; RCPT_COUNT_TWO(0.00)[2]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmx.de,none]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; RBL_DBL_DONT_QUERY_IPS(0.00)[212.227.17.21:from]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.17.21:from]; FAKE_REPLY(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RECEIVED_SPAMHAUS_PBL(0.00)[89.244.179.162:received]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[212.227.17.21:from:127.0.2.255]; FROM_NO_DN(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[212.227.17.21:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable,freebsd-pkgbase] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 19:36:16 -0000 >> >> On 1/31/21 12:04 PM, mj-mailinglist@gmx.de wrote: >>> I noticed that my jails, build from a recent pkg base don't start: >>> ... >>> Starting jails: cannot start jail "j6": >>> 7 >>> jail: j6: getpwnam root: No such file or directory >>> jail: j6: /bin/sh /etc/rc: failed >>> . >>> >> >> [snip] >> >> Just a thought, did you run >> >> make distribution DESTDIR=3D > > This shouldn't be necessary with pkgbase, but I haven't looked at > pkgbase in 12 on a long time -- it sounds like we managed to regress > it somewhere. The last step in my weekly system rebuild is "make packages" to create a p= kgbase repository from the newly build system. The last build, i could use to cre= ate jails is r368786M, with "stable/12-c58-g53bc32215" the jails are broken. I didn't check since which commit the error occurs. Reading the commit log= s doesn't show anything obvious to me. Should pkgbase be operational in stable/12? If the consensus is no, i just= skip it and concentrate on 13. Will pkgbase be a supported (and working) featur= e on 13? =2D- Martin From owner-freebsd-stable@freebsd.org Tue Feb 2 20:02:22 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4EDC053BCDF for ; Tue, 2 Feb 2021 20:02:22 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DVbM21Mgrz4Vqh; Tue, 2 Feb 2021 20:02:22 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: by mail-ed1-x52a.google.com with SMTP id c6so24349100ede.0; Tue, 02 Feb 2021 12:02:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=srtbbNCR2qwW2EXFCM+joLPBvawOxH9eQtOWBDia0Vw=; b=XQDDDvyPhtMD/UwATCvez+eqefEvjsWtygjGOUUP52LMpNwWxOGVrcj98pbS7cgq/H Fi0+QZ8aXX/CfqPUL996a+elNhk7i+47m3+G4nfJ2gBAviWeO8dSaCF0TsmnsggvtXTa wVzS18Trk+6pmwycroLDUV9yjdZ/4ToOvYCdTxlrdEhX7MbYvnfUrovTqy7/EpUOMLni 04jBPaZtrXwNiZh76/S2f2R1tOgcY7gv4mzTa5vEQ0CwP+eco18v7LIHSO4o8NWf91P/ 8Q7gksp97BV+BICfltCP3ZEc80TuIgZ9tv7weixtG60kYSKxkWzFm9cw0r3ZvRdXn8Av qSgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=srtbbNCR2qwW2EXFCM+joLPBvawOxH9eQtOWBDia0Vw=; b=METF8fYuwgOno6H5BDiaMRxcp3YpLnMcISEYE80Wmzx93AIL2IOEn9dlqALPvQwZEa n3G1Z0FCLUpsJWLq/mlrfjObRmcUzQsR2XtQEqP3dZePoxyGUuyDCl7ycVHmg38joum5 q9Gpowxks5XsTeOTcjOe6dhHjspvg2kSNZo3Jorc3xQaFDxw5oXHug5vgNbkTQj3R4uf HpSR+8PSaqGlh2wqBLvDnpirPCWXUOiuM9SkiEX4ehkS7cR6OfpS3jwBA2mElC3Thjxa 13cI5Zc6n/HaBA/naTYuMmWw2TIFy2EXR23XVbHAYZzZBQK6JlCmKjQpZNB8j6NR/2jb 1jIQ== X-Gm-Message-State: AOAM533g6b0F0G3OknO/AZnxcI5YeF9ZKWCVpP3b7Gmj1g66nt2gFq/v gfgE/yU8qQtdYjSaAIDApeTPA9FUpc4= X-Google-Smtp-Source: ABdhPJwNM85yZjiDjvMVyEQ+Ir7KxPNCiSz4zncY8hGq1lUWAM2WLbW69OrddYLbIMzsqc3xb7fO6Q== X-Received: by 2002:a05:6402:215:: with SMTP id t21mr525624edv.363.1612296140526; Tue, 02 Feb 2021 12:02:20 -0800 (PST) Received: from MacBook-Pro-van-Johan.local (85-147-130-226.cable.dynamic.v4.ziggo.nl. [85.147.130.226]) by smtp.gmail.com with ESMTPSA id a22sm10466722edv.67.2021.02.02.12.02.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 02 Feb 2021 12:02:19 -0800 (PST) Subject: Re: latest stable13.0-ALPHA3 can not start varnish anymore. To: Mateusz Guzik , Kyle Evans Cc: Kristof Provost , FreeBSD-STABLE Mailing List References: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> <9821a854-9c86-060b-607b-01a2fbbc076e@gmail.com> <44315237-2B01-4C1B-A992-E235A5A911B0@FreeBSD.org> From: Johan Hendriks Message-ID: <74f1dcc9-51e0-8272-4fbd-c3bdd8cf9e89@gmail.com> Date: Tue, 2 Feb 2021 21:02:13 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4DVbM21Mgrz4Vqh X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_FROM(0.00)[]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 20:02:22 -0000 On 02/02/2021 19:20, Mateusz Guzik wrote: > I fixed the problem and given the impending builds merged it > immediately. Can be seen here: > https://cgit.freebsd.org/src/commit/?id=4e29933d0936fd053f7591ee118dc1fc42617514 > > Should you run into any other problems, they can be temporarily worked > around with: > sysctl vfs.cache_fast_lookup=0 > > For problem reports in the area please collect: > dtrace -n 'vfs:fplookup:lookup:done { @[arg1, arg2, > stringof(args[0].ni_cnd.cn_pnbuf)] = count(); }' -c "failing command" > > On 2/2/21, Kyle Evans wrote: >> On Tue, Feb 2, 2021 at 10:42 AM Kristof Provost wrote: >>> On 2 Feb 2021, at 14:05, Johan Hendriks wrote: >>>> On 01/02/2021 22:48, Johan Hendriks wrote: >>>>> I just updated my FreeBSD 13.0-APLPHA3 to the latest revision and now >>>>> i can not start varnish anymore. >>>>> This is on two machines. >>>>> >>>>> if i start varnish it errors out as the startup script does a config >>>>> file check. >>>>> If i try to do a test of the config file i get the following error. >>>>> >>>>> root@jhost002:~ # varnishd -C -f /usr/local/etc/varnish/default.vcl >>>>> Error: Cannot create working directory '/tmp/varnishd_C_dwbl7mn/': Is >>>>> a directory >>>>> Error: Cannot create working directory (/tmp/varnishd_C_dwbl7mn/): No >>>>> error: 0 >>>>> (-? gives usage) >>>>> root@jhost002:~ # >>>>> >>>>> This is on: >>>>> FreeBSD jhost002.mydomain.nl 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #35 >>>>> stable/13-c256281-gc415d0df47fc: Mon Feb 1 17:04:49 CET 2021 >>>>> root@srv-01.home.local:/usr/obj/usr/src/amd64.amd64/sys/KRNL amd64 >>>>> >>>>> I did not update the package or installed any other software besides >>>>> the buildworld. >>>>> >>>>> regards, >>>>> Johan >>>>> >>>>> >>>> I have tried some bisecting as far as my understanding of git goes. I >>>> do not know which one is the latest, but on these revisions varnish >>>> works. >>>> >>>> FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #8 >>>> c256261-g9375a93b6c22: Tue Feb 2 13:33:05 CET 2021 >>>> root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 >>>> >>>> uname -a >>>> FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #7 >>>> c256260-g247f652e622d: Tue Feb 2 13:07:37 CET 2021 >>>> root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 >>>> >>> Can you try setting `sysctl vfs.cache_fast_lookup=0` ? >>> >>> (As suggested by Mateusz elsewhere.) >>> >> We confirmed that reverting 006ec2ed fixes it, I'm still trying to get >> an idea of what's happening. My initial analysis suggests that >> fplookup hits the trailing slash and aborts, leaving us at the >> degenerate state back up in the slow-path. >> >> I have a small reproducer that I'm working with: >> https://people.freebsd.org/~kevans/namereg.c >> Just did a full rebuild for stable 13 and i can run varnishd -C -f /usr/local/etc/varnish/default.vcl again. FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #14 stable/13-n244422-4e29933d0936: Tue Feb  2 20:49:09 CET 2021 root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64 Thank you all for your time and work. From owner-freebsd-stable@freebsd.org Wed Feb 3 05:02:31 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AF26E52A242 for ; Wed, 3 Feb 2021 05:02:31 +0000 (UTC) (envelope-from timp87@gmail.com) Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DVqLG3wXYz3Krf for ; Wed, 3 Feb 2021 05:02:30 +0000 (UTC) (envelope-from timp87@gmail.com) Received: by mail-ej1-x636.google.com with SMTP id p20so14244877ejb.6 for ; Tue, 02 Feb 2021 21:02:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=YNqYQ3wq+puUfvPdmz0sHRQ1VfiBycKtSJN+6O/7/uU=; b=uZiSraIJC/u1hjwH3l8V+aKVZTwOA5wVD3de/rttAO71lbMmNLF6lRIJIOmVUAq6+S 5r1A3K46wYm2LDTKY7FbtnmAKhTTdFlhxrB/Pr0zcnUtQWyHXTLvB/DwwxPtSBcn0Irf rfh8q5aVf9NOAgx9NnlysKiMwro+DHs72HpnZwr3HXZ3UveY4+kXZjA/DrBm5q5OMxoz ebfJDIASrvULEceeG2qSz0MqKqd0kVCJOpw3SsSJBA2EHTO4PF5HTjQxl4AWSIXVlmcw nSN/ZCDHIim9hgqrcvTVNtzNrT9cO+GJUxfS75d/lAzbbpQm6VHU37mWU4KQgirGHSJ0 ONkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=YNqYQ3wq+puUfvPdmz0sHRQ1VfiBycKtSJN+6O/7/uU=; b=IGqQypM/jwhm+f4b/swJ46hW+AEMcfS6+2SUWsCOpIUk0YHIkgxKK4dDQON/Z8zef6 CChjMl+PP3rOROgZOByooqUE3bAOzlpuJHzIiX3yS6xdbOyF7s1TtnxRRBgMS6uLumrg 4xlUjeHMeWb45XVyAfraSIL2wzxxvWqET+oaU6UwcdzoHHWZy7rZId8JalWBPRAfBGJW DPZLk3zJ74JIf4VUh3sz0Jl5izqOeYNT6pFePXhXMnPZcBa+KVng/ssKQkuaU9D1YnZu gY/6CUZhhAbf4d8MDBH6GSOiTA7ysS1e9JbDYUHB1NCTW79bOom/m62z0WrTbkC5LNyE KuKw== X-Gm-Message-State: AOAM533AoWrlrD0ErUk8NFBOUUeBUO/z1ChEGwTQ2V5DVtmpFJUluQ9J 2M+4brMBL6JiMj/dSYEKhN34tYWrDR90HxrTEGBxsybccXA= X-Google-Smtp-Source: ABdhPJzIjjSrw3HfPXmppU+ZtDxGSgeh5eoGbvSD57aDIYOk5i+6+QrzP0K043+9tywDfp51V/0DCvaOFl100owGq60= X-Received: by 2002:a17:906:c9cc:: with SMTP id hk12mr1466363ejb.134.1612328546847; Tue, 02 Feb 2021 21:02:26 -0800 (PST) MIME-Version: 1.0 From: Pavel Timofeev Date: Tue, 2 Feb 2021 22:02:15 -0700 Message-ID: Subject: x1 gen3 long boot with usb errors To: freebsd-stable stable X-Rspamd-Queue-Id: 4DVqLG3wXYz3Krf X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uZiSraIJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of timp87@gmail.com designates 2a00:1450:4864:20::636 as permitted sender) smtp.mailfrom=timp87@gmail.com X-Spamd-Result: default: False [-2.07 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::636:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::636:from:127.0.2.255]; NEURAL_SPAM_SHORT(0.93)[0.931]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::636:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2021 05:02:31 -0000 Hi I have Lenovo X1 Carbon Gen 3 laptop and I'm suffering from long boot with lots of repeating > Root mount waiting for: usbus0 messages on FreeBSD 13.0-ALPHA3 stable/13-c256238-g3f185aacc58. Some googling told me it's a common issue with FreeBSD's xhci driver. I'm wondering what can be done to fix or workaround this. ---<>--- APIC: Using the MADT enumerator. Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-ALPHA3 #0 stable/13-c256238-g3f185aacc58: Sun Jan 31 23:03:32 MST 2021 pavel.timofeev@carbon:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 FreeBSD clang version 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe) PPIM 0: PA=0xc0000000, VA=0xffffffff82910000, size=0x7e9000, mode=0x1 pmap: large map 8 PML4 slots (4096 GB) VT(efifb): resolution 1920x1080 Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff827af000. Preloaded elf obj module "/boot/kernel/cryptodev.ko" at 0xffffffff827b8718. Preloaded elf obj module "/boot/kernel/coretemp.ko" at 0xffffffff827b8e88. Preloaded boot_entropy_cache "/boot/entropy" at 0xffffffff827b9478. Preloaded elf obj module "/boot/kernel/acpi_ibm.ko" at 0xffffffff827b94d0. Preloaded elf obj module "/boot/kernel/cuse.ko" at 0xffffffff827b9b40. Preloaded elf obj module "/boot/kernel/zfs.ko" at 0xffffffff827ba228. Preloaded hostuuid "/etc/hostid" at 0xffffffff827baa10. Calibrating TSC clock ... TSC clock: 2194973264 Hz CPU: Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz (2194.97-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306d4 Family=0x6 Model=0x3d Stepping=4 Features=0xbfebfbff Features2=0x7ffafbbf AMD Features=0x2c100800 AMD Features2=0x121 Structured Extended Features=0x21c27ab Structured Extended Features3=0x9c000600 XSAVE Features=0x1 VT-x: Basic Features=0xda0400 Pin-Based Controls=0x7f Primary Processor Controls=0xfff9fffe Secondary Processor Controls=0x53cff Exit Controls=0xda0400 Entry Controls=0xda0400 EPT Features=0x6334141 VPID Features=0xf01 TSC: P-state invariant, performance statistics Data TLB: 2 MByte or 4 MByte pages, 4-way set associative, 32 entries and a separate array with 1 GByte pages, 4-way set associative, 4 entries Data TLB: 4 KB pages, 4-way set associative, 64 entries Instruction TLB: 2M/4M pages, fully associative, 8 entries Instruction TLB: 4KByte pages, 8-way set associative, 64 entries 64-Byte prefetching Shared 2nd-Level TLB: 4 KByte /2 MByte pages, 6-way associative, 1536 entries. Also 1GBbyte pages, 4-way, 16 entries L2 cache: 256 kbytes, 8-way associative, 64 bytes/line real memory = 8589934592 (8192 MB) Physical memory chunk(s): 0x0000000000010000 - 0x0000000000057fff, 294912 bytes (72 pages) 0x0000000000059000 - 0x000000000009bfff, 274432 bytes (67 pages) 0x0000000000104000 - 0x00000000001fffff, 1032192 bytes (252 pages) 0x0000000002901000 - 0x00000000aa7a4fff, 2817146880 bytes (687780 pages) 0x00000000acdff000 - 0x00000000acdfffff, 4096 bytes (1 pages) 0x0000000100001000 - 0x000000023ed46fff, 5349072896 bytes (1305926 pages) 0x000000024de00000 - 0x000000024dfa7fff, 1736704 bytes (424 pages) avail memory = 8140484608 (7763 MB) intel stolen mem: base 0xae000000 size 32 MB x2APIC available but disabled by DMAR table MADT: Found CPU APIC ID 0 ACPI ID 1: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 1 ACPI ID 2: enabled SMP: Added CPU 1 (AP) MADT: Found CPU APIC ID 2 ACPI ID 3: enabled SMP: Added CPU 2 (AP) MADT: Found CPU APIC ID 3 ACPI ID 4: enabled SMP: Added CPU 3 (AP) MADT: Found CPU APIC ID 0 ACPI ID 5: disabled MADT: Found CPU APIC ID 0 ACPI ID 6: disabled MADT: Found CPU APIC ID 0 ACPI ID 7: disabled MADT: Found CPU APIC ID 0 ACPI ID 8: disabled Event timer "LAPIC" quality 600 LAPIC: ipi_wait() us multiplier 38 (r 5657252 tsc 2194973264) ACPI APIC Table: Package ID shift: 4 L3 cache ID shift: 4 L2 cache ID shift: 1 L1 cache ID shift: 1 Core ID shift: 1 INTR: Adding local APIC 2 as a target FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads Package HW ID = 0 Core HW ID = 0 CPU0 (BSP): APIC ID: 0 CPU1 (AP/HT): APIC ID: 1 Core HW ID = 1 CPU2 (AP): APIC ID: 2 CPU3 (AP/HT): APIC ID: 3 APIC: CPU 0 has ACPI ID 1 APIC: CPU 1 has ACPI ID 2 APIC: CPU 2 has ACPI ID 3 APIC: CPU 3 has ACPI ID 4 lapic0: CMCI unmasked Pentium Pro MTRR support enabled x86bios: IVT 0x000000-0x0004ff at 0xfffff80000000000 x86bios: SSEG 0x059000-0x059fff at 0xfffffe0010a25000 x86bios: ROM 0x0a0000-0x0fefff at 0xfffff800000a0000 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: read 4096 bytes from preloaded cache random: unblocking device. VIMAGE (virtualized network stack) enabled hostuuid: using 184c4101-53f3-11cb-917e-f03a6d13030b ULE: setup cpu 0 ULE: setup cpu 1 ULE: setup cpu 2 ULE: setup cpu 3 ACPI: RSDP 0x00000000ACDFE014 000024 (v02 LENOVO) ACPI: XSDT 0x00000000ACDFE1C0 0000EC (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: FACP 0x00000000ACDF9000 00010C (v05 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: DSDT 0x00000000ACDE3000 011432 (v02 LENOVO TP-N14 00001320 INTL 20120711) ACPI: FACS 0x00000000ACD68000 000040 ACPI: ASF! 0x00000000ACDFD000 0000A5 (v32 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: HPET 0x00000000ACDFC000 000038 (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: ECDT 0x00000000ACDFB000 000052 (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: APIC 0x00000000ACDF8000 000098 (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: MCFG 0x00000000ACDF7000 00003C (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: SSDT 0x00000000ACDF6000 000033 (v01 LENOVO TP-SSDT1 00000100 INTL 20120711) ACPI: SSDT 0x00000000ACDF5000 000486 (v01 LENOVO TP-SSDT2 00000200 INTL 20120711) ACPI: SSDT 0x00000000ACDE2000 000290 (v01 LENOVO PcieAhci 00001000 INTL 20120711) ACPI: SSDT 0x00000000ACDE1000 0009CB (v01 LENOVO SataAhci 00001000 INTL 20120711) ACPI: SSDT 0x00000000ACDE0000 000152 (v01 LENOVO Rmv_Batt 00001000 INTL 20120711) ACPI: SSDT 0x00000000ACDDF000 000691 (v01 LENOVO Cpu0Ist 00003000 INTL 20120711) ACPI: SSDT 0x00000000ACDDE000 000B74 (v02 LENOVO CpuSsdt 00003000 INTL 20120711) ACPI: SSDT 0x00000000ACDDD000 000369 (v02 LENOVO CtdpB 00001000 INTL 20120711) ACPI: SSDT 0x00000000ACDDB000 001475 (v01 LENOVO SaSsdt 00003000 INTL 20120711) ACPI: SSDT 0x00000000ACDDA000 000394 (v02 LENOVO CppcTabl 00001000 INTL 20120711) ACPI: PCCT 0x00000000ACDD9000 00006E (v05 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: SSDT 0x00000000ACDD8000 000AC4 (v02 LENOVO Cpc_Tabl 00001000 INTL 20120711) ACPI: UEFI 0x00000000ACDD7000 000042 (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: MSDM 0x00000000ACD41000 000055 (v03 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: BATB 0x00000000ACDD6000 00004A (v02 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: FPDT 0x00000000ACDD5000 000064 (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: UEFI 0x00000000ACDD4000 000332 (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: BGRT 0x00000000ACDD3000 000038 (v01 LENOVO TP-N14 00001320 PTEC 00000002) ACPI: DMAR 0x00000000ACDD2000 0000A8 (v01 LENOVO TP-N14 00001320 PTEC 00000002) MADT: Found IO APIC ID 2, Interrupt 0 at 0xfec00000 ioapic0: ver 0x20 maxredir 0x27 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 0, irq 2 ioapic0: Routing IRQ 0 -> intpin 2 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level MADT: Ignoring local NMI routed to ACPI CPU 0 lapic0: Routing NMI -> LINT1 lapic0: LINT1 trigger: edge lapic0: LINT1 polarity: high ioapic0 irqs 0-39 lapic: Divisor 2, Frequency 49885757 Hz lapic: deadline tsc mode, Frequency 2194973264 Hz cpu0 BSP: ID: 0x00000000 VER: 0x01060015 LDR: 0x00000000 DFR: 0xffffffff x2APIC: 0 lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000011ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 cmci: 0x000000f2 SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x01000000 VER: 0x01060015 LDR: 0x00000000 DFR: 0xffffffff x2APIC: 0 lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000011ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 cmci: 0x000100f2 lapic3: CMCI unmasked SMP: AP CPU #3 Launched! cpu3 AP: ID: 0x03000000 VER: 0x01060015 LDR: 0x00000000 DFR: 0xffffffff x2APIC: 0 lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000011ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 cmci: 0x000000f2 SMP: AP CPU #2 Launched! cpu2 AP: ID: 0x02000000 VER: 0x01060015 LDR: 0x00000000 DFR: 0xffffffff x2APIC: 0 lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000011ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 cmci: 0x000100f2 SMP: passed TSC synchronization test TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1097486632 Hz quality 1000 KTLS: Initialized 4 threads Cuse v0.1.36 @ /dev/cuse snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=2 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 random: entropy device external interface wlan: <802.11 Link Layer> 000.000020 [4350] netmap_init netmap: loaded module null: [ath_hal] loaded crypto: tcp_log: tcp_log device io: kbd: new array size 4 WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 13.0. kbd1 at kbdmux0 mem: nexus0 efirtc0: efirtc0: registered as a time-of-day clock, resolution 1.000000s cryptosoft0: crypto: assign cryptosoft0 driver id 0, flags 0x6000000 aesni0: crypto: assign aesni0 driver id 1, flags 0xe000000 acpi0: ACPI: 12 ACPI AML tables successfully acquired and loaded PCIe: Memory Mapped configuration base @ 0xf8000000 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 acpi_ec0: port 0x62,0x66 on acpi0 acpi0: Power Button (fixed) acpi0: wakeup code va 0xfffffe0010bff000 pa 0x5a000 acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, aff00000 (3) failed cpu0: Processor \134_PR_.CPU0 (ACPI ID 1) -> APIC ID 0 cpu0: on acpi0 ACPI: Dynamic OEM Table Load: ACPI: SSDT 0xFFFFF800038FF800 000436 (v01 PmRef Cpu0Cst 00003001 INTL 20120711) cpu1: Processor \134_PR_.CPU1 (ACPI ID 2) -> APIC ID 1 cpu1: on acpi0 ACPI: Dynamic OEM Table Load: ACPI: SSDT 0xFFFFF800038EF800 0005AA (v01 PmRef ApIst 00003000 INTL 20120711) ACPI: Dynamic OEM Table Load: ACPI: SSDT 0xFFFFF80003752480 000119 (v02 PmRef ApCst 00003000 INTL 20120711) cpu2: Processor \134_PR_.CPU2 (ACPI ID 3) -> APIC ID 2 cpu2: on acpi0 cpu3: Processor \134_PR_.CPU3 (ACPI ID 4) -> APIC ID 3 cpu3: on acpi0 ACPI: Processor \134_PR_.CPU4 (ACPI ID 5) ignored ACPI: Processor \134_PR_.CPU5 (ACPI ID 6) ignored ACPI: Processor \134_PR_.CPU6 (ACPI ID 7) ignored ACPI: Processor \134_PR_.CPU7 (ACPI ID 8) ignored attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 2 vector 48 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 hpet0: vendor 0x8086, rev 0x1, 14318180Hz 64bit, 8 timers, legacy route hpet0: t0: irqs 0x00f00000 (0), MSI, 64bit, periodic hpet0: t1: irqs 0x00f00000 (0), MSI hpet0: t2: irqs 0x00f00800 (0), MSI hpet0: t3: irqs 0x00f01000 (0), MSI hpet0: t4: irqs 0x00000000 (0), MSI hpet0: t5: irqs 0x00000000 (0), MSI hpet0: t6: irqs 0x00000000 (0), MSI hpet0: t7: irqs 0x00000000 (0), MSI Timecounter "HPET" frequency 14318180 Hz quality 950 msi: routing MSI-X IRQ 40 to local APIC 0 vector 49 msi: routing MSI-X IRQ 41 to local APIC 2 vector 49 msi: routing MSI-X IRQ 42 to local APIC 0 vector 50 msi: routing MSI-X IRQ 43 to local APIC 2 vector 50 msi: routing MSI-X IRQ 44 to local APIC 0 vector 51 msi: routing MSI-X IRQ 45 to local APIC 2 vector 51 msi: routing MSI-X IRQ 46 to local APIC 0 vector 52 msi: routing MSI-X IRQ 47 to local APIC 2 vector 52 msi: Assigning MSI-X IRQ 41 to local APIC 1 vector 48 msi: Assigning MSI-X IRQ 42 to local APIC 2 vector 49 msi: Assigning MSI-X IRQ 43 to local APIC 3 vector 48 msi: Assigning MSI-X IRQ 45 to local APIC 0 vector 50 msi: Assigning MSI-X IRQ 47 to local APIC 0 vector 53 Event timer "HPET" frequency 14318180 Hz quality 550 Event timer "HPET1" frequency 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz quality 440 Event timer "HPET3" frequency 14318180 Hz quality 440 Event timer "HPET4" frequency 14318180 Hz quality 440 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 54 Event timer "RTC" frequency 32768 Hz quality 0 ACPI timer: 0/4 0/5 0/5 0/4 0/5 0/4 0/5 0/5 0/4 0/5 -> 0 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 9 10 11 Validation 0 255 N 0 3 4 5 6 7 9 10 11 After Disable 0 255 N 0 3 4 5 6 7 9 10 11 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 9 10 11 Validation 0 255 N 0 3 4 5 6 7 9 10 11 After Disable 0 255 N 0 3 4 5 6 7 9 10 11 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 9 10 11 Validation 0 255 N 0 3 4 5 6 7 9 10 11 After Disable 0 255 N 0 3 4 5 6 7 9 10 11 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 9 10 11 Validation 0 255 N 0 3 4 5 6 7 9 10 11 After Disable 0 255 N 0 3 4 5 6 7 9 10 11 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 9 10 11 Validation 0 255 N 0 3 4 5 6 7 9 10 11 After Disable 0 255 N 0 3 4 5 6 7 9 10 11 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 9 10 11 Validation 0 255 N 0 3 4 5 6 7 9 10 11 After Disable 0 255 N 0 3 4 5 6 7 9 10 11 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 9 10 11 Validation 0 255 N 0 3 4 5 6 7 9 10 11 After Disable 0 255 N 0 3 4 5 6 7 9 10 11 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 9 10 11 Validation 0 255 N 0 3 4 5 6 7 9 10 11 After Disable 0 255 N 0 3 4 5 6 7 9 10 11 acpi_lid0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pcib0: decoding 5 range 0-0x3f pcib0: decoding 4 range 0-0xcf7 pcib0: decoding 4 range 0xd00-0xffff pcib0: decoding 3 range 0xa0000-0xbffff pcib0: decoding 3 range 0xc0000-0xc3fff pcib0: decoding 3 range 0xc4000-0xc7fff pcib0: decoding 3 range 0xc8000-0xcbfff pcib0: decoding 3 range 0xcc000-0xcffff pcib0: decoding 3 range 0xd0000-0xd3fff pcib0: decoding 3 range 0xd4000-0xd7fff pcib0: decoding 3 range 0xd8000-0xdbfff pcib0: decoding 3 range 0xdc000-0xdffff pcib0: decoding 3 range 0xb0000000-0xfebfffff pcib0: decoding 3 range 0xfed40000-0xfed4bfff pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x8086, dev=0x1604, revid=0x09 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x2090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x1616, revid=0x09 domain=0, bus=0, slot=2, func=0 class=03-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 1 message map[10]: type Memory, range 64, base 0xe0000000, size 24, enabled pcib0: allocated type 3 (0xe0000000-0xe0ffffff) for rid 10 of pci0:0:2:0 map[18]: type Prefetchable Memory, range 64, base 0xc0000000, size 29, enabled pcib0: allocated type 3 (0xc0000000-0xdfffffff) for rid 18 of pci0:0:2:0 map[20]: type I/O Port, range 32, base 0x3000, size 6, port disabled pcib0: allocated type 4 (0x3000-0x303f) for rid 20 of pci0:0:2:0 found-> vendor=0x8086, dev=0x160c, revid=0x09 domain=0, bus=0, slot=3, func=0 class=04-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 1 message map[10]: type Memory, range 64, base 0xe1130000, size 14, enabled pcib0: allocated type 3 (0xe1130000-0xe1133fff) for rid 10 of pci0:0:3:0 found-> vendor=0x8086, dev=0x9cb1, revid=0x03 domain=0, bus=0, slot=20, func=0 class=0c-03-30, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 8 messages, 64 bit map[10]: type Memory, range 64, base 0xe1120000, size 16, enabled pcib0: allocated type 3 (0xe1120000-0xe112ffff) for rid 10 of pci0:0:20:0 found-> vendor=0x8086, dev=0x9cba, revid=0x03 domain=0, bus=0, slot=22, func=0 class=07-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xe1139000, size 5, enabled pcib0: allocated type 3 (0xe1139000-0xe113901f) for rid 10 of pci0:0:22:0 found-> vendor=0x8086, dev=0x15a3, revid=0x03 domain=0, bus=0, slot=25, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0003, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0xe1100000, size 17, enabled pcib0: allocated type 3 (0xe1100000-0xe111ffff) for rid 10 of pci0:0:25:0 map[14]: type Memory, range 32, base 0xe113d000, size 12, enabled pcib0: allocated type 3 (0xe113d000-0xe113dfff) for rid 14 of pci0:0:25:0 map[18]: type I/O Port, range 32, base 0x3080, size 5, enabled pcib0: allocated type 4 (0x3080-0x309f) for rid 18 of pci0:0:25:0 found-> vendor=0x8086, dev=0x9ca0, revid=0x03 domain=0, bus=0, slot=27, func=0 class=04-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x20 (960 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xe1134000, size 14, enabled pcib0: allocated type 3 (0xe1134000-0xe1137fff) for rid 10 of pci0:0:27:0 found-> vendor=0x8086, dev=0x9c92, revid=0xe3 domain=0, bus=0, slot=28, func=0 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0003, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message secbus=3, subbus=3 found-> vendor=0x8086, dev=0x9c94, revid=0xe3 domain=0, bus=0, slot=28, func=1 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0003, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message secbus=4, subbus=4 found-> vendor=0x8086, dev=0x9cc3, revid=0x03 domain=0, bus=0, slot=31, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x9c83, revid=0x03 domain=0, bus=0, slot=31, func=2 class=01-06-01, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x02b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message map[10]: type I/O Port, range 32, base 0x30a8, size 3, enabled pcib0: allocated type 4 (0x30a8-0x30af) for rid 10 of pci0:0:31:2 map[14]: type I/O Port, range 32, base 0x30b4, size 2, enabled pcib0: allocated type 4 (0x30b4-0x30b7) for rid 14 of pci0:0:31:2 map[18]: type I/O Port, range 32, base 0x30a0, size 3, enabled pcib0: allocated type 4 (0x30a0-0x30a7) for rid 18 of pci0:0:31:2 map[1c]: type I/O Port, range 32, base 0x30b0, size 2, enabled pcib0: allocated type 4 (0x30b0-0x30b3) for rid 1c of pci0:0:31:2 map[20]: type I/O Port, range 32, base 0x3060, size 5, enabled pcib0: allocated type 4 (0x3060-0x307f) for rid 20 of pci0:0:31:2 map[24]: type Memory, range 32, base 0xe113c000, size 11, enabled pcib0: allocated type 3 (0xe113c000-0xe113c7ff) for rid 24 of pci0:0:31:2 found-> vendor=0x8086, dev=0x9ca2, revid=0x03 domain=0, bus=0, slot=31, func=3 class=0c-05-00, hdrtype=0x00, mfdev=0 cmdreg=0x0003, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=255 map[10]: type Memory, range 64, base 0xe1138000, size 8, enabled pcib0: allocated type 3 (0xe1138000-0xe11380ff) for rid 10 of pci0:0:31:3 map[20]: type I/O Port, range 32, base 0xefa0, size 5, enabled pcib0: allocated type 4 (0xefa0-0xefbf) for rid 20 of pci0:0:31:3 found-> vendor=0x8086, dev=0x9ca4, revid=0x03 domain=0, bus=0, slot=31, func=6 class=11-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message map[10]: type Memory, range 64, base 0xe113b000, size 12, enabled pcib0: allocated type 3 (0xe113b000-0xe113bfff) for rid 10 of pci0:0:31:6 vgapci0: port 0x3000-0x303f mem 0xe0000000-0xe0ffffff,0xc0000000-0xdfffffff at device 2.0 on pci0 vgapci0: Boot video device hdac0: mem 0xe1130000-0xe1133fff at device 3.0 on pci0 hdac0: PCI card vendor: 0x17aa, device: 0x2227 hdac0: HDA Driver Revision: 20120126_0002 hdac0: Config options: on=0x00000000 off=0x00000000 hdac0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 48 to local APIC 2 vector 50 hdac0: using IRQ 48 for MSI hdac0: Caps: OSS 3, ISS 0, BSS 0, NSDO 1, 64bit, CORB 256, RIRB 256 xhci0: mem 0xe1120000-0xe112ffff at device 20.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA xhci0: attempting to allocate 1 MSI vectors (8 supported) msi: routing MSI IRQ 49 to local APIC 0 vector 55 xhci0: using IRQ 49 for MSI xhci0: MSI enabled xhci0: Port routing mask set to 0xffffffff usbus0 on xhci0 xhci0: usbpf: Attached usbus0: 5.0Gbps Super Speed USB v3.0 pci0: at device 22.0 (no driver attached) em0: port 0x3080-0x309f mem 0xe1100000-0xe111ffff,0xe113d000-0xe113dfff at device 25.0 on pci0 em0: attach_pre capping queues at 1 em0: Using 1024 TX descriptors and 1024 RX descriptors em0: msix_init qsets capped at 1 em0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 50 to local APIC 2 vector 51 em0: using IRQ 50 for MSI em0: Using an MSI interrupt em0: allocated for 1 tx_queues em0: allocated for 1 rx_queues msi: Assigning MSI IRQ 50 to local APIC 3 vector 49 msi: Assigning MSI IRQ 50 to local APIC 2 vector 51 em0: bpf attached em0: Ethernet address: 54:ee:75:77:0b:ec em0: netmap queues/slots: TX 1/1024, RX 1/1024 hdac1: mem 0xe1134000-0xe1137fff at device 27.0 on pci0 hdac1: PCI card vendor: 0x17aa, device: 0x2227 hdac1: HDA Driver Revision: 20120126_0002 hdac1: Config options: on=0x00000000 off=0x00000000 hdac1: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 51 to local APIC 0 vector 56 hdac1: using IRQ 51 for MSI hdac1: Caps: OSS 4, ISS 4, BSS 0, NSDO 1, 64bit, CORB 256, RIRB 256 pcib1: at device 28.0 on pci0 pcib1: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 52 to local APIC 2 vector 52 pcib1: using IRQ 52 for MSI pcib1: HotPlug command: 0000 -> 1028 pcib1: domain 0 pcib1: secondary bus 3 pcib1: subordinate bus 3 pcib2: at device 28.1 on pci0 pcib0: allocated type 3 (0xe1000000-0xe10fffff) for rid 20 of pcib2 pcib2: domain 0 pcib2: secondary bus 4 pcib2: subordinate bus 4 pcib2: memory decode 0xe1000000-0xe10fffff pci1: on pcib2 pcib2: allocated bus range (4-4) for rid 0 of pci1 pci1: domain=0, physical bus=4 found-> vendor=0x8086, dev=0x08b3, revid=0x83 domain=0, bus=4, slot=0, func=0 class=02-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xe1000000, size 13, enabled pcib2: allocated memory range (0xe1000000-0xe1001fff) for rid 10 of pci0:4:0:0 pci1: at device 0.0 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0x30a8-0x30af,0x30b4-0x30b7,0x30a0-0x30a7,0x30b0-0x30b3,0x3060-0x307f mem 0xe113c000-0xe113c7ff at device 31.2 on pci0 ahci0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 53 to local APIC 0 vector 57 ahci0: using IRQ 53 for MSI ahci0: AHCI v1.30 with 1 6Gbps ports, Port Multiplier not supported ahci0: Caps: 64bit NCQ ALP AL CLO 6Gbps PMD SSC PSC 32cmd 1ports ahci0: Caps2: DESO SADM SDS APST ahcich0: not probed (disabled) ahcich1: not probed (disabled) ahcich2: not probed (disabled) ahcich3: at channel 3 on ahci0 ahcich3: Caps: DSP pci0: at device 31.3 (no driver attached) pci0: at device 31.6 (no driver attached) acpi_tz0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0047 atkbd: keyboard ID 0x54ab (2) kbdc: RESET_KBD return code:00fa kbdc: RESET_KBD status:00aa kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x1d0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 2 vector 53 atkbd0: [GIANT-LOCKED] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0047 kbdc: TEST_AUX_PORT status:0000 kbdc: RESET_AUX return code:00fa kbdc: RESET_AUX status:00aa kbdc: RESET_AUX ID:0000 kbdc: RESET_AUX return code:00fa kbdc: RESET_AUX status:00aa kbdc: RESET_AUX ID:0000 psm: status 00 02 64 psm: status 00 00 64 psm: status 00 03 64 psm: status 00 03 64 psm: data 08 00 00 psm: status 01 47 18 psm: status 01 e2 b1 psm: status f0 03 a3 psm: status 2d 80 44 psm: status 94 33 00 psm: status 12 e8 00 psm: status b1 b6 94 psm: status 27 49 22 psm: status 33 00 40 psm: status 00 02 64 psm0: irq 12 on atkbdc0 ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 58 psm0: [GIANT-LOCKED] WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 13.0. psm0: model Synaptics Touchpad, device ID 0-00, 7 buttons psm0: config:00004000, flags:00000008, packet size:6 psm0: syncmask:c0, syncbits:00 battery0: on acpi0 AcpiOsExecute: task queue not started acpi_acad0: on acpi0 AcpiOsExecute: task queue not started acpi_ibm0: on acpi0 acpi_ibm0: Firmware version is 0x100 AcpiOsExecute: task queue not started ACPI: Enabled 4 GPEs in block 00 to 7F pcib0: allocated type 3 (0xa0000-0xa07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa0800-0xa0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa1000-0xa17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa1800-0xa1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa2000-0xa27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa2800-0xa2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa3000-0xa37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa3800-0xa3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa4000-0xa47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa4800-0xa4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa5000-0xa57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa5800-0xa5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa6000-0xa67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa6800-0xa6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa7000-0xa77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa7800-0xa7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa8000-0xa87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa8800-0xa8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa9000-0xa97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa9800-0xa9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xaa000-0xaa7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xaa800-0xaafff) for rid 0 of orm0 pcib0: allocated type 3 (0xab000-0xab7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xab800-0xabfff) for rid 0 of orm0 pcib0: allocated type 3 (0xac000-0xac7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xac800-0xacfff) for rid 0 of orm0 pcib0: allocated type 3 (0xad000-0xad7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xad800-0xadfff) for rid 0 of orm0 pcib0: allocated type 3 (0xae000-0xae7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xae800-0xaefff) for rid 0 of orm0 pcib0: allocated type 3 (0xaf000-0xaf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xaf800-0xaffff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0 pcib0: allocated type 3 (0xc0000-0xc07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc0800-0xc0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc1000-0xc17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc1800-0xc1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc2000-0xc27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc2800-0xc2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc3000-0xc37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc3800-0xc3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc4000-0xc47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc4800-0xc4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc5000-0xc57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc5800-0xc5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc6000-0xc67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc6800-0xc6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc7000-0xc77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc7800-0xc7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc8000-0xc87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc8800-0xc8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc9000-0xc97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc9800-0xc9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xca000-0xca7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xca800-0xcafff) for rid 0 of orm0 pcib0: allocated type 3 (0xcb000-0xcb7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xcb800-0xcbfff) for rid 0 of orm0 pcib0: allocated type 3 (0xcc000-0xcc7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xcc800-0xccfff) for rid 0 of orm0 pcib0: allocated type 3 (0xcd000-0xcd7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xcd800-0xcdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xce000-0xce7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xce800-0xcefff) for rid 0 of orm0 pcib0: allocated type 3 (0xcf000-0xcf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xcf800-0xcffff) for rid 0 of orm0 pcib0: allocated type 3 (0xd0000-0xd07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd0800-0xd0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd1000-0xd17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd1800-0xd1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd2000-0xd27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd2800-0xd2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd3000-0xd37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd3800-0xd3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd4000-0xd47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd4800-0xd4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd5000-0xd57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd5800-0xd5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd6000-0xd67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd6800-0xd6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd7000-0xd77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd7800-0xd7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd8000-0xd87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd8800-0xd8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd9000-0xd97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd9800-0xd9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xda000-0xda7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xda800-0xdafff) for rid 0 of orm0 pcib0: allocated type 3 (0xdb000-0xdb7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xdb800-0xdbfff) for rid 0 of orm0 pcib0: allocated type 3 (0xdc000-0xdc7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xdc800-0xdcfff) for rid 0 of orm0 pcib0: allocated type 3 (0xdd000-0xdd7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xdd800-0xddfff) for rid 0 of orm0 pcib0: allocated type 3 (0xde000-0xde7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xde800-0xdefff) for rid 0 of orm0 pcib0: allocated type 3 (0xdf000-0xdf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xdf800-0xdffff) for rid 0 of orm0 ahc_isa_identify 0: ioport 0xc00 alloc failed ahc_isa_identify 1: ioport 0x1c00 alloc failed ahc_isa_identify 2: ioport 0x2c00 alloc failed ahc_isa_identify 3: ioport 0x3c00 alloc failed ahc_isa_identify 4: ioport 0x4c00 alloc failed ahc_isa_identify 5: ioport 0x5c00 alloc failed ahc_isa_identify 6: ioport 0x6c00 alloc failed ahc_isa_identify 7: ioport 0x7c00 alloc failed ahc_isa_identify 8: ioport 0x8c00 alloc failed ahc_isa_identify 9: ioport 0x9c00 alloc failed ahc_isa_identify 10: ioport 0xac00 alloc failed ahc_isa_identify 11: ioport 0xbc00 alloc failed ahc_isa_identify 12: ioport 0xcc00 alloc failed ahc_isa_identify 13: ioport 0xdc00 alloc failed ahc_isa_identify 14: ioport 0xec00 alloc failed isa_probe_children: disabling PnP devices atkbdc: atkbdc0 already exists; skipping it atrtc: atrtc0 already exists; skipping it attimer: attimer0 already exists; skipping it sc: sc0 already exists; skipping it isa_probe_children: probing non-PnP devices sc0 failed to probe on isa0 vga0 failed to probe on isa0 pcib0: allocated type 4 (0x3f0-0x3f5) for rid 0 of fdc0 pcib0: allocated type 4 (0x3f7-0x3f7) for rid 1 of fdc0 fdc0 failed to probe at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 ppc0: cannot reserve I/O port range ppc0 failed to probe at irq 7 on isa0 pcib0: allocated type 4 (0x3f8-0x3f8) for rid 0 of uart0 uart0 failed to probe at port 0x3f8 irq 4 on isa0 pcib0: allocated type 4 (0x2f8-0x2f8) for rid 0 of uart1 uart1 failed to probe at port 0x2f8 irq 3 on isa0 isa_probe_children: probing PnP devices AcpiOsExecute: task queue not started AcpiOsExecute: task queue not started AcpiOsExecute: task queue not started AcpiOsExecute: task queue not started coretemp0: on cpu0 coretemp0: Setting TjMax=105 est0: on cpu0 coretemp1: on cpu1 coretemp1: Setting TjMax=105 est1: on cpu1 coretemp2: on cpu2 coretemp2: Setting TjMax=105 est2: on cpu2 coretemp3: on cpu3 coretemp3: Setting TjMax=105 est3: on cpu3 Device configuration finished. procfs registered Timecounters tick every 1.000 msec ZFS filesystem version: 5 ZFS storage pool version: features support (5000) lo0: bpf attached vlan: initialized, using hash tables with chaining crypto: tcp_init: net.inet.tcp.tcbhashsize auto tuned to 65536 IPsec: Initialized Security Association Processing. AcpiOsExecute: enqueue 7 pending tasks battery0: battery enitialization start acpi_acad0: acline initialization start ugen0.1: <0x8086 XHCI root HUB> at usbus0 hdacc0: ahcich3: AHCI reset... at cad 0acpi_acad0: On Line ahcich3: SATA connect time=100us status=00000133 on hdac0 ahcich3: AHCI reset: device found hdaa0: acpi_acad0: acline initialization done, tried 1 times at nid 1ahcich3: AHCI reset: device ready after 0ms on hdacc0 hdaa0: Subsystem ID: 0x80860101 hdaa0: NumGPIO=0 NumGPO=0 NumGPI=0 GPIWake=0 GPIUnsol=0 hdaa0: Original pins configuration: hdaa0: nid 0x as seq device conn jack loc color misc battery0: rev = ffff hdaa0: 3 18560010 1 0 Digital-out Jack Digital 0x18 Unknown 0 hdaa0: Patched pins configuration: battery0: units = 0 battery0: dcap = 0 battery0: lfcap = 0 hdaa0: nid 0x as seq device conn jack loc color misc battery0: btech = 0 hdaa0: 3 18560010 1 0 Digital-out Jack Digital 0x18 Unknown 0 hdaa0: 1 associations found: hdaa0: Association 0 (1) out: hdaa0: Pin nid=3 seq=0 battery0: dvol = 0 hdaa0: Tracing association 0 (1) battery0: wcap = 0 battery0: lcap = 0 battery0: gra1 = 0 battery0: gra2 = 0 battery0: battery initialization done, tried 1 times hdaa0: Pin 3 traced to DAC 2 hdaa0: Association 0 (1) trace succeeded hdaa0: Looking for additional DAC for association 0 (1) hdaa0: Tracing input monitor hdaa0: Tracing other input monitors hdaa0: Tracing beeper hdaa0: FG config/quirks: forcestereo ivref50 ivref80 ivref100 ivref pcm0: at nid 3 on hdaa0 pcm0: Playback: pcm0: Stream cap: 0x00000005 AC3 PCM pcm0: PCM cap: 0x001e07f0 16 20 24 32 bits, 32 44 48 88 96 176 192 KHz pcm0: DAC: 2 pcm0: pcm0: nid=3 [pin: Digital-out (Jack)] pcm0: + <- nid=2 [audio output] [src: pcm] pcm0: pcm0: Master Volume (OSS: vol): 0/0dB pcm0: +- ctl 1 (nid 3 in ): mute pcm0: pcm0: PCM Volume (OSS: pcm): 0/0dB pcm0: +- ctl 1 (nid 3 in ): mute pcm0: pcm0: Mixer "vol": pcm0: Mixer "pcm": pcm0: Soft PCM mixer ENABLED pcm0: Playback channel matrix is: unknown, assuming 7.1 (disconnected) hdacc1: at cad 0 on hdac1 hdaa1: at nid 1 on hdacc1 hdaa1: Subsystem ID: 0x17aa2227 hdaa1: NumGPIO=5 NumGPO=0 NumGPI=0 GPIWake=0 GPIUnsol=1 hdaa1: GPIO0: disabled hdaa1: GPIO1: disabled hdaa1: GPIO2: disabled hdaa1: GPIO3: disabled hdaa1: GPIO4: disabled hdaa1: Original pins configuration: hdaa1: nid 0x as seq device conn jack loc color misc hdaa1: 18 90a60130 3 0 Mic Fixed Digital Internal Unknown 1 hdaa1: 19 40000000 0 0 Line-out None Unknown 0x00 Unknown 0 hdaa1: 20 90170110 1 0 Speaker Fixed Analog Internal Unknown 1 hdaa1: 21 03211040 4 0 Headphones Jack 1/8 Left Black 0 hdaa1: 22 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa1: 24 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa1: 25 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa1: 26 03a11020 2 0 Mic Jack 1/8 Left Black 0 hdaa1: 27 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa1: 29 40738105 0 5 Modem-handset None ATAPI 0x00 Purple 1 hdaa1: 30 411111f0 15 0 Speaker None 1/8 Rear Black 1 hdaa1: Patching pin config nid=21 0x03211040 -> 0x0321101f hdaa1: Patching widget caps nid=29 0x00400400 -> 0x00700400 hdaa1: Patched pins configuration: hdaa1: nid 0x as seq device conn jack loc color misc hdaa1: 18 90a60130 3 0 Mic Fixed Digital Internal Unknown 1 hdaa1: 19 40000000 0 0 Line-out None Unknown 0x00 Unknown 0 DISA hdaa1: 20 90170110 1 0 Speaker Fixed Analog Internal Unknown 1 hdaa1: 21 0321101f 1 15 Headphones Jack 1/8 Left Black 0 hdaa1: 22 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa1: 24 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa1: 25 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa1: 26 03a11020 2 0 Mic Jack 1/8 Left Black 0 hdaa1: 27 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa1: 30 411111f0 15 0 Speaker None 1/8 Rear Black 1 DISA hdaa1: 3 associations found: hdaa1: Association 0 (1) out: hdaa1: Pin nid=20 seq=0 hdaa1: Pin nid=21 seq=15 hdaa1: Association 1 (2) in: hdaa1: Pin nid=26 seq=0 hdaa1: Association 2 (3) in: hdaa1: Pin nid=18 seq=0 hdaa1: Tracing association 0 (1) hdaa1: Pin 20 traced to DAC 2 hdaa1: Pin 21 traced to DAC 2 and hpredir 0 hdaa1: Association 0 (1) trace succeeded hdaa1: Tracing association 1 (2) hdaa1: Pin 26 traced to ADC 8 hdaa1: Association 1 (2) trace succeeded hdaa1: Tracing association 2 (3) hdaa1: Pin 18 traced to ADC 9 hdaa1: Association 2 (3) trace succeeded hdaa1: Looking for additional DAC for association 0 (1) hdaa1: Looking for additional ADC for association 1 (2) hdaa1: Looking for additional ADC for association 2 (3) hdaa1: Tracing input monitor hdaa1: Tracing nid 11 to out hdaa1: nid 11 is input monitor hdaa1: Tracing other input monitors hdaa1: Tracing nid 18 to out hdaa1: Tracing nid 26 to out hdaa1: Tracing beeper hdaa1: Headphones redirection for association 0 nid=21 using unsolicited responses. hdaa1: Redirect output to: main hdaa1: FG config/quirks: forcestereo ivref50 ivref80 ivref100 ivref pcm1: at nid 20,21 and 26 on hdaa1 pcm1: Playback: pcm1: Stream cap: 0x00000001 PCM pcm1: PCM cap: 0x000e0560 16 20 24 bits, 44 48 96 192 KHz pcm1: DAC: 2 pcm1: pcm1: nid=20 [pin: Speaker (Fixed)] pcm1: + <- nid=12 [audio mixer] [src: pcm, mix] pcm1: + <- nid=2 [audio output] [src: pcm] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: nid=21 [pin: Headphones (Black Jack)] pcm1: + <- nid=12 [audio mixer] [src: pcm, mix] pcm1: + <- nid=2 [audio output] [src: pcm] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: Record: pcm1: Stream cap: 0x00000001 PCM pcm1: PCM cap: 0x000e0560 16 20 24 bits, 44 48 96 192 KHz pcm1: ADC: 8 pcm1: pcm1: nid=8 [audio input] pcm1: + <- nid=35 [audio selector] [src: speaker, mic, mix] pcm1: + <- nid=26 [pin: Mic (Black Jack)] [src: mic] pcm1: + <- nid=29 [beep widget] [src: speaker] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: Input Mix: pcm1: pcm1: nid=11 [audio mixer] pcm1: + <- nid=26 [pin: Mic (Black Jack)] [src: mic] pcm1: + <- nid=29 [beep widget] [src: speaker] pcm1: pcm1: Master Volume (OSS: vol): -65/0dB pcm1: +- ctl 1 (nid 2 out): -65/0dB (88 steps) pcm1: +- ctl 10 (nid 12 in 0): mute pcm1: +- ctl 11 (nid 12 in 1): mute pcm1: +- ctl 18 (nid 20 in ): mute pcm1: +- ctl 19 (nid 21 in ): mute pcm1: pcm1: PCM Volume (OSS: pcm): -65/0dB pcm1: +- ctl 1 (nid 2 out): -65/0dB (88 steps) pcm1: +- ctl 10 (nid 12 in 0): mute pcm1: pcm1: Microphone Volume (OSS: mic): 0/30dB pcm1: +- ctl 3 (nid 8 in 0): -17/30dB (64 steps) + mute pcm1: +- ctl 7 (nid 11 in 2): -34/12dB (32 steps) + mute pcm1: +- ctl 24 (nid 26 out): 0/30dB (4 steps) pcm1: pcm1: Speaker/Beep Volume (OSS: speaker): -17/12dB pcm1: +- ctl 3 (nid 8 in 0): -17/30dB (64 steps) + mute pcm1: +- ctl 9 (nid 11 in 4): -34/12dB (32 steps) + mute pcm1: pcm1: Recording Level (OSS: rec): -17/30dB pcm1: +- ctl 3 (nid 8 in 0): -17/30dB (64 steps) + mute pcm1: pcm1: Input Mix Level (OSS: mix): -17/30dB pcm1: +- ctl 3 (nid 8 in 0): -17/30dB (64 steps) + mute pcm1: +- ctl 11 (nid 12 in 1): mute pcm1: pcm1: Input Monitoring Level (OSS: igain): 0/0dB pcm1: +- ctl 11 (nid 12 in 1): mute pcm1: pcm1: Mixer "vol": pcm1: Mixer "pcm": pcm1: Mixer "speaker": pcm1: Mixer "mic": pcm1: Mixer "mix": pcm1: Mixer "rec": pcm1: Mixer "igain": pcm1: Mixer "ogain": pcm1: Playback channel set is: Front Left, Front Right, pcm1: Playback channel matrix is: 2.0 (unknown) pcm1: Recording channel set is: Front Left, Front Right, pcm1: Recording channel matrix is: 2.0 (disconnected) pcm2: at nid 18 on hdaa1 pcm2: Record: pcm2: Stream cap: 0x00000001 PCM pcm2: PCM cap: 0x000e0560 16 20 24 bits, 44 48 96 192 KHz pcm2: ADC: 9 pcm2: pcm2: nid=9 [audio input] pcm2: + <- nid=34 [audio selector] [src: speaker, monitor] pcm2: + <- nid=29 [beep widget] [src: speaker] pcm2: + <- nid=18 [pin: Mic (Fixed)] [src: monitor] pcm2: pcm2: Microphone2 Volume (OSS: monitor): 0/36dB pcm2: +- ctl 4 (nid 9 in 0): -17/30dB (64 steps) + mute pcm2: +- ctl 16 (nid 18 out): 0/36dB (4 steps) pcm2: pcm2: Speaker/Beep Volume (OSS: speaker) pcm2: +- ctl 4 (nid 9 in 0): -17/30dB (64 steps) + mute pcm2: pcm2: Recording Level (OSS: rec): -17/30dB pcm2: +- ctl 4 (nid 9 in 0): -17/30dB (64 steps) + mute pcm2: +- ctl 16 (nid 18 out): 0/36dB (4 steps) pcm2: pcm2: Mixer "rec": pcm2: Mixer "monitor": pcm2: Automatically set rec source to: monitor pcm2: Recording channel set is: Front Left, Front Right, pcm2: Recording channel matrix is: 2.0 (unknown) Trying to mount root from zfs:zroot/ROOT/default []... Root mount waiting for: usbus0 CAM uhub0 on usbus0 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 GEOM: new disk ada0 ada0 at ahcich3 bus 0 scbus0 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number S20GNYAG777654 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada0: Command Queueing enabled ada0: 244198MB (500118192 512 byte sectors) pass0 at ahcich3 bus 0 scbus0 target 0 lun 0 pass0: ACS-2 ATA SATA 3.x device pass0: Serial Number S20GNYAG777654 pass0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) pass0: Command Queueing enabled uhub0: 15 ports with 15 removable, self powered Root mount waiting for: usbus0 ugen0.2: at usbus0 Root mount waiting for: usbus0 ugen0.3: at usbus0 Root mount waiting for: usbus0 usb_alloc_device: set address 4 failed (USB_ERR_TIMEOUT, ignored) Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 usbd_setup_device_desc: getting device descriptor at addr 4 failed, USB_ERR_TIMEOUT Root mount waiting for: usbus0 usbd_req_re_enumerate: addr=4, set address failed! (USB_ERR_TIMEOUT, ignored) Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 usbd_setup_device_desc: getting device descriptor at addr 4 failed, USB_ERR_TIMEOUT Root mount waiting for: usbus0 Root mount waiting for: usbus0 usbd_req_re_enumerate: addr=4, set address failed! (USB_ERR_TIMEOUT, ignored) Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 usbd_setup_device_desc: getting device descriptor at addr 4 failed, USB_ERR_TIMEOUT Root mount waiting for: usbus0 usbd_req_re_enumerate: addr=4, set address failed! (USB_ERR_TIMEOUT, ignored) Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 usbd_setup_device_desc: getting device descriptor at addr 4 failed, USB_ERR_TIMEOUT Root mount waiting for: usbus0 Root mount waiting for: usbus0 usbd_req_re_enumerate: addr=4, set address failed! (USB_ERR_TIMEOUT, ignored) Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 usbd_setup_device_desc: getting device descriptor at addr 4 failed, USB_ERR_TIMEOUT ugen0.4: at usbus0 (disconnected) uhub_reattach_port: could not allocate new device Root mount waiting for: usbus0 ugen0.4: at usbus0 efirtc0: providing initial system time start_init: trying /sbin/init pci0: driver added found-> vendor=0x8086, dev=0x9cba, revid=0x03 domain=0, bus=0, slot=22, func=0 class=07-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit pci0:0:22:0: reprobing on driver added found-> vendor=0x8086, dev=0x9ca2, revid=0x03 domain=0, bus=0, slot=31, func=3 class=0c-05-00, hdrtype=0x00, mfdev=0 cmdreg=0x0003, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=255 pci0:0:31:3: reprobing on driver added ichsmb0: port 0xefa0-0xefbf mem 0xe1138000-0xe11380ff at device 31.3 on pci0 pcib0: matched entry for 0.31.INTC pcib0: slot 31 INTC hardwired to IRQ 18 ioapic0: routing intpin 18 (PCI IRQ 18) to lapic 2 vector 54 smbus0: on ichsmb0 found-> vendor=0x8086, dev=0x9ca4, revid=0x03 domain=0, bus=0, slot=31, func=6 class=11-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message pci0:0:31:6: reprobing on driver added pchtherm0: mem 0xe113b000-0xe113bfff at device 31.6 on pci0 pchtherm0: Catastrophic Power Down Locked Enable pchtherm0: SMBus report Locked Enable pchtherm0: SMI on alert Locked Enable pchtherm0: TSPM: 6 pchtherm0: MAX Thermal Sensor Shutdown Time 16s pchtherm0: Throttling 5 pchtherm0: TL2 flag 1 pci1: driver added found-> vendor=0x8086, dev=0x08b3, revid=0x83 domain=0, bus=4, slot=0, func=0 class=02-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit pci0:4:0:0: reprobing on driver added iwm0: mem 0xe1000000-0xe1001fff at device 0.0 on pci1 iwm0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 54 to local APIC 0 vector 59 iwm0: using IRQ 54 for MSI firmware: 'iwm3160fw' version 0: 918268 bytes loaded at 0xffffffff83121000 iwm0: hw rev 0x160, fw ver 17.352738.0, address 2c:6e:85:41:b1:8a iwm0: 11a rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps iwm0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps iwm0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps pci0: driver added found-> vendor=0x8086, dev=0x9cba, revid=0x03 domain=0, bus=0, slot=22, func=0 class=07-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit pci0:0:22:0: reprobing on driver added pci1: driver added pci0: driver added found-> vendor=0x8086, dev=0x9cba, revid=0x03 domain=0, bus=0, slot=22, func=0 class=07-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit pci0:0:22:0: reprobing on driver added pci1: driver added ACPI: Processor \134_PR_.CPU4 (ACPI ID 5) ignored ACPI: Processor \134_PR_.CPU5 (ACPI ID 6) ignored ACPI: Processor \134_PR_.CPU6 (ACPI ID 7) ignored ACPI: Processor \134_PR_.CPU7 (ACPI ID 8) ignored acpi_wmi0: on acpi0 acpi_wmi0: Embedded MOF found ACPI: \134_SB.WMI1.WQBA: 1 arguments were passed to a non-method ACPI object (Buffer) (20201113/nsarguments-361) acpi_wmi1: on acpi0 acpi_wmi1: Embedded MOF found ACPI: \134_SB.WMI2.WQBB: 1 arguments were passed to a non-method ACPI object (Buffer) (20201113/nsarguments-361) acpi_wmi2: on acpi0 acpi_wmi2: Embedded MOF found ACPI: \134_SB.WMI3.WQBC: 1 arguments were passed to a non-method ACPI object (Buffer) (20201113/nsarguments-361) anon_inodefs registered debugfs registered drmn0: on vgapci0 vgapci0: child drmn0 requested pci_enable_io vgapci0: child drmn0 requested pci_enable_io [drm] Unable to create a private tmpfs mount, hugepage support will be disabled(-19). Successfully added WC MTRR for [0xc0000000-0xdfffffff]: 0; [drm] Got stolen memory base 0xae000000, size 0x2000000 vgapci0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 55 to local APIC 2 vector 55 vgapci0: using IRQ 55 for MSI [drm] Supports vblank timestamp caching Rev 2 (21.10.2013). [drm] Driver supports precise vblank timestamp query. ACPI: Failed to enumerate DSM functions [drm] Connector eDP-1: get mode from tunables: [drm] - kern.vt.fb.modes.eDP-1 [drm] - kern.vt.fb.default_mode [drm] Connector DP-1: get mode from tunables: [drm] - kern.vt.fb.modes.DP-1 [drm] - kern.vt.fb.default_mode [drm] Connector HDMI-A-1: get mode from tunables: [drm] - kern.vt.fb.modes.HDMI-A-1 [drm] - kern.vt.fb.default_mode [drm] Connector DP-2: get mode from tunables: [drm] - kern.vt.fb.modes.DP-2 [drm] - kern.vt.fb.default_mode [drm] Connector HDMI-A-2: get mode from tunables: [drm] - kern.vt.fb.modes.HDMI-A-2 [drm] - kern.vt.fb.default_mode sysctl_warn_reuse: can't re-use a leaf (hw.dri.debug)! [drm] Initialized i915 1.6.0 20190822 for drmn0 on minor 0 WARNING: Device "fb" is Giant locked and may be deleted before FreeBSD 13.0. VT: Replacing driver "efifb" with new "fb". start FB_INFO: type=11 height=1080 width=1920 depth=32 cmsize=16 size=8294400 pbase=0xc0000000 vbase=0xfffff800c0000000 name=drmn0 flags=0x0 stride=7680 bpp=32 cmap[0]=0 cmap[1]=7f0000 cmap[2]=7f00 cmap[3]=c4a000 end FB_INFO drmn0: fb0: i915drmfb frame buffer device wlan0: bpf attached wlan0: bpf attached wlan0: Ethernet address: 2c:6e:85:41:b1:8a lo0: link state changed to UP wlan0: link state changed to UP cdce0 on uhub0 cdce0: on usbus0 ue0: on cdce0 ue0: bpf attached ue0: Ethernet address: 00:00:11:12:13:14 umodem0 on uhub0 umodem0: on usbus0 umodem0: data interface 3, has no CM over data, has break From owner-freebsd-stable@freebsd.org Tue Feb 2 18:20:19 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0814C539C27 for ; Tue, 2 Feb 2021 18:20:19 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DVY5G6g9Bz3v2W; Tue, 2 Feb 2021 18:20:18 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-wm1-x32f.google.com with SMTP id l12so2293449wmq.2; Tue, 02 Feb 2021 10:20:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=JP/+TiQjJ8asDE0VCj9Vs2CZQFdnlBByvNiilW3yLi0=; b=GGjn6YS4Aoqf8fyLGPjaOOxyMzW379G0vnNefyTi+ie0SILa5oN39qC2WZRz52/xmi F0373KYxx/60STCDtTYXxQulUTlVqxf5HE4OFDiLZOULWTOTIQVI5WGoguVewRNJ3xzc OqjuNwHkSjAVH/ySkmEPMSq4fK8p8OYEFDjGaf0C43rWRkB1CVVBvFAdkKFBma6cl84r L1SKG38n1TdUdkU7q8JMH9+UPPUpC5oqPiWl/fPUjdKHxRIsBMS9oEVwVrg9VGpDIdYA 5QVBYgqbcpB3HOmDB1uBv3jzlJfEqf6ALVofy29+fI7v2DDliDwsVt/GDQAAjE1ipD5U T5uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=JP/+TiQjJ8asDE0VCj9Vs2CZQFdnlBByvNiilW3yLi0=; b=byTU1Afwy3rjRXg9tkShHDHe23vpQBcOz0inO6dpCvRichEbkcFUBKNN2p85V7D/Wh Nvam+/aCobGm4flSRm2P5au17dKbfFWAzczuwUT3W+9NoYswR+U4BxMVRWp/d3xEkW/p PWeXnlTJkijnzANLt3zGj/k+iiBt+4MLbOBoZdxkf62D++nNzFEIgtl3f+r2VzRAbH6A qVoEBxEuphtmtXNSodxkvXGoRwJiV4kubMBw/MoDJhPj8sydkeW9qdZ0LWDXaMC36SHT 01sbYV0lI9afUvpDnsmndFmzDyuy9ma9tud3XjCQ5FwiatsCFiNncPQeztOC9MjGWBNt OVew== X-Gm-Message-State: AOAM53169KK3qcmvKee3Bw8RTDl4gG/u1gX3SuEHZYhhsa8KzY2Ues0O qv8JvWFnwYuZ42PUAzczBeQmLzAsV2+BYHgVciqR6XQHFt6F8g== X-Google-Smtp-Source: ABdhPJwR9v01ybuwyHrqi3MlcKJAco6bXF6KocVh3+yz3KjmIptAU1n//iJgR1pw8txjUNSr62Ez7rnr26tftV7y3X0= X-Received: by 2002:a1c:98d4:: with SMTP id a203mr4729756wme.10.1612290016722; Tue, 02 Feb 2021 10:20:16 -0800 (PST) MIME-Version: 1.0 Received: by 2002:a5d:464c:0:0:0:0:0 with HTTP; Tue, 2 Feb 2021 10:20:15 -0800 (PST) In-Reply-To: References: <890f2a63-62a9-f4a7-b57f-5b122a4a9709@gmail.com> <9821a854-9c86-060b-607b-01a2fbbc076e@gmail.com> <44315237-2B01-4C1B-A992-E235A5A911B0@FreeBSD.org> From: Mateusz Guzik Date: Tue, 2 Feb 2021 19:20:15 +0100 Message-ID: Subject: Re: latest stable13.0-ALPHA3 can not start varnish anymore. To: Kyle Evans Cc: Kristof Provost , Johan Hendriks , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4DVY5G6g9Bz3v2W X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[]; REPLY(-4.00)[] X-Mailman-Approved-At: Wed, 03 Feb 2021 18:11:08 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2021 18:20:19 -0000 I fixed the problem and given the impending builds merged it immediately. Can be seen here: https://cgit.freebsd.org/src/commit/?id=4e29933d0936fd053f7591ee118dc1fc42617514 Should you run into any other problems, they can be temporarily worked around with: sysctl vfs.cache_fast_lookup=0 For problem reports in the area please collect: dtrace -n 'vfs:fplookup:lookup:done { @[arg1, arg2, stringof(args[0].ni_cnd.cn_pnbuf)] = count(); }' -c "failing command" On 2/2/21, Kyle Evans wrote: > On Tue, Feb 2, 2021 at 10:42 AM Kristof Provost wrote: >> >> On 2 Feb 2021, at 14:05, Johan Hendriks wrote: >> > On 01/02/2021 22:48, Johan Hendriks wrote: >> >> I just updated my FreeBSD 13.0-APLPHA3 to the latest revision and now >> >> i can not start varnish anymore. >> >> This is on two machines. >> >> >> >> if i start varnish it errors out as the startup script does a config >> >> file check. >> >> If i try to do a test of the config file i get the following error. >> >> >> >> root@jhost002:~ # varnishd -C -f /usr/local/etc/varnish/default.vcl >> >> Error: Cannot create working directory '/tmp/varnishd_C_dwbl7mn/': Is >> >> a directory >> >> Error: Cannot create working directory (/tmp/varnishd_C_dwbl7mn/): No >> >> error: 0 >> >> (-? gives usage) >> >> root@jhost002:~ # >> >> >> >> This is on: >> >> FreeBSD jhost002.mydomain.nl 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #35 >> >> stable/13-c256281-gc415d0df47fc: Mon Feb 1 17:04:49 CET 2021 >> >> root@srv-01.home.local:/usr/obj/usr/src/amd64.amd64/sys/KRNL amd64 >> >> >> >> I did not update the package or installed any other software besides >> >> the buildworld. >> >> >> >> regards, >> >> Johan >> >> >> >> >> > I have tried some bisecting as far as my understanding of git goes. I >> > do not know which one is the latest, but on these revisions varnish >> > works. >> > >> > FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #8 >> > c256261-g9375a93b6c22: Tue Feb 2 13:33:05 CET 2021 >> > root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 >> > >> > uname -a >> > FreeBSD jhost002 13.0-ALPHA3 FreeBSD 13.0-ALPHA3 #7 >> > c256260-g247f652e622d: Tue Feb 2 13:07:37 CET 2021 >> > root@jhost002:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 >> > >> Can you try setting `sysctl vfs.cache_fast_lookup=0` ? >> >> (As suggested by Mateusz elsewhere.) >> > > We confirmed that reverting 006ec2ed fixes it, I'm still trying to get > an idea of what's happening. My initial analysis suggests that > fplookup hits the trailing slash and aborts, leaving us at the > degenerate state back up in the slow-path. > > I have a small reproducer that I'm working with: > https://people.freebsd.org/~kevans/namereg.c > -- Mateusz Guzik From owner-freebsd-stable@freebsd.org Thu Feb 4 16:08:20 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CB22A5478CF for ; Thu, 4 Feb 2021 16:08:20 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: from onyphe.fr (super1.onyphe.io [54.36.107.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWk436PFRz4Y1N for ; Thu, 4 Feb 2021 16:08:19 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: (qmail 41359 invoked by uid 0); 4 Feb 2021 16:08:12 -0000 Received: from unknown (HELO www.onyphe.io) (172.16.6.254) by smtpout.jail with SMTP; 4 Feb 2021 16:08:12 -0000 MIME-Version: 1.0 Date: Thu, 04 Feb 2021 17:08:12 +0100 From: GomoR To: freebsd-stable@freebsd.org Cc: jhb@freebsd.org Subject: Suspected mbuf leak with Nginx + sendfile + TLS in 12.2-STABLE User-Agent: Roundcube Webmail/1.4.8 Message-ID: X-Sender: freebsd-stable@gomor.org Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DWk436PFRz4Y1N X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-stable@gomor.org designates 54.36.107.100 as permitted sender) smtp.mailfrom=freebsd-stable@gomor.org X-Spamd-Result: default: False [-2.99 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[54.36.107.100:from]; R_SPF_ALLOW(-0.20)[+ip4:54.36.107.100]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[gomor.org]; SPAMHAUS_ZRD(0.00)[54.36.107.100:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.69)[-0.694]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:16276, ipnet:54.36.0.0/16, country:FR]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2021 16:08:20 -0000 Dear FreeBSD community, we are encountering a DoS condition on our production machines. Our use case is an Nginx reverse proxy serving large files via HTTPS. This problem arose when switching kernel and userland from 12.1-RELEASE to 12.2-RELEASE. Ports were not upgraded (at first). Each time a user downloads a file, mbuf & mbuf_clusters are raising to reach the maximum limit in a matter of seconds. Those values are asserted by 'netstat -m' as follows: Normal situation: mbuf: 256, 26031105, 16767, 5974,428087938, 0, 0 mbuf_cluster: 2048, 8135232, 18408, 2704,101644203, 0, 0 Warning situtation: mbuf: 256, 26031105, 2981516, 151205,1109483561, 0, 0 mbuf_cluster: 2048, 8135232, 2983155, 4201,319714617, 0, 0 We have seen a patch related to sendfile + KTLS + mbuf at the below link and we updated to -STABLE to apply: Don't transmit mbufs that aren't yet ready on TOE sockets. This includes mbufs waiting for data from sendfile() I/O requests, or mbufs awaiting encryption for KTLS. https://github.com/freebsd/freebsd-src/commit/14c77f30b201bf76119d59678e72051c093333c2 Unfortunately for us, applying it didn't solve the issue. When we stop the download early, mbufs are freed. But past a threshold, we must reboot the server. The only remaining thing we can do is to ping the server, it is no more possible to connect with SSH, for instance. We also tried to set some loader.conf values which fixed nothing: hw.ix.enable_msix=0 hw.pci.enable_msix=0 hw.pci.enable_msi=0 net.inet.tcp.tso=0 hw.ix.flow_control=0 We also updated Nginx & OpenSSL to latest versions and tried Nginx to compile against FreeBSD shipped OpenSSL library. It did change nothing. Versions: openssl-1.1.1i,1 nginx-1.18.0_45,2 # ldd /usr/local/sbin/nginx /usr/local/sbin/nginx: libcrypt.so.5 => /lib/libcrypt.so.5 (0x800323000) libpcre.so.1 => /usr/local/lib/libpcre.so.1 (0x800344000) libssl.so.11 => /usr/local/lib/libssl.so.11 (0x8003e7000) libcrypto.so.11 => /usr/local/lib/libcrypto.so.11 (0x80047e000) libz.so.6 => /lib/libz.so.6 (0x800772000) libc.so.7 => /lib/libc.so.7 (0x80078e000) libthr.so.3 => /lib/libthr.so.3 (0x800b84000) NIC is: ix0: What can we do to help you find the root cause? Best regards, P.S.: adding jhb@ in Cc from bapt@ suggestion From owner-freebsd-stable@freebsd.org Thu Feb 4 18:33:47 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C3C4952B7A0 for ; Thu, 4 Feb 2021 18:33:47 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWnHv5DQwz4l1R; Thu, 4 Feb 2021 18:33:47 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro.local (unknown [IPv6:2601:648:8681:1cb0:2563:9ea7:c86c:ced1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 61935A254; Thu, 4 Feb 2021 18:33:47 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Subject: Re: Suspected mbuf leak with Nginx + sendfile + TLS in 12.2-STABLE To: GomoR , freebsd-stable@freebsd.org References: From: John Baldwin Message-ID: <9c56bfda-725c-9c2a-9db3-4599abcfeaa0@FreeBSD.org> Date: Thu, 4 Feb 2021 10:33:45 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2021 18:33:47 -0000 On 2/4/21 8:08 AM, GomoR wrote: > Dear FreeBSD community, > > we are encountering a DoS condition on our production machines. > Our use case is an Nginx reverse proxy serving large files via HTTPS. > This problem arose when switching kernel and userland from 12.1-RELEASE > to 12.2-RELEASE. Ports were not upgraded (at first). > > Each time a user downloads a file, mbuf & mbuf_clusters are raising to > reach the maximum limit in a matter of seconds. Those values are > asserted by 'netstat -m' as follows: > > Normal situation: > > mbuf: 256, 26031105, 16767, 5974,428087938, 0, > 0 > mbuf_cluster: 2048, 8135232, 18408, 2704,101644203, 0, > 0 > > Warning situtation: > > mbuf: 256, 26031105, 2981516, 151205,1109483561, 0, > 0 > mbuf_cluster: 2048, 8135232, 2983155, 4201,319714617, 0, > 0 > > We have seen a patch related to sendfile + KTLS + mbuf at the below link > and we updated to -STABLE to apply: None of the sendfile or KTLS changes from Netflix are in 12, they are only in 13 and later. > Don't transmit mbufs that aren't yet ready on TOE sockets. > This includes mbufs waiting for data from sendfile() I/O requests, or > mbufs awaiting encryption for KTLS. > https://github.com/freebsd/freebsd-src/commit/14c77f30b201bf76119d59678e72051c093333c2 This patch only applies to Chelsio T5/T6 NICs when using TOE (TCP offload) and doesn't affect freeing mbufs, it just fixes a race when the NIC could potentially send random garbage if it sends the mbuf before the scheduled disk I/O to populate it with data from disk has completed. > NIC is: > ix0: > > What can we do to help you find the root cause? The first step I would do if possible would be to bisect between the last known working version and the version that is known to be broken to determine which commit introduced the problem. One thing that could help here is to see if you can reproduce the problem using a 12.2 kernel on a 12.1 world + ports. If you can, then you can limit your bisecting to just building new kernels which will make that process quicker. You might also see if using a different NIC shows the same problem. If not, then it might point to a regression in the NIC driver (or perhaps in iflib as ix uses iflib I believe). -- John Baldwin From owner-freebsd-stable@freebsd.org Thu Feb 4 21:31:51 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EE2DD52FAC5 for ; Thu, 4 Feb 2021 21:31:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWsFM5Ptlz3CGy for ; Thu, 4 Feb 2021 21:31:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id B99F952FAC4; Thu, 4 Feb 2021 21:31:51 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B967852F9DB for ; Thu, 4 Feb 2021 21:31:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWsFM0Rhxz3CMj for ; Thu, 4 Feb 2021 21:31:50 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f180.google.com with SMTP id k25so5237608oik.13 for ; Thu, 04 Feb 2021 13:31:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Wlex4Bc8wKvZSIo5XK+1ry7pCnUYiMAvVeyM4Qhm8jw=; b=emaEbxYODuDfMRisPn5H6tzR8MsEKw/kf2eR7+CPrrlsymD2OYc6Dix5w0OBMbtsBr iBwiwjezw/MoWxRm+4ObBBcbikv0mXn2pfnMMfGbAd+5xIXzML4acfkq6rmLJOK1Rakd BZfFAk2ugX+qm/6LZDOdWZPPSrXDgUAtMHrHk49K/8A6ux0JwnhnD0r90ByM156kVKVq wcumjzmeewrrsDLuuKiKjAEuCKckyd/gIhysLyCJUTespoUpFJSjXZ2JusgincanRati QbEZk/K5u91enXyOMyyThkdMyWLimQb3HXTHCBorVZbpxBBniQbSVqADDUdX3eOIxNzb Xzzw== X-Gm-Message-State: AOAM531ExU6I95h3VN21ka8Bu/Ld15DZj/xJrgCiOEzQ+0fbQtCZ3BR+ Uo3DMWVE+s76xV616rQ9SbZhf65r3mkbJizjoeNdRUT0n147Pg== X-Google-Smtp-Source: ABdhPJxGsGzT0rDT8NuBNni2oJn+L/1zxQGB/FpJJiF6DeOdBflEunQOlMMHTFpBIqsh5TV+kJhsjwY9J9QR+OxZMGc= X-Received: by 2002:a54:4813:: with SMTP id j19mr1009832oij.73.1612474309613; Thu, 04 Feb 2021 13:31:49 -0800 (PST) MIME-Version: 1.0 From: Alan Somers Date: Thu, 4 Feb 2021 14:31:38 -0700 Message-ID: Subject: Page fault in _mca_init during startup To: FreeBSD Stable ML X-Rspamd-Queue-Id: 4DWsFM0Rhxz3CMj X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.180 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-2.99 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.989]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.167.180:from]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_DOM_EQ_FROM_DOM(0.00)[]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[209.85.167.180:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.180:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.180:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2021 21:31:52 -0000 After upgrading a machine to FreeBSD, 12.2, it hit the following panic on its first reboot. I suspect that a few other servers have hit this too, but since it happens before swap is mounted there are no core dumps, and they usually reboot immediately. The code in question hasn't changed since 2018. The panic happened in cmci_monitor at line 930. Does anybody have any suggestions for how I could debug further? I can't readily reproduce it, and I can't dump core, but I'd like to investigate it any way I can. The server in question has dual Xeon Gold 6142 CPUs. if (!(ctl & MC_CTL2_CMCI_EN)) /* This bank does not support CMCI. */ return; cc = &cmc_state[PCPU_GET(cpuid)][i]; // <- panic here /* Determine maximum threshold. */ Fatal trap 12: page fault while in kernel mode cpuid = 26; apic id = 34 fault virtual address = 0xd0 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8125a009 stack pointer = 0x28:0xfffffe0000b65f20 frame pointer = 0x28:0xfffffe0000b65f50 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 11 (idle: cpu26) trap number = 12 panic: page fault cpuid = 26 time = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0000b65be0 vpanic() at vpanic+0x17b/frame 0xfffffe0000b65c30 panic() at panic+0x43/frame 0xfffffe0000b65c90 trap_fatal() at trap_fatal+0x391/frame 0xfffffe0000b65cf0 trap_pfault() at trap_pfault+0x4f/frame 0xfffffe0000b65d40 trap() at trap+0x286/frame 0xfffffe0000b65e50 calltrap() at calltrap+0x8/frame 0xfffffe0000b65e50 --- trap 0xc, rip = 0xffffffff8125a009, rsp = 0xfffffe0000b65f20, rbp = 0xfffffe0000b65f50 --- _mca_init() at _mca_init+0x5d9/frame 0xfffffe0000b65f50 init_secondary_tail() at init_secondary_tail+0xfd/frame 0xfffffe0000b65f80 init_secondary() at init_secondary+0x2d1/frame 0xfffffe0000b65ff0 KDB: enter: panic [ thread pid 11 tid 100029 ] Stopped at kdb_enter+0x37: movq $0,0x12bc1f6(%rip) From owner-freebsd-stable@freebsd.org Thu Feb 4 21:34:31 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B5E7452FBF8 for ; Thu, 4 Feb 2021 21:34:31 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWsJR4V3Tz3Ccm for ; Thu, 4 Feb 2021 21:34:31 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 5A6DD52FC63; Thu, 4 Feb 2021 21:34:30 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 53AA952FC62 for ; Thu, 4 Feb 2021 21:34:30 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWsJP4PWgz3ClG; Thu, 4 Feb 2021 21:34:29 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: mmacy) by smtp.freebsd.org (Postfix) with ESMTPSA id BE271BD9B; Thu, 4 Feb 2021 21:34:26 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mail-lj1-f177.google.com with SMTP id e18so5185000lja.12; Thu, 04 Feb 2021 13:34:26 -0800 (PST) X-Gm-Message-State: AOAM530ZQ7dPVKn+GAWvYHXFy5EDy9b5w4gv0r5CijPeBZLhdLDl4tqu uGSRfTYsDWhFD0gr1y31zzG6ZlQ7yp5SIUxgRY4= X-Google-Smtp-Source: ABdhPJy5iIfsdmRD1ZjZE/HiA0TPsQPZvfda7uaEPchih9r2rWgYBCCq9D6F1xfsVrdheX46+qVpm3lZbYdl1Ae18KE= X-Received: by 2002:a2e:9f4e:: with SMTP id v14mr743246ljk.497.1612474464684; Thu, 04 Feb 2021 13:34:24 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Macy Date: Thu, 4 Feb 2021 13:34:13 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Page fault in _mca_init during startup To: Alan Somers Cc: FreeBSD Stable ML Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2021 21:34:31 -0000 On Thu, Feb 4, 2021 at 1:31 PM Alan Somers wrote: > > After upgrading a machine to FreeBSD, 12.2, it hit the following panic on > its first reboot. I suspect that a few other servers have hit this too, > but since it happens before swap is mounted there are no core dumps, and > they usually reboot immediately. The code in question hasn't changed since > 2018. The panic happened in cmci_monitor at line 930. Does anybody have > any suggestions for how I could debug further? I can't readily reproduce > it, and I can't dump core, but I'd like to investigate it any way I can. > The server in question has dual Xeon Gold 6142 CPUs. > I can't actually help :( but I can add a +1 with similar hardware or equivalent specs. It's not frequent, but it's often enough to be annoying. -M > if (!(ctl & MC_CTL2_CMCI_EN)) > /* This bank does not support CMCI. */ > return; > > cc = &cmc_state[PCPU_GET(cpuid)][i]; // <- panic here > > /* Determine maximum threshold. */ > > > Fatal trap 12: page fault while in kernel mode > cpuid = 26; apic id = 34 > fault virtual address = 0xd0 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff8125a009 > stack pointer = 0x28:0xfffffe0000b65f20 > frame pointer = 0x28:0xfffffe0000b65f50 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = resume, IOPL = 0 > current process = 11 (idle: cpu26) > trap number = 12 > panic: page fault > cpuid = 26 > time = 1 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe0000b65be0 > vpanic() at vpanic+0x17b/frame 0xfffffe0000b65c30 > panic() at panic+0x43/frame 0xfffffe0000b65c90 > trap_fatal() at trap_fatal+0x391/frame 0xfffffe0000b65cf0 > trap_pfault() at trap_pfault+0x4f/frame 0xfffffe0000b65d40 > trap() at trap+0x286/frame 0xfffffe0000b65e50 > calltrap() at calltrap+0x8/frame 0xfffffe0000b65e50 > --- trap 0xc, rip = 0xffffffff8125a009, rsp = 0xfffffe0000b65f20, rbp = > 0xfffffe0000b65f50 --- > _mca_init() at _mca_init+0x5d9/frame 0xfffffe0000b65f50 > init_secondary_tail() at init_secondary_tail+0xfd/frame 0xfffffe0000b65f80 > init_secondary() at init_secondary+0x2d1/frame 0xfffffe0000b65ff0 > KDB: enter: panic > [ thread pid 11 tid 100029 ] > Stopped at kdb_enter+0x37: movq $0,0x12bc1f6(%rip) > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu Feb 4 22:58:49 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 38BEF53240C for ; Thu, 4 Feb 2021 22:58:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWv9j0LJHz3KRd for ; Thu, 4 Feb 2021 22:58:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 097525322AF; Thu, 4 Feb 2021 22:58:49 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 093AA532240 for ; Thu, 4 Feb 2021 22:58:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWv9h3lZlz3KDQ; Thu, 4 Feb 2021 22:58:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 114MwYFk030871 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Feb 2021 00:58:37 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 114MwYFk030871 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 114MwYlv030870; Fri, 5 Feb 2021 00:58:34 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Feb 2021 00:58:34 +0200 From: Konstantin Belousov To: Matthew Macy Cc: Alan Somers , FreeBSD Stable ML Subject: Re: Page fault in _mca_init during startup Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DWv9h3lZlz3KDQ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2021 22:58:49 -0000 On Thu, Feb 04, 2021 at 01:34:13PM -0800, Matthew Macy wrote: > On Thu, Feb 4, 2021 at 1:31 PM Alan Somers wrote: > > > > After upgrading a machine to FreeBSD, 12.2, it hit the following panic on > > its first reboot. I suspect that a few other servers have hit this too, > > but since it happens before swap is mounted there are no core dumps, and > > they usually reboot immediately. The code in question hasn't changed since > > 2018. The panic happened in cmci_monitor at line 930. Does anybody have > > any suggestions for how I could debug further? I can't readily reproduce > > it, and I can't dump core, but I'd like to investigate it any way I can. > > The server in question has dual Xeon Gold 6142 CPUs. > > > > I can't actually help :( but I can add a +1 with similar hardware or > equivalent specs. It's not frequent, but it's often enough to be > annoying. > -M > > > if (!(ctl & MC_CTL2_CMCI_EN)) > > /* This bank does not support CMCI. */ > > return; > > > > cc = &cmc_state[PCPU_GET(cpuid)][i]; // <- panic here > > > > /* Determine maximum threshold. */ > > > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 26; apic id = 34 > > fault virtual address = 0xd0 > > fault code = supervisor read data, page not present > > instruction pointer = 0x20:0xffffffff8125a009 > > stack pointer = 0x28:0xfffffe0000b65f20 > > frame pointer = 0x28:0xfffffe0000b65f50 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags = resume, IOPL = 0 > > current process = 11 (idle: cpu26) > > trap number = 12 > > panic: page fault > > cpuid = 26 > > time = 1 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > 0xfffffe0000b65be0 > > vpanic() at vpanic+0x17b/frame 0xfffffe0000b65c30 > > panic() at panic+0x43/frame 0xfffffe0000b65c90 > > trap_fatal() at trap_fatal+0x391/frame 0xfffffe0000b65cf0 > > trap_pfault() at trap_pfault+0x4f/frame 0xfffffe0000b65d40 > > trap() at trap+0x286/frame 0xfffffe0000b65e50 > > calltrap() at calltrap+0x8/frame 0xfffffe0000b65e50 > > --- trap 0xc, rip = 0xffffffff8125a009, rsp = 0xfffffe0000b65f20, rbp = > > 0xfffffe0000b65f50 --- > > _mca_init() at _mca_init+0x5d9/frame 0xfffffe0000b65f50 > > init_secondary_tail() at init_secondary_tail+0xfd/frame 0xfffffe0000b65f80 > > init_secondary() at init_secondary+0x2d1/frame 0xfffffe0000b65ff0 > > KDB: enter: panic > > [ thread pid 11 tid 100029 ] > > Stopped at kdb_enter+0x37: movq $0,0x12bc1f6(%rip) Try this. I think that there is no other dependencies in the startup order, but cannot know it for sure. commit 19584e3d3e9606d591fa30999b370ed758960e8c Author: Konstantin Belousov Date: Fri Feb 5 00:56:09 2021 +0200 x86: init mca before APs are started diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c index 03100e77d455..e2bf2673cf69 100644 --- a/sys/x86/x86/mca.c +++ b/sys/x86/x86/mca.c @@ -1371,7 +1371,7 @@ mca_init_bsp(void *arg __unused) mca_init(); } -SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_ANY, mca_init_bsp, NULL); +SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_SECOND, mca_init_bsp, NULL); /* Called when a machine check exception fires. */ void From owner-freebsd-stable@freebsd.org Thu Feb 4 23:05:54 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AF4635325BD for ; Thu, 4 Feb 2021 23:05:54 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWvKt3lshz3KtQ for ; Thu, 4 Feb 2021 23:05:54 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 80E1B5325BC; Thu, 4 Feb 2021 23:05:54 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 80AB2532654 for ; Thu, 4 Feb 2021 23:05:54 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWvKt3BMBz3LFf; Thu, 4 Feb 2021 23:05:54 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f170.google.com with SMTP id d20so5505751oiw.10; Thu, 04 Feb 2021 15:05:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZThsNgRXoBd4UaDevIuYSx1e7iptUw5KTSSSvAuvSSg=; b=I7CXPTrSATqK+xnDCfmpjUPjFU3P8JBRRbkYBkl9gFaQN0GmegWBShmM7o5Mi0vU6M 3iOxuYExhSeYq9yYVlkZ2QcvjmlFb5A4/73OZR5qDoHX/C28mvcYXXTF6y5jaZkhZONb cKwrlTMHu5fKnyHTHMdDjbSyp8DERuZQ2BjnrEkByxjBmWkXdnKguxGpZHp9F3PL6i+r lQAt8KUTq20T8GMrvDlzW1Wv0ATqH5h+874SQQLnZx2X5H79uCkZjMgW8EHiYfzuDMk2 EjFEF/23RZm/K+D3MjSJ3gtbc4nbYmeOGjjAXPQodxhA0duENku+zZ7gsgl3Z68fjIpB /lPw== X-Gm-Message-State: AOAM53385Q2cg1TqSBrzeHJzVa0A/3NwWlU/Zh7MB5I0zutQvic0lzOR VcN8rNfRzOCP7E2FKt/73jm5+2esnGrESeDgrpDRq4ESm4g= X-Google-Smtp-Source: ABdhPJzI2a6VDGKGzkaaOfAW4eDwI7vo6AicwCGY6CUptj0v8paGz9CiMe2baq4Xwi3DELZ9KHDFGjcWMQpK1zJQ7QM= X-Received: by 2002:aca:cd06:: with SMTP id d6mr1218249oig.57.1612479953446; Thu, 04 Feb 2021 15:05:53 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Thu, 4 Feb 2021 16:05:42 -0700 Message-ID: Subject: Re: Page fault in _mca_init during startup To: Konstantin Belousov Cc: Matthew Macy , FreeBSD Stable ML X-Rspamd-Queue-Id: 4DWvKt3BMBz3LFf X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2021 23:05:54 -0000 On Thu, Feb 4, 2021 at 3:58 PM Konstantin Belousov wrote: > On Thu, Feb 04, 2021 at 01:34:13PM -0800, Matthew Macy wrote: > > On Thu, Feb 4, 2021 at 1:31 PM Alan Somers wrote: > > > > > > After upgrading a machine to FreeBSD, 12.2, it hit the following panic > on > > > its first reboot. I suspect that a few other servers have hit this > too, > > > but since it happens before swap is mounted there are no core dumps, > and > > > they usually reboot immediately. The code in question hasn't changed > since > > > 2018. The panic happened in cmci_monitor at line 930. Does anybody > have > > > any suggestions for how I could debug further? I can't readily > reproduce > > > it, and I can't dump core, but I'd like to investigate it any way I > can. > > > The server in question has dual Xeon Gold 6142 CPUs. > > > > > > > I can't actually help :( but I can add a +1 with similar hardware or > > equivalent specs. It's not frequent, but it's often enough to be > > annoying. > > -M > > > > > if (!(ctl & MC_CTL2_CMCI_EN)) > > > /* This bank does not support CMCI. */ > > > return; > > > > > > cc = &cmc_state[PCPU_GET(cpuid)][i]; // <- panic here > > > > > > /* Determine maximum threshold. */ > > > > > > > > > Fatal trap 12: page fault while in kernel mode > > > cpuid = 26; apic id = 34 > > > fault virtual address = 0xd0 > > > fault code = supervisor read data, page not present > > > instruction pointer = 0x20:0xffffffff8125a009 > > > stack pointer = 0x28:0xfffffe0000b65f20 > > > frame pointer = 0x28:0xfffffe0000b65f50 > > > code segment = base 0x0, limit 0xfffff, type 0x1b > > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > > processor eflags = resume, IOPL = 0 > > > current process = 11 (idle: cpu26) > > > trap number = 12 > > > panic: page fault > > > cpuid = 26 > > > time = 1 > > > KDB: stack backtrace: > > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > > 0xfffffe0000b65be0 > > > vpanic() at vpanic+0x17b/frame 0xfffffe0000b65c30 > > > panic() at panic+0x43/frame 0xfffffe0000b65c90 > > > trap_fatal() at trap_fatal+0x391/frame 0xfffffe0000b65cf0 > > > trap_pfault() at trap_pfault+0x4f/frame 0xfffffe0000b65d40 > > > trap() at trap+0x286/frame 0xfffffe0000b65e50 > > > calltrap() at calltrap+0x8/frame 0xfffffe0000b65e50 > > > --- trap 0xc, rip = 0xffffffff8125a009, rsp = 0xfffffe0000b65f20, rbp = > > > 0xfffffe0000b65f50 --- > > > _mca_init() at _mca_init+0x5d9/frame 0xfffffe0000b65f50 > > > init_secondary_tail() at init_secondary_tail+0xfd/frame > 0xfffffe0000b65f80 > > > init_secondary() at init_secondary+0x2d1/frame 0xfffffe0000b65ff0 > > > KDB: enter: panic > > > [ thread pid 11 tid 100029 ] > > > Stopped at kdb_enter+0x37: movq $0,0x12bc1f6(%rip) > > Try this. > > I think that there is no other dependencies in the startup order, but > cannot know it for sure. > > commit 19584e3d3e9606d591fa30999b370ed758960e8c > Author: Konstantin Belousov > Date: Fri Feb 5 00:56:09 2021 +0200 > > x86: init mca before APs are started > > diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c > index 03100e77d455..e2bf2673cf69 100644 > --- a/sys/x86/x86/mca.c > +++ b/sys/x86/x86/mca.c > @@ -1371,7 +1371,7 @@ mca_init_bsp(void *arg __unused) > > mca_init(); > } > -SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_ANY, mca_init_bsp, NULL); > +SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_SECOND, mca_init_bsp, NULL); > > /* Called when a machine check exception fires. */ > void > I can test this patch on development servers, but so far I've only seen the crash on production servers. Do you have any suggestions for how to force the crash, or how to test this patch besides simply making sure that my dev servers can boot? -Alan From owner-freebsd-stable@freebsd.org Thu Feb 4 23:16:44 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 76163532D37 for ; Thu, 4 Feb 2021 23:16:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWvZN2BRHz3M8W for ; Thu, 4 Feb 2021 23:16:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 4AD5E532E8C; Thu, 4 Feb 2021 23:16:44 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4A992532DD4 for ; Thu, 4 Feb 2021 23:16:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWvZM60VKz3LgZ; Thu, 4 Feb 2021 23:16:43 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 114NGaBl035567 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Feb 2021 01:16:39 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 114NGaBl035567 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 114NGZHV035566; Fri, 5 Feb 2021 01:16:35 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Feb 2021 01:16:35 +0200 From: Konstantin Belousov To: Alan Somers Cc: Matthew Macy , FreeBSD Stable ML Subject: Re: Page fault in _mca_init during startup Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DWvZM60VKz3LgZ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2021 23:16:44 -0000 On Thu, Feb 04, 2021 at 04:05:42PM -0700, Alan Somers wrote: > On Thu, Feb 4, 2021 at 3:58 PM Konstantin Belousov > wrote: > > > On Thu, Feb 04, 2021 at 01:34:13PM -0800, Matthew Macy wrote: > > > On Thu, Feb 4, 2021 at 1:31 PM Alan Somers wrote: > > > > > > > > After upgrading a machine to FreeBSD, 12.2, it hit the following panic > > on > > > > its first reboot. I suspect that a few other servers have hit this > > too, > > > > but since it happens before swap is mounted there are no core dumps, > > and > > > > they usually reboot immediately. The code in question hasn't changed > > since > > > > 2018. The panic happened in cmci_monitor at line 930. Does anybody > > have > > > > any suggestions for how I could debug further? I can't readily > > reproduce > > > > it, and I can't dump core, but I'd like to investigate it any way I > > can. > > > > The server in question has dual Xeon Gold 6142 CPUs. > > > > > > > > > > I can't actually help :( but I can add a +1 with similar hardware or > > > equivalent specs. It's not frequent, but it's often enough to be > > > annoying. > > > -M > > > > > > > if (!(ctl & MC_CTL2_CMCI_EN)) > > > > /* This bank does not support CMCI. */ > > > > return; > > > > > > > > cc = &cmc_state[PCPU_GET(cpuid)][i]; // <- panic here > > > > > > > > /* Determine maximum threshold. */ > > > > > > > > > > > > Fatal trap 12: page fault while in kernel mode > > > > cpuid = 26; apic id = 34 > > > > fault virtual address = 0xd0 > > > > fault code = supervisor read data, page not present > > > > instruction pointer = 0x20:0xffffffff8125a009 > > > > stack pointer = 0x28:0xfffffe0000b65f20 > > > > frame pointer = 0x28:0xfffffe0000b65f50 > > > > code segment = base 0x0, limit 0xfffff, type 0x1b > > > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > > > processor eflags = resume, IOPL = 0 > > > > current process = 11 (idle: cpu26) > > > > trap number = 12 > > > > panic: page fault > > > > cpuid = 26 > > > > time = 1 > > > > KDB: stack backtrace: > > > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > > > 0xfffffe0000b65be0 > > > > vpanic() at vpanic+0x17b/frame 0xfffffe0000b65c30 > > > > panic() at panic+0x43/frame 0xfffffe0000b65c90 > > > > trap_fatal() at trap_fatal+0x391/frame 0xfffffe0000b65cf0 > > > > trap_pfault() at trap_pfault+0x4f/frame 0xfffffe0000b65d40 > > > > trap() at trap+0x286/frame 0xfffffe0000b65e50 > > > > calltrap() at calltrap+0x8/frame 0xfffffe0000b65e50 > > > > --- trap 0xc, rip = 0xffffffff8125a009, rsp = 0xfffffe0000b65f20, rbp = > > > > 0xfffffe0000b65f50 --- > > > > _mca_init() at _mca_init+0x5d9/frame 0xfffffe0000b65f50 > > > > init_secondary_tail() at init_secondary_tail+0xfd/frame > > 0xfffffe0000b65f80 > > > > init_secondary() at init_secondary+0x2d1/frame 0xfffffe0000b65ff0 > > > > KDB: enter: panic > > > > [ thread pid 11 tid 100029 ] > > > > Stopped at kdb_enter+0x37: movq $0,0x12bc1f6(%rip) > > > > Try this. > > > > I think that there is no other dependencies in the startup order, but > > cannot know it for sure. > > > > commit 19584e3d3e9606d591fa30999b370ed758960e8c > > Author: Konstantin Belousov > > Date: Fri Feb 5 00:56:09 2021 +0200 > > > > x86: init mca before APs are started > > > > diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c > > index 03100e77d455..e2bf2673cf69 100644 > > --- a/sys/x86/x86/mca.c > > +++ b/sys/x86/x86/mca.c > > @@ -1371,7 +1371,7 @@ mca_init_bsp(void *arg __unused) > > > > mca_init(); > > } > > -SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_ANY, mca_init_bsp, NULL); > > +SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_SECOND, mca_init_bsp, NULL); > > > > /* Called when a machine check exception fires. */ > > void > > > > I can test this patch on development servers, but so far I've only seen the > crash on production servers. Do you have any suggestions for how to force > the crash, or how to test this patch besides simply making sure that my dev > servers can boot? The race, as I see it, is that we call mca_init() on BSP too late, so malloc() that provides the storage for cmc_state array, could be called too late, before one of the APs was IPIed for startup. Patch ensures that mca_init_bsp() SYSINIT is finished before we go to start the APs. I do not think there is any reliable way to trigger the panic while keeping the patch usable, except to observe enough successfull boots. From owner-freebsd-stable@freebsd.org Thu Feb 4 23:27:20 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D6E08532FEC for ; Thu, 4 Feb 2021 23:27:20 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWvpc4lBcz3MSj for ; Thu, 4 Feb 2021 23:27:20 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id A296A53313A; Thu, 4 Feb 2021 23:27:20 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A25D6532F4C for ; Thu, 4 Feb 2021 23:27:20 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWvpc3y35z3MVS; Thu, 4 Feb 2021 23:27:20 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x735.google.com with SMTP id x81so5240996qkb.0; Thu, 04 Feb 2021 15:27:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=IkZHq5LeeTpUk3RKXqrzcSr4lHDdm6ouVzCw46HszDE=; b=b0UJZLnKomJhNXxlfJiDHPMUrmfQDee9/optOAPZ0+eYUKchRCpJA/ymD27pAFivNy SzWO6UgZEcFAxgTHBMIvwFCpnE0pgY/GGUKC/p9sHzOQw/AYDtFH/iEf3Lj/+fY4y54b QYEMGg6ooWWifNOuHx4DMx6/1SPuk2WJMhzDqkvm14J1mJb/9wPlfS4bJwobv4RZBm6d jU7XZGNN6CtCbjMuB3vzxJsE70P6B9WxCmb7ywpBBkLaN7PYjVun6+bjvCNJxv9Uz2G/ MEbXbYFtIyS6P79WYy6QlzrDvfAl9XDEO8gKn4huCldQ0alRHmddGPaQnR74v152I6uV RphA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=IkZHq5LeeTpUk3RKXqrzcSr4lHDdm6ouVzCw46HszDE=; b=ApGilaMbVz0YP3/03pjJmLhtK7qogw7+XLqG8n+qN4WWwrp9U0q8bOxybNJ1HgnzNA Y9OIcefU6V8VrTzaUlR0amFItHR/eCwcqnrKIMrz/5XLwSBop/BNSqPxs3BYdOim6VRg wpJIDiHPWh/UXFLQNbbWN6IlGt8GAFs3R3DRkFTOcka1MYq/Sqb5jMTcX3PoCYzKOP49 zPq4hn+/wzaOPlRjXuD/rDi1OaxnKRmYLI6los/1iY04FJf80bShoDllihtFR7FWClTF samdxXrgARlm8Z4ReUUul74ZVcs3qGYpveyXP8bvAf9cs7sB2a7U8yr4Y3xNvz2ejEjk cfCQ== X-Gm-Message-State: AOAM532JQJc2N5zLK732lYWzrVl78vROSxfi53y+2GbXih3f5sqoZkFp mzsFhss1Y7TF3KbLMvwL3i8= X-Google-Smtp-Source: ABdhPJzYuFKxe0Nn86MvlmEybSD+go0UhFS2mbHwqAyVuAtfOluDXYJu1KzorhJio/uRlQlixkAR0g== X-Received: by 2002:a37:aa90:: with SMTP id t138mr1664651qke.150.1612481239808; Thu, 04 Feb 2021 15:27:19 -0800 (PST) Received: from raichu ([142.126.164.150]) by smtp.gmail.com with ESMTPSA id s23sm6772948qke.3.2021.02.04.15.27.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 15:27:19 -0800 (PST) Sender: Mark Johnston Date: Thu, 4 Feb 2021 18:27:17 -0500 From: Mark Johnston To: Konstantin Belousov Cc: Matthew Macy , Alan Somers , FreeBSD Stable ML Subject: Re: Page fault in _mca_init during startup Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4DWvpc3y35z3MVS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2021 23:27:20 -0000 On Fri, Feb 05, 2021 at 12:58:34AM +0200, Konstantin Belousov wrote: > On Thu, Feb 04, 2021 at 01:34:13PM -0800, Matthew Macy wrote: > > On Thu, Feb 4, 2021 at 1:31 PM Alan Somers wrote: > > > > > > After upgrading a machine to FreeBSD, 12.2, it hit the following panic on > > > its first reboot. I suspect that a few other servers have hit this too, > > > but since it happens before swap is mounted there are no core dumps, and > > > they usually reboot immediately. The code in question hasn't changed since > > > 2018. The panic happened in cmci_monitor at line 930. Does anybody have > > > any suggestions for how I could debug further? I can't readily reproduce > > > it, and I can't dump core, but I'd like to investigate it any way I can. > > > The server in question has dual Xeon Gold 6142 CPUs. > > > > Try this. > > I think that there is no other dependencies in the startup order, but > cannot know it for sure. > > commit 19584e3d3e9606d591fa30999b370ed758960e8c > Author: Konstantin Belousov > Date: Fri Feb 5 00:56:09 2021 +0200 > > x86: init mca before APs are started APs only call mca_init() after they have been released by the BSP though, and that happens later in SI_SUB_SMP. > diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c > index 03100e77d455..e2bf2673cf69 100644 > --- a/sys/x86/x86/mca.c > +++ b/sys/x86/x86/mca.c > @@ -1371,7 +1371,7 @@ mca_init_bsp(void *arg __unused) > > mca_init(); > } > -SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_ANY, mca_init_bsp, NULL); > +SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_SECOND, mca_init_bsp, NULL); > > /* Called when a machine check exception fires. */ > void > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Fri Feb 5 00:19:56 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B22E4534B68 for ; Fri, 5 Feb 2021 00:19:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWwzJ3pLlz3RG2 for ; Fri, 5 Feb 2021 00:19:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 8073E534EC2; Fri, 5 Feb 2021 00:19:56 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7F0D853500F for ; Fri, 5 Feb 2021 00:19:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f169.google.com (mail-oi1-f169.google.com [209.85.167.169]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWwzJ2wG3z3hPD; Fri, 5 Feb 2021 00:19:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f169.google.com with SMTP id v193so287224oie.8; Thu, 04 Feb 2021 16:19:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zCPs58WalTYkW7haZNjP+PcYbYMNed3nPPRe+7eBMak=; b=Tt0BQ7mFvuZtawnT2jT8bIkY5D5LoXbOGFzMH6kjw5LdLh1AibACXo2FXo28CsZFUy lLlG8K2Ksora6FKNwLw1XTwKZFsYsxqAZmiMCSwfmdtXJzPjemnkn3hRNy/7PouhSEXW LafP9AQ89MtMemi+gyFFxr9xbo+Rat2D8hI+la0hEzxMwxJhJe6vXProNExEYZory56V lg+3p13SdI8T+rOfKakdBzWsv+uY5JNlOZxmd8RlW+LwOQfmRR4P93iOrGpo9eb3z+jI meQjh9z9ashr90GY8fYw+r+Qo0QvHkDOfB5pY9L98IFkzBqGWMS+kNILM9CaVPveu75O gxJA== X-Gm-Message-State: AOAM530ldC2pWa1ZLuNof7dX+8Qss7Npp+Y/PQl8UGq/ICRv7OxzUFDo h9JYUZZufwT14HPeuPc7U5oRHejPxoPOXLTCfmDCZwEpoCLcjQ== X-Google-Smtp-Source: ABdhPJz114MqaI8Z/O0agu+o0E9B+jUPHvP+tEIoa7WTFjvDAsAAA8Cg283LF+9RKV4CQlOoTuiMb7X28EThOUtZ5o8= X-Received: by 2002:aca:dd08:: with SMTP id u8mr1451696oig.55.1612484395141; Thu, 04 Feb 2021 16:19:55 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Thu, 4 Feb 2021 17:19:43 -0700 Message-ID: Subject: Re: Page fault in _mca_init during startup To: Mark Johnston Cc: Konstantin Belousov , Matthew Macy , FreeBSD Stable ML X-Rspamd-Queue-Id: 4DWwzJ2wG3z3hPD X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 00:19:56 -0000 On Thu, Feb 4, 2021 at 4:27 PM Mark Johnston wrote: > On Fri, Feb 05, 2021 at 12:58:34AM +0200, Konstantin Belousov wrote: > > On Thu, Feb 04, 2021 at 01:34:13PM -0800, Matthew Macy wrote: > > > On Thu, Feb 4, 2021 at 1:31 PM Alan Somers > wrote: > > > > > > > > After upgrading a machine to FreeBSD, 12.2, it hit the following > panic on > > > > its first reboot. I suspect that a few other servers have hit this > too, > > > > but since it happens before swap is mounted there are no core dumps, > and > > > > they usually reboot immediately. The code in question hasn't > changed since > > > > 2018. The panic happened in cmci_monitor at line 930. Does anybody > have > > > > any suggestions for how I could debug further? I can't readily > reproduce > > > > it, and I can't dump core, but I'd like to investigate it any way I > can. > > > > The server in question has dual Xeon Gold 6142 CPUs. > > > > > > Try this. > > > > I think that there is no other dependencies in the startup order, but > > cannot know it for sure. > > > > commit 19584e3d3e9606d591fa30999b370ed758960e8c > > Author: Konstantin Belousov > > Date: Fri Feb 5 00:56:09 2021 +0200 > > > > x86: init mca before APs are started > > APs only call mca_init() after they have been released by the BSP > though, and that happens later in SI_SUB_SMP. > > > diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c > > index 03100e77d455..e2bf2673cf69 100644 > > --- a/sys/x86/x86/mca.c > > +++ b/sys/x86/x86/mca.c > > @@ -1371,7 +1371,7 @@ mca_init_bsp(void *arg __unused) > > > > mca_init(); > > } > > -SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_ANY, mca_init_bsp, NULL); > > +SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_SECOND, mca_init_bsp, NULL); > > > > /* Called when a machine check exception fires. */ > > void > kib's patch causes a different problem, and this one is reproducible: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x18 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8125762c stack pointer = 0x28:0xffffffff828dad90 frame pointer = 0x28:0xffffffff828dad90 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 0 () trap number = 12 panic: page fault cpuid = 0 time = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff828daa50 vpanic() at vpanic+0x17b/frame 0xffffffff828daaa0 panic() at panic+0x43/frame 0xffffffff828dab00 trap_fatal() at trap_fatal+0x391/frame 0xffffffff828dab60 trap_pfault() at trap_pfault+0x4f/frame 0xffffffff828dabb0 trap() at trap+0x286/frame 0xffffffff828dacc0 calltrap() at calltrap+0x8/frame 0xffffffff828dacc0 --- trap 0xc, rip = 0xffffffff8125762c, rsp = 0xffffffff828dad90, rbp = 0xffffffff828dad90 --- native_lapic_enable_cmc() at native_lapic_enable_cmc+0x1c/frame 0xffffffff828dad90 _mca_init() at _mca_init+0x94c/frame 0xffffffff828dadd0 mi_startup() at mi_startup+0xdf/frame 0xffffffff828dadf0 btext() at btext+0x2c KDB: enter: panic [ thread pid 0 tid 0 ] Stopped at kdb_enter+0x37: movq $0,0x12bc396(%rip) If you're wondering, the panic happens at this point in native_lapic_enable_cmc: apic_id = PCPU_GET(apic_id); KASSERT(lapics[apic_id].la_present, ("%s: missing APIC %u", __func__, apic_id)); lapics[apic_id].la_lvts[APIC_LVT_CMCI].lvt_masked = 0; <- panic here lapics[apic_id].la_lvts[APIC_LVT_CMCI].lvt_active = 1; if (bootverbose) printf("lapic%u: CMCI unmasked\n", apic_id); } From owner-freebsd-stable@freebsd.org Fri Feb 5 00:59:25 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4EDFA53754C for ; Fri, 5 Feb 2021 00:59:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWxrs0qDzz3lRk for ; Fri, 5 Feb 2021 00:59:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 1C1B753763C; Fri, 5 Feb 2021 00:59:25 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1BE7A5375D6 for ; Fri, 5 Feb 2021 00:59:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWxrr50MCz3lr4; Fri, 5 Feb 2021 00:59:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1150xGWq059341 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Feb 2021 02:59:19 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1150xGWq059341 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1150xGfm059340; Fri, 5 Feb 2021 02:59:16 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Feb 2021 02:59:16 +0200 From: Konstantin Belousov To: Alan Somers Cc: Mark Johnston , Matthew Macy , FreeBSD Stable ML Subject: Re: Page fault in _mca_init during startup Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DWxrr50MCz3lr4 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 00:59:25 -0000 On Thu, Feb 04, 2021 at 05:19:43PM -0700, Alan Somers wrote: > On Thu, Feb 4, 2021 at 4:27 PM Mark Johnston wrote: > > > On Fri, Feb 05, 2021 at 12:58:34AM +0200, Konstantin Belousov wrote: > > > On Thu, Feb 04, 2021 at 01:34:13PM -0800, Matthew Macy wrote: > > > > On Thu, Feb 4, 2021 at 1:31 PM Alan Somers > > wrote: > > > > > > > > > > After upgrading a machine to FreeBSD, 12.2, it hit the following > > panic on > > > > > its first reboot. I suspect that a few other servers have hit this > > too, > > > > > but since it happens before swap is mounted there are no core dumps, > > and > > > > > they usually reboot immediately. The code in question hasn't > > changed since > > > > > 2018. The panic happened in cmci_monitor at line 930. Does anybody > > have > > > > > any suggestions for how I could debug further? I can't readily > > reproduce > > > > > it, and I can't dump core, but I'd like to investigate it any way I > > can. > > > > > The server in question has dual Xeon Gold 6142 CPUs. > > > > > > > > Try this. > > > > > > I think that there is no other dependencies in the startup order, but > > > cannot know it for sure. > > > > > > commit 19584e3d3e9606d591fa30999b370ed758960e8c > > > Author: Konstantin Belousov > > > Date: Fri Feb 5 00:56:09 2021 +0200 > > > > > > x86: init mca before APs are started > > > > APs only call mca_init() after they have been released by the BSP > > though, and that happens later in SI_SUB_SMP. > > > > > diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c > > > index 03100e77d455..e2bf2673cf69 100644 > > > --- a/sys/x86/x86/mca.c > > > +++ b/sys/x86/x86/mca.c > > > @@ -1371,7 +1371,7 @@ mca_init_bsp(void *arg __unused) > > > > > > mca_init(); > > > } > > > -SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_ANY, mca_init_bsp, NULL); > > > +SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_SECOND, mca_init_bsp, NULL); > > > > > > /* Called when a machine check exception fires. */ > > > void > > > > kib's patch causes a different problem, and this one is reproducible: > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x18 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff8125762c > stack pointer = 0x28:0xffffffff828dad90 > frame pointer = 0x28:0xffffffff828dad90 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = resume, IOPL = 0 > current process = 0 () > trap number = 12 > panic: page fault > cpuid = 0 > time = 1 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xffffffff828daa50 > vpanic() at vpanic+0x17b/frame 0xffffffff828daaa0 > panic() at panic+0x43/frame 0xffffffff828dab00 > trap_fatal() at trap_fatal+0x391/frame 0xffffffff828dab60 > trap_pfault() at trap_pfault+0x4f/frame 0xffffffff828dabb0 > trap() at trap+0x286/frame 0xffffffff828dacc0 > calltrap() at calltrap+0x8/frame 0xffffffff828dacc0 > --- trap 0xc, rip = 0xffffffff8125762c, rsp = 0xffffffff828dad90, rbp = > 0xffffffff828dad90 --- > native_lapic_enable_cmc() at native_lapic_enable_cmc+0x1c/frame > 0xffffffff828dad90 > _mca_init() at _mca_init+0x94c/frame 0xffffffff828dadd0 > mi_startup() at mi_startup+0xdf/frame 0xffffffff828dadf0 > btext() at btext+0x2c > KDB: enter: panic > [ thread pid 0 tid 0 ] > Stopped at kdb_enter+0x37: movq $0,0x12bc396(%rip) > > If you're wondering, the panic happens at this point in > native_lapic_enable_cmc: > > apic_id = PCPU_GET(apic_id); > KASSERT(lapics[apic_id].la_present, > ("%s: missing APIC %u", __func__, apic_id)); > lapics[apic_id].la_lvts[APIC_LVT_CMCI].lvt_masked = 0; <- panic here > lapics[apic_id].la_lvts[APIC_LVT_CMCI].lvt_active = 1; > if (bootverbose) > printf("lapic%u: CMCI unmasked\n", apic_id); > } Scratch this patch. Do you have INVARIANTS enabled? If not, I am curious if enabling them would convert that rare page fault into rare "CPU %d has more MC banks" assert. Also might be the output of the # for x in $(jot $(sysctl -n hw.ncpu) 0) ; do cpucontrol -m 0x179 /dev/cpuctl$x; done command will show the issue (0x179 is the MCG_CAP MSR). You need to load cpuctl(4) if it is not loaded yet. From owner-freebsd-stable@freebsd.org Fri Feb 5 02:01:42 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BF7C0538C63 for ; Fri, 5 Feb 2021 02:01:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DWzDk44Wrz3q6H for ; Fri, 5 Feb 2021 02:01:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 897F6538F53; Fri, 5 Feb 2021 02:01:42 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 89458538ECC for ; Fri, 5 Feb 2021 02:01:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWzDk3PMCz3py5; Fri, 5 Feb 2021 02:01:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f44.google.com with SMTP id y11so5494604otq.1; Thu, 04 Feb 2021 18:01:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=GrOra/F0s0LWNRQxeDb1xXnZL0wNHRpphqVhyunagFU=; b=rF1t0xi9C/Xn7XKmd3/3iuVhOljkVVRJ8AsiEJvnz91PVM9OnHce1KWPh/U2fL5Sbb WWLo9KqCqrIkiBAXZ6tET9ixPc0MtQQ3gOmlNRUb91dJ+lu5bedC2ASIbj5MJqjIDNGM 8BllG+wsR/N+2oOO9idFSEfZoWskeDta1TP9EW0Is2Zv7wrXlUdRSVU5GAb+JSup2GgX Jar6YzJgm6ZRADZ7U8oz0DtolicK4S7Qs1NZvZN/QVfr0EYSV+YXdhevKnDliUpDglzN FzcoUoYS54hL0oYFE/5cw1oiLkUwxb8+qp8NXuRwjF6HnvfAvOLZA1mlttZ4IhtyrM6l 2+6w== X-Gm-Message-State: AOAM531cHkPRnP5HAK6CtzXDVrQc616cUWWx/YZ2xmbcX4VlPzl7SHaj yXV1659YIhwjBRuOAhuvINbWCVBgpJdLsrpi32c= X-Google-Smtp-Source: ABdhPJwvMAKiD50VHMFHVNNMHym59OwKz0zzX9+xBNdzZ3EsvFuF4SiPtdN16DODTKpNWM5Kg2UhUZ4JZ2Z56rMmaUs= X-Received: by 2002:a9d:2925:: with SMTP id d34mr1605886otb.291.1612490501378; Thu, 04 Feb 2021 18:01:41 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Thu, 4 Feb 2021 19:01:30 -0700 Message-ID: Subject: Re: Page fault in _mca_init during startup To: Konstantin Belousov Cc: Mark Johnston , Matthew Macy , FreeBSD Stable ML X-Rspamd-Queue-Id: 4DWzDk3PMCz3py5 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 02:01:42 -0000 On Thu, Feb 4, 2021 at 5:59 PM Konstantin Belousov wrote: > On Thu, Feb 04, 2021 at 05:19:43PM -0700, Alan Somers wrote: > > On Thu, Feb 4, 2021 at 4:27 PM Mark Johnston wrote: > > > > > On Fri, Feb 05, 2021 at 12:58:34AM +0200, Konstantin Belousov wrote: > > > > On Thu, Feb 04, 2021 at 01:34:13PM -0800, Matthew Macy wrote: > > > > > On Thu, Feb 4, 2021 at 1:31 PM Alan Somers > > > wrote: > > > > > > > > > > > > After upgrading a machine to FreeBSD, 12.2, it hit the following > > > panic on > > > > > > its first reboot. I suspect that a few other servers have hit > this > > > too, > > > > > > but since it happens before swap is mounted there are no core > dumps, > > > and > > > > > > they usually reboot immediately. The code in question hasn't > > > changed since > > > > > > 2018. The panic happened in cmci_monitor at line 930. Does > anybody > > > have > > > > > > any suggestions for how I could debug further? I can't readily > > > reproduce > > > > > > it, and I can't dump core, but I'd like to investigate it any > way I > > > can. > > > > > > The server in question has dual Xeon Gold 6142 CPUs. > > > > > > > > > > Try this. > > > > > > > > I think that there is no other dependencies in the startup order, but > > > > cannot know it for sure. > > > > > > > > commit 19584e3d3e9606d591fa30999b370ed758960e8c > > > > Author: Konstantin Belousov > > > > Date: Fri Feb 5 00:56:09 2021 +0200 > > > > > > > > x86: init mca before APs are started > > > > > > APs only call mca_init() after they have been released by the BSP > > > though, and that happens later in SI_SUB_SMP. > > > > > > > diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c > > > > index 03100e77d455..e2bf2673cf69 100644 > > > > --- a/sys/x86/x86/mca.c > > > > +++ b/sys/x86/x86/mca.c > > > > @@ -1371,7 +1371,7 @@ mca_init_bsp(void *arg __unused) > > > > > > > > mca_init(); > > > > } > > > > -SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_ANY, mca_init_bsp, NULL); > > > > +SYSINIT(mca_init_bsp, SI_SUB_CPU, SI_ORDER_SECOND, mca_init_bsp, > NULL); > > > > > > > > /* Called when a machine check exception fires. */ > > > > void > > > > > > > kib's patch causes a different problem, and this one is reproducible: > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 0; apic id = 00 > > fault virtual address = 0x18 > > fault code = supervisor read data, page not present > > instruction pointer = 0x20:0xffffffff8125762c > > stack pointer = 0x28:0xffffffff828dad90 > > frame pointer = 0x28:0xffffffff828dad90 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags = resume, IOPL = 0 > > current process = 0 () > > trap number = 12 > > panic: page fault > > cpuid = 0 > > time = 1 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > 0xffffffff828daa50 > > vpanic() at vpanic+0x17b/frame 0xffffffff828daaa0 > > panic() at panic+0x43/frame 0xffffffff828dab00 > > trap_fatal() at trap_fatal+0x391/frame 0xffffffff828dab60 > > trap_pfault() at trap_pfault+0x4f/frame 0xffffffff828dabb0 > > trap() at trap+0x286/frame 0xffffffff828dacc0 > > calltrap() at calltrap+0x8/frame 0xffffffff828dacc0 > > --- trap 0xc, rip = 0xffffffff8125762c, rsp = 0xffffffff828dad90, rbp = > > 0xffffffff828dad90 --- > > native_lapic_enable_cmc() at native_lapic_enable_cmc+0x1c/frame > > 0xffffffff828dad90 > > _mca_init() at _mca_init+0x94c/frame 0xffffffff828dadd0 > > mi_startup() at mi_startup+0xdf/frame 0xffffffff828dadf0 > > btext() at btext+0x2c > > KDB: enter: panic > > [ thread pid 0 tid 0 ] > > Stopped at kdb_enter+0x37: movq $0,0x12bc396(%rip) > > > > If you're wondering, the panic happens at this point in > > native_lapic_enable_cmc: > > > > apic_id = PCPU_GET(apic_id); > > KASSERT(lapics[apic_id].la_present, > > ("%s: missing APIC %u", __func__, apic_id)); > > lapics[apic_id].la_lvts[APIC_LVT_CMCI].lvt_masked = 0; <- panic here > > lapics[apic_id].la_lvts[APIC_LVT_CMCI].lvt_active = 1; > > if (bootverbose) > > printf("lapic%u: CMCI unmasked\n", apic_id); > > } > > Scratch this patch. > > Do you have INVARIANTS enabled? If not, I am curious if enabling them > would convert that rare page fault into rare "CPU %d has more MC banks" > assert. > > Also might be the output of the > # for x in $(jot $(sysctl -n hw.ncpu) 0) ; do cpucontrol -m 0x179 > /dev/cpuctl$x; done > command will show the issue (0x179 is the MCG_CAP MSR). > You need to load cpuctl(4) if it is not loaded yet. > I don't have INVARIANTS enabled, and I can't enable it on the production servers. However, I can turn those three KASSERTs into VERIFYs and see what happens. Here is what your command shows on the server that panicked: $ for x in $(jot $(sysctl -n hw.ncpu) 0) ; do sudo cpucontrol -m 0x179 /dev/cpuctl$x; done | uniq -c 16 MSR 0x179: 0x00000000 0x0f000c14 16 MSR 0x179: 0x00000000 0x0f000814 -Alan From owner-freebsd-stable@freebsd.org Fri Feb 5 02:40:12 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 77FE853A0A0 for ; Fri, 5 Feb 2021 02:40:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4DX05825PHz3rwg for ; Fri, 5 Feb 2021 02:40:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 47C72539FB1; Fri, 5 Feb 2021 02:40:12 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 47933539FB0 for ; Fri, 5 Feb 2021 02:40:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DX0576zqKz3ryj; Fri, 5 Feb 2021 02:40:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1152e4ho084476 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Feb 2021 04:40:07 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1152e4ho084476 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1152e3wL084467; Fri, 5 Feb 2021 04:40:03 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Feb 2021 04:40:03 +0200 From: Konstantin Belousov To: Alan Somers Cc: Mark Johnston , Matthew Macy , FreeBSD Stable ML Subject: Re: Page fault in _mca_init during startup Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DX0576zqKz3ryj X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 02:40:12 -0000 On Thu, Feb 04, 2021 at 07:01:30PM -0700, Alan Somers wrote: > On Thu, Feb 4, 2021 at 5:59 PM Konstantin Belousov > wrote: > > Do you have INVARIANTS enabled? If not, I am curious if enabling them > > would convert that rare page fault into rare "CPU %d has more MC banks" > > assert. > > > > Also might be the output of the > > # for x in $(jot $(sysctl -n hw.ncpu) 0) ; do cpucontrol -m 0x179 > > /dev/cpuctl$x; done > > command will show the issue (0x179 is the MCG_CAP MSR). > > You need to load cpuctl(4) if it is not loaded yet. > > > > I don't have INVARIANTS enabled, and I can't enable it on the production > servers. However, I can turn those three KASSERTs into VERIFYs and see > what happens. Here is what your command shows on the server that panicked: > $ for x in $(jot $(sysctl -n hw.ncpu) 0) ; do sudo cpucontrol -m 0x179 > /dev/cpuctl$x; done | uniq -c > 16 MSR 0x179: 0x00000000 0x0f000c14 > 16 MSR 0x179: 0x00000000 0x0f000814 It probably explains it, but it would be more telling if you left the output as is, so that we can see which CPUs have MCG_CMCI_P (10) bit set. I suspect that your machine has two sockets, and processor in one socket has CPUs reporting MCG_CMCI_P, while other processor does not. Your SMP is not quite symmetric, perhaps processors were from different bins? If BSP is selected on reporting socket, everything boots well. If other socket wins the BSP selection race, cmci is not initialized, but when per-cpu mca_init() sees CMCI_P bit, it calls cmci_setup() without allocated cmc state, because BSP did not needed it. If I am right, then unconditionally allocating the memory is probably the only choice there. commit 2e2c925ac3b626edc6492a57a80f6b87895801c2 Author: Konstantin Belousov Date: Fri Feb 5 04:32:05 2021 +0200 x86 mca: unconditionally allocate memory for cmc state diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c index 03100e77d455..dff3f7631f5c 100644 --- a/sys/x86/x86/mca.c +++ b/sys/x86/x86/mca.c @@ -1047,7 +1047,7 @@ mca_setup(uint64_t mcg_cap) "force_scan", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, NULL, 0, sysctl_mca_scan, "I", "Force an immediate scan for machine checks"); #ifdef DEV_APIC - if (cmci_supported(mcg_cap)) + if (cpu_vendor_id == CPU_VENDOR_INTEL) cmci_setup(); else if (amd_thresholding_supported()) amd_thresholding_setup(); From owner-freebsd-stable@freebsd.org Fri Feb 5 02:53:21 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E97B653A6F8 for ; Fri, 5 Feb 2021 02:53:21 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4DX0NK5RJZz3t6V for ; Fri, 5 Feb 2021 02:53:21 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id BA88D53A248; Fri, 5 Feb 2021 02:53:21 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BA51253A4B3 for ; Fri, 5 Feb 2021 02:53:21 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DX0NK4qPzz3t1h; Fri, 5 Feb 2021 02:53:21 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f176.google.com with SMTP id k142so6045966oib.7; Thu, 04 Feb 2021 18:53:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sI1Uk0oe3vONQQgbDvRuvGXBYIzBaWrz8rTlYoAOg6k=; b=nc6N+zGBZxD1JWlNmLoMKQQeKqGjL0UOsjFkq4AiC+OgLY0TzWzfUbrtcAFPz6ZkiK i4QtzpWKAPGJtjqPArpO5BwTIref//q0WCDb2ljhgSNT8Ibmf1ioi+5+rY+dm408AVrW glI1+TLBuDGJlKyMb7SaTovLgxO0zWfYIYDsU7IzfVhwHVFEW7HGmm/OMP3UdgoYVaev jyN3nppO1NwHJD7c9NM9hG/q0szAsC6h5ZFMYti6BCSPifB9DJRVJfH8zcYiJTpyttfh LzCyHZ/BfrnaHT8/816suSVQ2cXY29r9UVQT2Hj0W3WYFnr35vpU0PdpJ9k6GwCiOb+2 7h9A== X-Gm-Message-State: AOAM532Z7iAc6pVo4785fbQWF7MIbuUAkR4gbsWzoY4INBmDN3AWBX40 yrO9AxQzkwNmHlmJDnmZpzsWgHH2z2cSvuMdDuA= X-Google-Smtp-Source: ABdhPJwAF1CHcR1AG3xlo1HW/P3d4I2tix8W0c7HHssFdNR0NWZzT99T9YXTadxF+LxMzRNuOf2M4JwlHTY2vnx20oo= X-Received: by 2002:a54:4813:: with SMTP id j19mr1775503oij.73.1612493600456; Thu, 04 Feb 2021 18:53:20 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Thu, 4 Feb 2021 19:53:09 -0700 Message-ID: Subject: Re: Page fault in _mca_init during startup To: Konstantin Belousov Cc: Mark Johnston , Matthew Macy , FreeBSD Stable ML X-Rspamd-Queue-Id: 4DX0NK4qPzz3t1h X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 02:53:22 -0000 On Thu, Feb 4, 2021 at 7:40 PM Konstantin Belousov wrote: > On Thu, Feb 04, 2021 at 07:01:30PM -0700, Alan Somers wrote: > > On Thu, Feb 4, 2021 at 5:59 PM Konstantin Belousov > > wrote: > > > Do you have INVARIANTS enabled? If not, I am curious if enabling them > > > would convert that rare page fault into rare "CPU %d has more MC banks" > > > assert. > > > > > > Also might be the output of the > > > # for x in $(jot $(sysctl -n hw.ncpu) 0) ; do cpucontrol -m 0x179 > > > /dev/cpuctl$x; done > > > command will show the issue (0x179 is the MCG_CAP MSR). > > > You need to load cpuctl(4) if it is not loaded yet. > > > > > > > I don't have INVARIANTS enabled, and I can't enable it on the production > > servers. However, I can turn those three KASSERTs into VERIFYs and see > > what happens. Here is what your command shows on the server that > panicked: > > $ for x in $(jot $(sysctl -n hw.ncpu) 0) ; do sudo cpucontrol -m 0x179 > > /dev/cpuctl$x; done | uniq -c > > 16 MSR 0x179: 0x00000000 0x0f000c14 > > 16 MSR 0x179: 0x00000000 0x0f000814 > > It probably explains it, but it would be more telling if you left the > output as is, so that we can see which CPUs have MCG_CMCI_P (10) bit set. > I didn't sort them, so the first 16 have bit 10 set and the second 16 don't. > > I suspect that your machine has two sockets, and processor in one socket > has CPUs reporting MCG_CMCI_P, while other processor does not. Your SMP > is not quite symmetric, perhaps processors were from different bins? > Could be. Is there some MSR that reports a more specific version number? > > If BSP is selected on reporting socket, everything boots well. If > other socket wins the BSP selection race, cmci is not initialized, but > when per-cpu mca_init() sees CMCI_P bit, it calls cmci_setup() without > allocated cmc state, because BSP did not needed it. > > If I am right, then unconditionally allocating the memory is probably the > only choice there. > > commit 2e2c925ac3b626edc6492a57a80f6b87895801c2 > Author: Konstantin Belousov > Date: Fri Feb 5 04:32:05 2021 +0200 > > x86 mca: unconditionally allocate memory for cmc state > > diff --git a/sys/x86/x86/mca.c b/sys/x86/x86/mca.c > index 03100e77d455..dff3f7631f5c 100644 > --- a/sys/x86/x86/mca.c > +++ b/sys/x86/x86/mca.c > @@ -1047,7 +1047,7 @@ mca_setup(uint64_t mcg_cap) > "force_scan", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, NULL, > 0, > sysctl_mca_scan, "I", "Force an immediate scan for machine > checks"); > #ifdef DEV_APIC > - if (cmci_supported(mcg_cap)) > + if (cpu_vendor_id == CPU_VENDOR_INTEL) > cmci_setup(); > else if (amd_thresholding_supported()) > amd_thresholding_setup(); > From owner-freebsd-stable@freebsd.org Fri Feb 5 08:11:48 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0C0A9540EAF for ; Fri, 5 Feb 2021 08:11:48 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: from onyphe.fr (super1.onyphe.io [54.36.107.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DX7Rl5D4Cz4g5B for ; Fri, 5 Feb 2021 08:11:46 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: (qmail 76521 invoked by uid 0); 5 Feb 2021 08:11:45 -0000 Received: from unknown (HELO www.onyphe.io) (172.16.6.254) by smtpout.jail with SMTP; 5 Feb 2021 08:11:45 -0000 MIME-Version: 1.0 Date: Fri, 05 Feb 2021 09:11:45 +0100 From: GomoR To: John Baldwin Cc: freebsd-stable@freebsd.org Subject: Re: Suspected mbuf leak with Nginx + sendfile + TLS in 12.2-STABLE In-Reply-To: <9c56bfda-725c-9c2a-9db3-4599abcfeaa0@FreeBSD.org> References: <9c56bfda-725c-9c2a-9db3-4599abcfeaa0@FreeBSD.org> User-Agent: Roundcube Webmail/1.4.8 Message-ID: <069535216479ce00859e4bcbf499f8a1@gomor.org> X-Sender: freebsd-stable@gomor.org Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DX7Rl5D4Cz4g5B X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 08:11:48 -0000 On 2021-02-04 19:33, John Baldwin wrote: > None of the sendfile or KTLS changes from Netflix are in 12, they are > only > in 13 and later. I thought about that possibility, thank you for the clarification. >> Don't transmit mbufs that aren't yet ready on TOE sockets. >> This includes mbufs waiting for data from sendfile() I/O requests, or >> mbufs awaiting encryption for KTLS. >> https://github.com/freebsd/freebsd-src/commit/14c77f30b201bf76119d59678e72051c093333c2 > > This patch only applies to Chelsio T5/T6 NICs when using TOE (TCP > offload) > and doesn't affect freeing mbufs, it just fixes a race when the NIC > could > potentially send random garbage if it sends the mbuf before the > scheduled > disk I/O to populate it with data from disk has completed. Understood. >> NIC is: >> ix0: >> >> What can we do to help you find the root cause? > > The first step I would do if possible would be to bisect between the > last > known working version and the version that is known to be broken to > determine which commit introduced the problem. One thing that could > help > here is to see if you can reproduce the problem using a 12.2 kernel on > a > 12.1 world + ports. If you can, then you can limit your bisecting to > just > building new kernels which will make that process quicker. Thank you for the tip, I'll try that path and let you know. > You might also see if using a different NIC shows the same problem. If > not, then it might point to a regression in the NIC driver (or perhaps > in > iflib as ix uses iflib I believe). Unfortunately, not a possibility here. I did some other tests and found where the problem arise. In fact, we use proxy_pass directive within Nginx and the network flow goes through one public interface (ix0) and proxy_pass through a second (ix1) towards a remote machine. Changing the Nginx configuration to only go through ix0 does not cause the issue. So that's something about with passing packets between 2 NICs. I'll keep you posted. Regards, From owner-freebsd-stable@freebsd.org Fri Feb 5 10:54:11 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 469D1544846 for ; Fri, 5 Feb 2021 10:54:11 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: from onyphe.fr (super1.onyphe.io [54.36.107.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DXC360LRbz4r9K for ; Fri, 5 Feb 2021 10:54:09 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: (qmail 10758 invoked by uid 0); 5 Feb 2021 10:54:07 -0000 Received: from unknown (HELO www.onyphe.io) (172.16.6.254) by smtpout.jail with SMTP; 5 Feb 2021 10:54:07 -0000 MIME-Version: 1.0 Date: Fri, 05 Feb 2021 11:54:07 +0100 From: GomoR To: John Baldwin Cc: freebsd-stable@freebsd.org Subject: Re: Suspected mbuf leak with Nginx + sendfile + TLS in 12.2-STABLE In-Reply-To: <069535216479ce00859e4bcbf499f8a1@gomor.org> References: <9c56bfda-725c-9c2a-9db3-4599abcfeaa0@FreeBSD.org> <069535216479ce00859e4bcbf499f8a1@gomor.org> User-Agent: Roundcube Webmail/1.4.8 Message-ID: <8f02057bee5e8196644e85bbe7f8b31e@gomor.org> X-Sender: freebsd-stable@gomor.org Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DXC360LRbz4r9K X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-stable@gomor.org designates 54.36.107.100 as permitted sender) smtp.mailfrom=freebsd-stable@gomor.org X-Spamd-Result: default: False [-3.30 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.36.107.100]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[gomor.org]; RBL_DBL_DONT_QUERY_IPS(0.00)[54.36.107.100:from]; SPAMHAUS_ZRD(0.00)[54.36.107.100:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.999]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:16276, ipnet:54.36.0.0/16, country:FR]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 10:54:11 -0000 On 2021-02-05 09:11, GomoR wrote: >> The first step I would do if possible would be to bisect between the >> last >> known working version and the version that is known to be broken to >> determine which commit introduced the problem. One thing that could >> help >> here is to see if you can reproduce the problem using a 12.2 kernel on >> a >> 12.1 world + ports. If you can, then you can limit your bisecting to >> just >> building new kernels which will make that process quicker. We have reinstalled from scratch our system with FreeBSD 12.1-RELEASE. We then have installed just enough of our software stack to reproduce the issue. No problem with a stock 12.1-RELEASE kernel, but problem arise after installkernel with the latest 12.2-STABLE. We then turned off all our customizations, including some specific sysctl.conf values. The bug didn't triggered. After dissecting our sysctl values, the faulty one has been identified: kern.ipc.maxsockbuf=157286400 This value is 75 times the default value (2097152). Restoring the default value fixes the issue. After some tests, the bug is triggered starting somewhere to 64 times the default value (134217728). There was no issue with this setting in 12.1-RELEASE, but there is in 12.2-RELEASE. Do you have some insights onto why it causes that mbuf problems? In the meantime, we have our solution, but we are willing to help identify if that's a kernel bug or just a real bad idea to set maxsockbuf to such a high value. Best regards, From owner-freebsd-stable@freebsd.org Fri Feb 5 14:41:25 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AA04F54983E for ; Fri, 5 Feb 2021 14:41:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DXJ5K34Lqz3Ljf for ; Fri, 5 Feb 2021 14:41:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 68D115497D8; Fri, 5 Feb 2021 14:41:25 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 689225498C0 for ; Fri, 5 Feb 2021 14:41:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DXJ5K0x68z3LpM; Fri, 5 Feb 2021 14:41:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 115EfGB7057236 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Feb 2021 16:41:19 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 115EfGB7057236 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 115EfGf3057235; Fri, 5 Feb 2021 16:41:16 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Feb 2021 16:41:16 +0200 From: Konstantin Belousov To: Alan Somers Cc: Mark Johnston , Matthew Macy , FreeBSD Stable ML Subject: Re: Page fault in _mca_init during startup Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DXJ5K0x68z3LpM X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 14:41:25 -0000 On Thu, Feb 04, 2021 at 07:53:09PM -0700, Alan Somers wrote: > On Thu, Feb 4, 2021 at 7:40 PM Konstantin Belousov > wrote: > > > On Thu, Feb 04, 2021 at 07:01:30PM -0700, Alan Somers wrote: > > > On Thu, Feb 4, 2021 at 5:59 PM Konstantin Belousov > > > wrote: > > > > Do you have INVARIANTS enabled? If not, I am curious if enabling them > > > > would convert that rare page fault into rare "CPU %d has more MC banks" > > > > assert. > > > > > > > > Also might be the output of the > > > > # for x in $(jot $(sysctl -n hw.ncpu) 0) ; do cpucontrol -m 0x179 > > > > /dev/cpuctl$x; done > > > > command will show the issue (0x179 is the MCG_CAP MSR). > > > > You need to load cpuctl(4) if it is not loaded yet. > > > > > > > > > > I don't have INVARIANTS enabled, and I can't enable it on the production > > > servers. However, I can turn those three KASSERTs into VERIFYs and see > > > what happens. Here is what your command shows on the server that > > panicked: > > > $ for x in $(jot $(sysctl -n hw.ncpu) 0) ; do sudo cpucontrol -m 0x179 > > > /dev/cpuctl$x; done | uniq -c > > > 16 MSR 0x179: 0x00000000 0x0f000c14 > > > 16 MSR 0x179: 0x00000000 0x0f000814 > > > > It probably explains it, but it would be more telling if you left the > > output as is, so that we can see which CPUs have MCG_CMCI_P (10) bit set. > > > > I didn't sort them, so the first 16 have bit 10 set and the second 16 > don't. > > > > > > I suspect that your machine has two sockets, and processor in one socket > > has CPUs reporting MCG_CMCI_P, while other processor does not. Your SMP > > is not quite symmetric, perhaps processors were from different bins? > > > > Could be. Is there some MSR that reports a more specific version number? There are CPUID %eax=1 values returned in %eax, but then it requires some interpretation. # cpucontrol -i 1 /dev/cpuctl$x for $x iterating over the cpus. From owner-freebsd-stable@freebsd.org Fri Feb 5 16:01:40 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5255854B05A for ; Fri, 5 Feb 2021 16:01:40 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DXKsw111Jz3R5Y for ; Fri, 5 Feb 2021 16:01:40 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 2293D54B39C; Fri, 5 Feb 2021 16:01:40 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 225D554B059 for ; Fri, 5 Feb 2021 16:01:40 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DXKsw04sZz3h0F; Fri, 5 Feb 2021 16:01:39 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f49.google.com with SMTP id i20so7320788otl.7; Fri, 05 Feb 2021 08:01:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=EHSan6n//yaZyOfPcUv1LgZOK1GuNW2plrr5BdUCKB8=; b=U7wmJdeMShP6FACd1dinbV2UzzRqmGCGjCrEYHTLh59OqG26yRyeq3gb2eT6T4O1+v BgQIj4L0AIxwCAwUG7evfOjEK6Rjado3bHvrpOnb07betRFx2Cho9sZaxwbVOBPd6AjS gk6dqmCscaX5XI8jMRBk3ZSiKM3y/DfO/bWR8cgWeWPftLHNyApdXIArRdlC7DQDrQJs KObzvaoAEqowml3P+OYPna7IFf5rTaG46Wa0EphUeQ7k1j5U9xhTtwVNeIIjXWIviM7Z BRtjeFiidrqDkeBUvk4uw//udXp72sT7eKRpA/pB7yygsB7ZBciWdgDvue7PmM1RWCsZ ympA== X-Gm-Message-State: AOAM530caIPUVTUq0XiAG0RFFtFLJXoF1Mq05MZrbFL4lGH1QCIWvzcM ov+PKHN6waNwQ2P6znAiPjtfeHz1pVAdSuHWDpA= X-Google-Smtp-Source: ABdhPJxV3v7BJZc+dZmLM5daDZQr3jXWSRw54QETDs+92TfJvkxl0+X3dBVRRPVLnOuCmvmZQ57vVjJOR0e346m+NA4= X-Received: by 2002:a05:6830:1256:: with SMTP id s22mr3957420otp.251.1612540898380; Fri, 05 Feb 2021 08:01:38 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Fri, 5 Feb 2021 09:01:26 -0700 Message-ID: Subject: Re: Page fault in _mca_init during startup To: Konstantin Belousov Cc: Mark Johnston , Matthew Macy , FreeBSD Stable ML X-Rspamd-Queue-Id: 4DXKsw04sZz3h0F X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 16:01:40 -0000 On Fri, Feb 5, 2021 at 7:41 AM Konstantin Belousov wrote: > On Thu, Feb 04, 2021 at 07:53:09PM -0700, Alan Somers wrote: > > On Thu, Feb 4, 2021 at 7:40 PM Konstantin Belousov > > wrote: > > > > > On Thu, Feb 04, 2021 at 07:01:30PM -0700, Alan Somers wrote: > > > > On Thu, Feb 4, 2021 at 5:59 PM Konstantin Belousov < > kostikbel@gmail.com> > > > > wrote: > > > > > Do you have INVARIANTS enabled? If not, I am curious if enabling > them > > > > > would convert that rare page fault into rare "CPU %d has more MC > banks" > > > > > assert. > > > > > > > > > > Also might be the output of the > > > > > # for x in $(jot $(sysctl -n hw.ncpu) 0) ; do cpucontrol -m 0x179 > > > > > /dev/cpuctl$x; done > > > > > command will show the issue (0x179 is the MCG_CAP MSR). > > > > > You need to load cpuctl(4) if it is not loaded yet. > > > > > > > > > > > > > I don't have INVARIANTS enabled, and I can't enable it on the > production > > > > servers. However, I can turn those three KASSERTs into VERIFYs and > see > > > > what happens. Here is what your command shows on the server that > > > panicked: > > > > $ for x in $(jot $(sysctl -n hw.ncpu) 0) ; do sudo cpucontrol -m > 0x179 > > > > /dev/cpuctl$x; done | uniq -c > > > > 16 MSR 0x179: 0x00000000 0x0f000c14 > > > > 16 MSR 0x179: 0x00000000 0x0f000814 > > > > > > It probably explains it, but it would be more telling if you left the > > > output as is, so that we can see which CPUs have MCG_CMCI_P (10) bit > set. > > > > > > > I didn't sort them, so the first 16 have bit 10 set and the second 16 > > don't. > > > > > > > > > > I suspect that your machine has two sockets, and processor in one > socket > > > has CPUs reporting MCG_CMCI_P, while other processor does not. Your SMP > > > is not quite symmetric, perhaps processors were from different bins? > I found 2 other servers that exhibit the same problem: the first 16 cores have bit 10 set and the second 16 don't. All 3 have dual Xeon Gold 6142 CPUs and SuperMicro X11DPU motherboards with BIOS revision 5.12. I have other examples of X11DPU motherboards that don't exhibit the problem, but they all have both different CPUs and different BIOS revisions. So I can't be sure whether the bug follows the CPU model or the BIOS version. > > > > > > > Could be. Is there some MSR that reports a more specific version number? > There are CPUID %eax=1 values returned in %eax, but then it requires > some interpretation. > # cpucontrol -i 1 /dev/cpuctl$x > for $x iterating over the cpus. > Apart from the Local APIC ID field, that returns the same value for all processors. Your second patch doesn't cause any obvious problems on my dev system. From owner-freebsd-stable@freebsd.org Fri Feb 5 17:21:22 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6F30654D0E1 for ; Fri, 5 Feb 2021 17:21:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4DXMdt1y0fz3nPr for ; Fri, 5 Feb 2021 17:21:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 42EA654CE78; Fri, 5 Feb 2021 17:21:22 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 42B5D54D307 for ; Fri, 5 Feb 2021 17:21:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DXMds5stDz3nV9; Fri, 5 Feb 2021 17:21:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 115HLDWQ097161 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Feb 2021 19:21:17 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 115HLDWQ097161 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 115HLD6U097160; Fri, 5 Feb 2021 19:21:13 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Feb 2021 19:21:13 +0200 From: Konstantin Belousov To: Alan Somers Cc: Mark Johnston , Matthew Macy , FreeBSD Stable ML Subject: Re: Page fault in _mca_init during startup Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DXMds5stDz3nV9 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 17:21:22 -0000 On Fri, Feb 05, 2021 at 09:01:26AM -0700, Alan Somers wrote: > On Fri, Feb 5, 2021 at 7:41 AM Konstantin Belousov > wrote: > > > On Thu, Feb 04, 2021 at 07:53:09PM -0700, Alan Somers wrote: > > > On Thu, Feb 4, 2021 at 7:40 PM Konstantin Belousov > > > wrote: > > > > > > > On Thu, Feb 04, 2021 at 07:01:30PM -0700, Alan Somers wrote: > > > > > On Thu, Feb 4, 2021 at 5:59 PM Konstantin Belousov < > > kostikbel@gmail.com> > > > > > wrote: > > > > > > Do you have INVARIANTS enabled? If not, I am curious if enabling > > them > > > > > > would convert that rare page fault into rare "CPU %d has more MC > > banks" > > > > > > assert. > > > > > > > > > > > > Also might be the output of the > > > > > > # for x in $(jot $(sysctl -n hw.ncpu) 0) ; do cpucontrol -m 0x179 > > > > > > /dev/cpuctl$x; done > > > > > > command will show the issue (0x179 is the MCG_CAP MSR). > > > > > > You need to load cpuctl(4) if it is not loaded yet. > > > > > > > > > > > > > > > > I don't have INVARIANTS enabled, and I can't enable it on the > > production > > > > > servers. However, I can turn those three KASSERTs into VERIFYs and > > see > > > > > what happens. Here is what your command shows on the server that > > > > panicked: > > > > > $ for x in $(jot $(sysctl -n hw.ncpu) 0) ; do sudo cpucontrol -m > > 0x179 > > > > > /dev/cpuctl$x; done | uniq -c > > > > > 16 MSR 0x179: 0x00000000 0x0f000c14 > > > > > 16 MSR 0x179: 0x00000000 0x0f000814 > > > > > > > > It probably explains it, but it would be more telling if you left the > > > > output as is, so that we can see which CPUs have MCG_CMCI_P (10) bit > > set. > > > > > > > > > > I didn't sort them, so the first 16 have bit 10 set and the second 16 > > > don't. > > > > > > > > > > > > > > I suspect that your machine has two sockets, and processor in one > > socket > > > > has CPUs reporting MCG_CMCI_P, while other processor does not. Your SMP > > > > is not quite symmetric, perhaps processors were from different bins? > > > > I found 2 other servers that exhibit the same problem: the first 16 cores > have bit 10 set and the second 16 don't. All 3 have dual Xeon Gold 6142 > CPUs and SuperMicro X11DPU motherboards with BIOS revision 5.12. I have > other examples of X11DPU motherboards that don't exhibit the problem, but > they all have both different CPUs and different BIOS revisions. So I can't > be sure whether the bug follows the CPU model or the BIOS version. I looked at the full spec update errata list for the first gen Skylake Xeons, but did not noticed anything relevant. EDS doc does not provide much useful info on the MSR 0x179 bit 10 either, except rewording SDM definition. In fact I am not sure but this bit might be writeable by software. Try to flip the bit with cpucontrol(8). Might be it is a BIOS bug after all. If you have Intel representative contact, or Supermicro contact, try to engage them. I do not have any further ideas, since spec update does not mention the problem. > > > > > > > > > > > > Could be. Is there some MSR that reports a more specific version number? > > There are CPUID %eax=1 values returned in %eax, but then it requires > > some interpretation. > > # cpucontrol -i 1 /dev/cpuctl$x > > for $x iterating over the cpus. > > > > Apart from the Local APIC ID field, that returns the same value for all > processors. > > Your second patch doesn't cause any obvious problems on my dev system. I hope that you would confirm that the issue is solved by it, after some time. From owner-freebsd-stable@freebsd.org Sat Feb 6 12:19:10 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 65DB5547662 for ; Sat, 6 Feb 2021 12:19:10 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DXrtj2ZyDz4XSb; Sat, 6 Feb 2021 12:19:08 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1l8MYF-000A0q-VV; Sat, 06 Feb 2021 15:18:59 +0300 Date: Sat, 6 Feb 2021 15:18:59 +0300 From: Slawa Olhovchenkov To: GomoR Cc: John Baldwin , freebsd-stable@freebsd.org Subject: Re: Suspected mbuf leak with Nginx + sendfile + TLS in 12.2-STABLE Message-ID: <20210206121859.GE75195@zxy.spb.ru> References: <9c56bfda-725c-9c2a-9db3-4599abcfeaa0@FreeBSD.org> <069535216479ce00859e4bcbf499f8a1@gomor.org> <8f02057bee5e8196644e85bbe7f8b31e@gomor.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8f02057bee5e8196644e85bbe7f8b31e@gomor.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-Rspamd-Queue-Id: 4DXrtj2ZyDz4XSb X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of slw@zxy.spb.ru has no SPF policy when checking 195.70.199.98) smtp.mailfrom=slw@zxy.spb.ru X-Spamd-Result: default: False [-0.73 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zxy.spb.ru]; RBL_DBL_DONT_QUERY_IPS(0.00)[195.70.199.98:from]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.37)[0.373]; SPAMHAUS_ZRD(0.00)[195.70.199.98:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:5495, ipnet:195.70.192.0/19, country:RU]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2021 12:19:10 -0000 On Fri, Feb 05, 2021 at 11:54:07AM +0100, GomoR wrote: > On 2021-02-05 09:11, GomoR wrote: > >> The first step I would do if possible would be to bisect between the > >> last > >> known working version and the version that is known to be broken to > >> determine which commit introduced the problem. One thing that could > >> help > >> here is to see if you can reproduce the problem using a 12.2 kernel on > >> a > >> 12.1 world + ports. If you can, then you can limit your bisecting to > >> just > >> building new kernels which will make that process quicker. > > We have reinstalled from scratch our system with FreeBSD 12.1-RELEASE. > We then > have installed just enough of our software stack to reproduce the issue. > > No problem with a stock 12.1-RELEASE kernel, but problem arise after > installkernel > with the latest 12.2-STABLE. We then turned off all our customizations, > including > some specific sysctl.conf values. The bug didn't triggered. > > After dissecting our sysctl values, the faulty one has been identified: > > kern.ipc.maxsockbuf=157286400 > > This value is 75 times the default value (2097152). Restoring the > default value > fixes the issue. After some tests, the bug is triggered starting > somewhere to > 64 times the default value (134217728). > > There was no issue with this setting in 12.1-RELEASE, but there is in > 12.2-RELEASE. > > Do you have some insights onto why it causes that mbuf problems? In the > meantime, > we have our solution, but we are willing to help identify if that's a > kernel bug > or just a real bad idea to set maxsockbuf to such a high value. === > Each time a user downloads a file, mbuf & mbuf_clusters are raising to > reach the maximum limit in a matter of seconds. Those values are > asserted by 'netstat -m' as follows: > > Normal situation: > > mbuf: 256, 26031105, 16767, 5974,428087938, 0, 0 > mbuf_cluster: 2048, 8135232, 18408, 2704,101644203, 0, 0 > > Warning situtation: > > mbuf: 256, 26031105, 2981516, 151205,1109483561, 0, 0 > mbuf_cluster: 2048, 8135232, 2983155, 4201,319714617, 0, 0 === Can you clarified what is problem? I.e. under load system used more resources and this is not bug. Do you see more resources usage compared to load? Or resources don't freed after drop load? From owner-freebsd-stable@freebsd.org Sat Feb 6 15:29:58 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6F1A252C8A4 for ; Sat, 6 Feb 2021 15:29:58 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: from onyphe.fr (super1.onyphe.io [54.36.107.100]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DXx6t0rhlz4kd1 for ; Sat, 6 Feb 2021 15:29:57 +0000 (UTC) (envelope-from freebsd-stable@gomor.org) Received: (qmail 68259 invoked by uid 0); 6 Feb 2021 15:29:50 -0000 Received: from unknown (HELO www.onyphe.io) (172.16.6.254) by smtpout.jail with SMTP; 6 Feb 2021 15:29:50 -0000 MIME-Version: 1.0 Date: Sat, 06 Feb 2021 16:29:50 +0100 From: GomoR To: Slawa Olhovchenkov Cc: John Baldwin , freebsd-stable@freebsd.org Subject: Re: Suspected mbuf leak with Nginx + sendfile + TLS in 12.2-STABLE In-Reply-To: <20210206121859.GE75195@zxy.spb.ru> References: <9c56bfda-725c-9c2a-9db3-4599abcfeaa0@FreeBSD.org> <069535216479ce00859e4bcbf499f8a1@gomor.org> <8f02057bee5e8196644e85bbe7f8b31e@gomor.org> <20210206121859.GE75195@zxy.spb.ru> User-Agent: Roundcube Webmail/1.4.8 Message-ID: X-Sender: freebsd-stable@gomor.org Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DXx6t0rhlz4kd1 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2021 15:29:58 -0000 On 2021-02-06 13:18, Slawa Olhovchenkov wrote: > Can you clarified what is problem? > I.e. under load system used more resources and this is not bug. > Do you see more resources usage compared to load? > Or resources don't freed after drop load? Hello, the problem is: a user clicks on an HTTPS link to download a file and in a matter of seconds the server hangs. Absolutely no load on the server involved, just mbufs that reach maximum limit allowed in seconds. Stopping the download after a threshold and mbufs can't be freed anymore. Regards, From owner-freebsd-stable@freebsd.org Sat Feb 6 19:37:27 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 06037532D8A; Sat, 6 Feb 2021 19:37:27 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DY2cQ6rNPz3Gqq; Sat, 6 Feb 2021 19:37:26 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1612640247; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=na9KEK0bgLe50ryo1R9ZKIMPYJdCuBQwIAeA23uqlSU=; b=q0iFqcL7pROKmLOBulbd/Qu0JDjKW34s6hTl28G5XX6q83YC2qlLfvyn9EZClBfCF3Vdt+ Rr1RwV+pqjTQp5jrDad0p+Jwu8m7anTEn5EvZYGWgMLzV6N1VTtQb35Sjl/WeTZAKh0znv ThcBQHiQUfmID7xX59dRvlOc/QnPVH7iugI/Hg3D9avQZ9sKU8KioVWYxtVuoud/SUxISy 9h4bsYhxhz/zwuqgPDSh2+O0Gj3S5RHxvz6nbKyowuhbcLe1AvoHUZkonzbe2Y42xMgYUA RV1MORFBG9HlEy7NZ8I62uq1thcNSEfKpMQNa2DSxPA/B7n6qemv4K4rcLWuRw== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 7DD1E12BC3; Sat, 6 Feb 2021 19:37:26 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Sat, 6 Feb 2021 19:37:23 +0000 From: Glen Barber To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org Cc: FreeBSD Release Engineering Team Subject: FreeBSD 13.0-BETA1 Now Available Message-ID: <20210206193723.GL6785@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1612640247; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=na9KEK0bgLe50ryo1R9ZKIMPYJdCuBQwIAeA23uqlSU=; b=kSZXnxj0/j0fwy8eJQGiTQKBkkCj2w0IKhlC/xYebNkA6jZBIkOQDAkndUoTl6OdQ/MjmN xDFm1r7okJWaglo1RetSCn/eH9p1r3nU23AB4PlpUrX+mrhxwLwYJvrf9J2Ootru+WX6BM k/lKeWQz4scsZ6lG44YrriIba7HOtL6mA8n90oSGenw/CZnh7PKD4nJqz5+nTKcs8WkBFw aZpm29AdUmyyPl0PXIRXPJVBdKg61UB7JLTu+acRiy8p4Tr+IMuIMxI0NoX0PlEHkt9GQy 0nBoji9FIzVdF5OlklqhBHsyEhWIAreAWHOk87AckZ2DTeLXNp4iwVUMvZzSLw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1612640247; a=rsa-sha256; cv=none; b=DE5FcMefA6axasDEXjnX/9kwJGobNU9cDg92t8nQLiTt/FI0rVXnqS9ebpeAV7OTPUkCDO ywfFiQK6jbhqpYGE1A4ZBlJKc9Xo++/zusFQsq8eGjFrM0DEMw41TiAFN4aPkVclxxRkJO oTWDfU5ZxbG9glh1irzvsNZl3frpjXq9+dAGPSzlkYG7mnO1QjWmBDxW7HkLO8F+jRqB3n hnu6FeKVsJolqHsst/otX3+Fbwo6u+MxpjOtvUcX0NgzyLutTeDMTAV/DP+dEjprGPh+0U r16mCgItNQAYztwy0H3I4/rGB/n3vhk04Sob1K/P9ltmlnJDodebZI/rXKJDsw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2021 19:37:27 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 The first BETA build of the 13.0-RELEASE release cycle is now available. Installation images are available for: o 13.0-BETA1 amd64 GENERIC o 13.0-BETA1 powerpc64 GENERIC64 o 13.0-BETA1 powerpc64le GENERIC64LE o 13.0-BETA1 powerpcspe MPC85XXSPE o 13.0-BETA1 aarch64 GENERIC o 13.0-BETA1 aarch64 RPI o 13.0-BETA1 aarch64 PINE64 o 13.0-BETA1 aarch64 PINE64-LTS o 13.0-BETA1 aarch64 PINEBOOK o 13.0-BETA1 aarch64 ROCK64 o 13.0-BETA1 aarch64 ROCKPRO64 o 13.0-BETA1 riscv64 GENERIC o 13.0-BETA1 riscv64 GENERICSD Note regarding arm SD card images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root. It is strongly recommended to change the password for both users after gaining access to the system. Installer images and memory stick images are available here: https://download.freebsd.org/ftp/releases/ISO-IMAGES/13.0/ The image checksums follow at the end of this e-mail. If you notice problems you can report them through the Bugzilla PR system or on the -stable mailing list. If you would like to use Git to do a source based update of an existing system, use the "releng/13.0" branch. Please note, the release notes page is not yet complete, and will be updated on an ongoing basis as the 13.0-RELEASE cycle progresses. === Virtual Machine Disk Images === VM disk images are available for the amd64, and aarch64 architectures. Disk images may be downloaded from the following URL (or any of the FreeBSD download mirrors): https://download.freebsd.org/ftp/releases/VM-IMAGES/13.0-BETA1/ The partition layout is: ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label) ~ 1 GB - freebsd-swap GPT partition type (swapfs GPT label) ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label) The disk images are available in QCOW2, VHD, VMDK, and raw disk image formats. The image download size is approximately 135 MB and 165 MB respectively (amd64/i386), decompressing to a 21 GB sparse image. Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI loader file is needed for qemu-system-aarch64 to be able to boot the virtual machine images. See this page for more information: https://wiki.freebsd.org/arm64/QEMU To boot the VM image, run: % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt \ -bios QEMU_EFI.fd -serial telnet::4444,server -nographic \ -drive if=none,file=VMDISK,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -device virtio-net-device,netdev=net0 \ -netdev user,id=net0 Be sure to replace "VMDISK" with the path to the virtual machine image. BASIC-CI images can be found at: https://download.freebsd.org/ftp/releases/CI-IMAGES/13.0-BETA1/ === Amazon EC2 AMI Images === FreeBSD/amd64 EC2 AMIs are available in the following regions: af-south-1 region: ami-0b73d1612908a4a44 eu-north-1 region: ami-01598fe2c2801d2ad ap-south-1 region: ami-06d1b0fd9c5080340 eu-west-3 region: ami-0b5b08516fa6b9516 eu-west-2 region: ami-0a36d92c5c4a2d895 eu-south-1 region: ami-0b1f18f26cde8703e eu-west-1 region: ami-0fb3f046729fe283a ap-northeast-2 region: ami-0fb015c5f16d895cd me-south-1 region: ami-0f3fcb76b522aa8ec ap-northeast-1 region: ami-04131e8d488a81715 sa-east-1 region: ami-0b71fe0c26e1b66ec ca-central-1 region: ami-035634001169005ca ap-east-1 region: ami-0f5f427b679c87e7e ap-southeast-1 region: ami-03a34e19c1f849294 ap-southeast-2 region: ami-087b81741007cb3a0 eu-central-1 region: ami-037955ce4fff3288e us-east-1 region: ami-0c516702a0317d73e us-east-2 region: ami-0ed1e964233416530 us-west-1 region: ami-01e7215b6c673cce5 us-west-2 region: ami-009d2f232eae501b7 FreeBSD/aarch64 EC2 AMIs are available in the following regions: af-south-1 region: ami-08c04627b063f6ab5 eu-north-1 region: ami-07033e3966f43030c ap-south-1 region: ami-03c045241cce2e199 eu-west-3 region: ami-059387349d87254f5 eu-west-2 region: ami-0ae0e486833625264 eu-south-1 region: ami-0bbba0906ca01eeb3 eu-west-1 region: ami-026d7bcdc6acde794 ap-northeast-2 region: ami-0b7544eff7e1c8a26 me-south-1 region: ami-053cb544670f5129e ap-northeast-1 region: ami-04e066702451c597c sa-east-1 region: ami-0ef8e02f34b1d6976 ca-central-1 region: ami-05436cfaa24b3a00c ap-east-1 region: ami-0362c8f1e2a700094 ap-southeast-1 region: ami-05b1cf5c8e05418e8 ap-southeast-2 region: ami-0834a0f346c931136 eu-central-1 region: ami-05e82803b24b6097a us-east-1 region: ami-06af003d9d818164f us-east-2 region: ami-08ef1a3f0f078b4a8 us-west-1 region: ami-069a26eb191eb3271 us-west-2 region: ami-05e25d5025e93b193 === Vagrant Images === FreeBSD/amd64 images are available on the Hashicorp Atlas site, and can be installed by running: % vagrant init freebsd/FreeBSD-13.0-BETA1 % vagrant up === Upgrading === Due to unforeseen issues with BETA1, freebsd-update(8) builds are not available for 13.0-BETA1. == ISO CHECKSUMS == o 13.0-BETA1 amd64 GENERIC: SHA512 (FreeBSD-13.0-BETA1-amd64-bootonly.iso) = f09bd58cdf4c2a649f622225b6210f3af57330ce88ab0a773867d45b4642f4f14285f8875d6f2cfdccbd1eaf736218f53a9e43c42615bc3812dac183fbfa28cd SHA512 (FreeBSD-13.0-BETA1-amd64-bootonly.iso.xz) = f0fd85d15c5e791012de705580403d66ba8b53c8c76d2c2ae4d5709593f21f66b61384a82feb0e5f1d649f90393bfc312a81958bd8f5694c249687f17863826d SHA512 (FreeBSD-13.0-BETA1-amd64-disc1.iso) = e43bfce6f0bcb5cd3b1cad9a84804f7db87345080c3ce74cf84ea5c827c89575b0aac5647b09723c31b2dd698554be7a8d2e0caa977992f88be798e0a86128b6 SHA512 (FreeBSD-13.0-BETA1-amd64-disc1.iso.xz) = f6ddde375443e6c43834727aafdc3136c9060e939c680ab805bc2fcfa4995ce0cfc49f3f695f76046eed0b3ec4329fdd7ff85e2b7abad92bfbdfa12efce8b394 SHA512 (FreeBSD-13.0-BETA1-amd64-memstick.img) = 137fcbba5136a8bc08c861e17d4d504685ef67d4f8ef5d4c11e0bbb98f1eab6d51d6453d07e7ef990496f90bbb656b9dfeb784c7bd4e4cbf7f8952e08cb6c2dd SHA512 (FreeBSD-13.0-BETA1-amd64-memstick.img.xz) = f20e2ffebf4125babe386dcb1c38852ad11cb38fdef11a773f4842a03f396e26c0f49c11bf9fa137268df73fce4b7b3ea2d03d167ef898edce925aa2f1445060 SHA512 (FreeBSD-13.0-BETA1-amd64-mini-memstick.img) = 0d9b489018d24538cf9ec7441fc601f8781d6c613c1a3d9f23281af9370efd0db9a93bde34f998de9bb8ec7f5bc557e15400538c857b7e0c9ead20a87274b1ee SHA512 (FreeBSD-13.0-BETA1-amd64-mini-memstick.img.xz) = 2e095615984b8c60b9247755640f227a8d4313559a4a0fd100414ada4901a8c30d2e31a16bde40e67d33408934a244010c3d6ca8538f42b02f815d36fc4e0742 SHA256 (FreeBSD-13.0-BETA1-amd64-bootonly.iso) = 984d3b81e0d0987d6ea90d44e976ef72728396cbef68da68c5d7261d9a5ad1e7 SHA256 (FreeBSD-13.0-BETA1-amd64-bootonly.iso.xz) = a81acbeb0d75e787bbd85b65cc1e0dc34b020e8bbf0f9cfbac6099a7d1454cbd SHA256 (FreeBSD-13.0-BETA1-amd64-disc1.iso) = aaf21006ee32a15afa30fe307401a4869f87e405edd3cc8b2eaefb007fc640e5 SHA256 (FreeBSD-13.0-BETA1-amd64-disc1.iso.xz) = b65628b993e2d9c1fa6683791f32548a4a003331474ad9c174d62d2c8fc91168 SHA256 (FreeBSD-13.0-BETA1-amd64-memstick.img) = f31421df1d893b5a55404c9e3fd920e08d1f9ade4f363de3503a3c55780bf8bd SHA256 (FreeBSD-13.0-BETA1-amd64-memstick.img.xz) = 671a16efbc2471e4fd5cf73ec9f8a459b71b6357b07cf0ed2eb8e5655e45ad34 SHA256 (FreeBSD-13.0-BETA1-amd64-mini-memstick.img) = 1e1d8b3f682e7339de5fdbb40bc1e6e33e4ba41a2bc7fd0987cb2cf9fb35fc10 SHA256 (FreeBSD-13.0-BETA1-amd64-mini-memstick.img.xz) = 33f7cd1ddc02cb01b8d3fa0f02f1501dbab843e6268e83ec2a697c850bbb3e9d o 13.0-BETA1 powerpc64 GENERIC64: SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64-bootonly.iso) = 47aaa2cdd0809c7e9b087a44ef07cd38eee8ca20bd57f23a2dfd4f76c73342c87dedfdf572cc45f0547e0734153b678ca15d9ce33fbbe98725001ef9cb830bb8 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64-bootonly.iso.xz) = 85b967a6b00bfb4a87f7396f0f2cd66426b497abfdc0d8908bc0679b40e752c6c3423a66d5192f8a22b10b5365ec2a06ab55143839363f580379d09504104259 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64-disc1.iso) = 72f4dcd367c25c8be86a27931fc72c4b6326a5b7d034271a69b03ab695057383d24902f63ab8fefaa8c9dfab7d093ed170963f00c194337f282e0919252b30f3 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64-disc1.iso.xz) = e84e2edff3dde6642f241cd016b4880ccd0943df7a3275eb8d3d7756eb0ccb007033165c15e8a4ccb06bfeffa8658143ea956e25048de85699c1d3c880495d07 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64-memstick.img) = 27cdd0b11c1e8b702934b85bd8ee344e220fdbb2404f1778e466815b564c2bd816c42d805a58e8e9d0f5eefe07798528afada75b5c5cae279ba199a6fb5faff8 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64-memstick.img.xz) = 95e394afe5a95202aefbbc1609c289ce9661d268c8866c29e1c90bd3f772095b496fe0eb73ae253d2bc23e20f5aea706d71cbc2ddca932c43d8cd27cebd946ba SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64-mini-memstick.img) = e4f6119d8865a3fc5ac2a9360b4b0bf45fcbfdbc3545f5546f6d5075ffb14bcea4966ec23610dd036269cce01c771e6cdc797d7c8402e42f3e56a1b970e5146c SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64-mini-memstick.img.xz) = 7794dc962254ef91bdae0cd66470582d50022b35d19b6f0441935eeb85201774368e2c2bff9d7f2838befe29c8f23cbe9a86ccb64c1cfa2178ba743436fe502d SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64-bootonly.iso) = b3f70d6c36a21989cf498d3cff7080761c345940fdf06738a9c961c455ae1e12 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64-bootonly.iso.xz) = 89fc4ff1b7105cd8dfc1afce31e60380a526861e96d009a6bc991d40e328f3c2 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64-disc1.iso) = e68ab8047229fdf96c837aabbc1ab9f64d86849b4d1f286bae7c8e14e59dd33f SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64-disc1.iso.xz) = 687e9b2eac0b5b1ec1cf4e59d305b9eb34182f88ae001afd830645b144ecd5f5 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64-memstick.img) = 4e82de3b184aa10db1986018cabd6190b828639e3eff024676748dc4dee8f776 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64-memstick.img.xz) = e94cac0c1f2f65480a264a8c3696b1620a83f2cdb813c0080b8ad8e0cc88ff8e SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64-mini-memstick.img) = c05431a34a8da91b24502d5b35645ebbe88954ec377d70c5f2c52b5329ee6110 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64-mini-memstick.img.xz) = a94abdb9ce04d65eecb8eed5c43aac8f6d7cf5478d149a77c3b59a102b8d7928 o 13.0-BETA1 powerpc64le GENERIC64LE: SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-bootonly.iso) = 368427556532a6900a8527d8ac35f5208ba9131692fedb74753d952b43b8ddbd2b42f9ec1ce101a270847d2bf3071e97b09b8a50b60c36df29058f8fcbbf306e SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-bootonly.iso.xz) = 10b73726ce72399eeeed760b3bd9b50b6ecfd29982f3e14dc0bd9a0d8be55d3bf7b2bf99f17bd98f3fb90db3b851c5b162855c14bc971e887d1b92b353c72617 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-disc1.iso) = 047748c65535c3d9100152a26a6cf515bf88a994f63994959f1020c902df062ec6b785b83240e7b671f2099a753f31f98024657da7b7560a5981eeb834b0de37 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-disc1.iso.xz) = a6412f9260329416d3e7f39b14c4436c4861ddd248246775ab6836a4904d5d31e5faca61f5b7469b0c9bf47841a9d348df879f439c8280ea5db8ebce81e20031 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-memstick.img) = 79af5472861d5f2ef1dfee2d59bc9b4aaa1ab6f11174e2c7fbd54853ebde6f3ca6ef9d7556677b52826edb2031189c7dfccd8952947c8a8dfb6ddf23061d368f SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-memstick.img.xz) = 7a2392c3f8360054422de0a06062fb579f022937343c309f43d7f8c50cabd618133474503e4e95d76e07f6435635229bb06981816ac4f06f8c0145d67557a6b1 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-mini-memstick.img) = 1e756d51d1a18b7430bc82f15b5202158ccf90c40cc8eab990986a3ad38958571ec051cff1dadda28d1f94d3b28ab81ca57f64509d2364e5c4aa7d015bbbb57a SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-mini-memstick.img.xz) = d16861c7bd2431f15e1e120b3a87dd05a75557419c89b695d1c1ad5b188322a34272e16a4e6b1314bf0ddd249c2a87564ed05f9971aeca0bd15daaaae3825034 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-bootonly.iso) = f19f4e624a6780aa25bf9d9b784c43bd77f59ebc197c15ff9bd8da0cfaf923f0 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-bootonly.iso.xz) = 7879be992057f9a57335d3f963b51907d7826b4deb4b0454b4437344c2402dae SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-disc1.iso) = 70265a644c7968251499e251fd96ef14a6add05c3e6ea19321aba55bb6a74a20 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-disc1.iso.xz) = 67dbc36091e1f4dd405085231f28f0b4443794830ccd0062d2e11a48d891392d SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-memstick.img) = e3966b7f4a74f834166bd1372e2a912455d053c4231d8c5a0d67e97342498c94 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-memstick.img.xz) = b8bea2895ef694960170dd84e5e6b298554ed30e2d0f7dbd7634bf43f510f533 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-mini-memstick.img) = 0dbbc84176d0f6c6af991c9d336634237bf67a75d4af1fc8d3ff47ac56a3595f SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpc64le-mini-memstick.img.xz) = 2df74c9d172722fe683751dcf6c026ffb827b2cea6ad3838981cd0935388d0e0 o 13.0-BETA1 powerpcspe MPC85XXSPE: SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-bootonly.iso) = 72695fd6bf4b71bcc82dabb28082e3c0a2edaf799c2518a67546eb2d6b046fb27a6bc1c906cdabae1b29243c50b0c117c9e4a7586af7872a293ab22c237375b6 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-bootonly.iso.xz) = 1a98a3c7894b77dc141846a30b43bbf1f3685ae3000d06616956294f75014e3877718ee1a0ea6d2445e87b8ca98c38c58b827a980a1bad64d12d126ae0d12f5d SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-disc1.iso) = 851c3c4090e2f82f9373aad6a1d4a4fd21cc7a506b57154a563b5dbf27a52255c64650a6ce671128db25cd4b22f5b0811c26ddd74fec3b7ad0095fca60411bc1 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-disc1.iso.xz) = e3605ac09a772eebce6ccc5d3514a644ce37aece28e91e560bbc188e9e2389dcc89519a8a53107bab29a325bed7fe090ab49eb87574f36c7d8cd3c12c87ff656 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-memstick.img) = 1154e87b15bee04ef6a0894582cf7c78a3e5b55fc94696647d9ebd76fa06b5c80550eb1f9b547fadae30fde80e17f76e09ff9c24a18848d887174a3c6255c498 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-memstick.img.xz) = 195f4ec317a9dad7b45eeb8113c24661a3eaf53b511873dc0de2c964b2025a107499fc461b2bd368b803718b6fae9192e2d921b72658ba2eadf9554b3b682e9b SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-mini-memstick.img) = d0039bf31776b8ef245889a591220197e2a71180408425cdb36b86b2df005cda903dc8860f2982b54a817d0846e8220a0909d7770a22fca30dc397ad58f384b5 SHA512 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-mini-memstick.img.xz) = 618eb8d4e745d47f48fa8c970de4d757a77f28785c6b7e38d9f3a20df89ef40377aaad249cb1bc9ea605ef6bb2e0c8045e28f2bfdf68ca7fd4313adf4dedd54a SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-bootonly.iso) = 76f5639d79a0c1e4215d95352530318110a03166ca9da34e9d852ff9f67cc0ef SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-bootonly.iso.xz) = 255127e2c4753889d1665253dfde7f8619c2f647609899f28233c00077a9d6ca SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-disc1.iso) = 4bb1f9bf539e0e8c72469a99a5e9df5e89eb871f6a8c9a9a7e4912a844a2545f SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-disc1.iso.xz) = 3b7be95c9896efd859e007910a80957609be8c694f3a34a7fc7cd476f3a79ed3 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-memstick.img) = 42a8050c99adf4f2c4740b244d8d90f2f4ccdc61ad4d2cc48fb91dc204760b1c SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-memstick.img.xz) = ba12a0f2a46782c8c4f68705b54680861b9b6faa16358ba4f7f21c5dcea3acfc SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-mini-memstick.img) = 2892b5400858d4e340239e6e6cd5df1dceba5ea82ab9ac28718703abebd1b361 SHA256 (FreeBSD-13.0-BETA1-powerpc-powerpcspe-mini-memstick.img.xz) = f39e4b7337a324a5835b30659b1b7daccbff17379d4781038de19ac40780f2a4 o 13.0-BETA1 aarch64 GENERIC: SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-bootonly.iso) = 63a2bc0dfe17402066473bbdd082658dd3a3fc1ac5ed875d940ddc6bf40436bf7b41c96bc5780bca3015fe95f44666cab70be53200d4251c6680ac7abde3e7e6 SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-bootonly.iso.xz) = ff1e84a2108e88119dd5bf91bc8f5d441444351c5bf84be938f6e913328cdeabf549da2a2435c4dbfe7774e5dce2be6f104870183fdb419540fec1fa4bc886a2 SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-disc1.iso) = 8a60bcbbe4bcdd39d1a4570251e1deeb86025ef42772f3b53c0e07bc574dad4e45a88fdad220b55c8363ce310b78b3a90c4dd94a723090bdd7b48bf77f9d32a0 SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-disc1.iso.xz) = 8b5baaf68a8a28b8f67ee164c3709701e9d9eef319d258e34c44766850e74dee79a127c81e8045aa790b9fd252bf1a4731a43c343b97bd991545e8420245017b SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-memstick.img) = fef845589677952edbf907fa4ac5141ae1846ed5722d3937d9d645f998345acbbcaac6f70d190aa7289ac3e4dfb14639b9db403ac6c63765073d7204b5543d48 SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-memstick.img.xz) = 352ecea05aae653b494264c01efa22e83d5b4a0b00c57290e52810c32cecb842c824ae2a325351c39cd4eae7006ba2e6450188e38b9c4db8c1783bb39d3ed6d2 SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-mini-memstick.img) = 03d74eaff27b31d490559b34558d3c716215fd2efdf657e9cfb8873d24106e90263023466d087794f7bc35c78e88478770276d2641a56a5ddbf17dda1f486e95 SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-mini-memstick.img.xz) = e8d1192fb225307d449fa59609cf1a12584d0020c6d449559b0f75e7c489be6eaa879be6a2501a43bf9fb70bca4df036b6c1addf0749dc9a14f57f76ef8373f1 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-bootonly.iso) = 3238e71a6e8364b18b68ab0269c6f8d951a91923eaa42c3b4d4813520440b797 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-bootonly.iso.xz) = 6230eafd49490036f669df92ac5aed82f6142d8549bfbc49280b20dde5cca7aa SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-disc1.iso) = b995f2fd7cd8435544fe244b650dc5ed32439f218d954a159fbb4cac29833f85 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-disc1.iso.xz) = 6c324da5d670a5f271ab76db9c524f65260f4ebf94591ade6ee7a12283d3ddcc SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-memstick.img) = 1ddb7f68ab7d489660ad752ea142cb869f084277fc8d8b69218c446f1e42d896 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-memstick.img.xz) = 80b715472a0af8a46dc62464b1e883d60f5b866f87fbb52de4becf672e0f3a7f SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-mini-memstick.img) = 5d2de3e0a4a1637ab323a9ef016f6dd57f2b56ddc9721f74a205255917759741 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-mini-memstick.img.xz) = 3604247a611e9bcbc2d3f1b36ab10ff7aad88c0cc12528438ae18376772c8bc1 o 13.0-BETA1 aarch64 RPI: SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-RPI.img.xz) = 6cf88b7ba9253b6a8df59f2072a78b188625f4aa0e6137b26a4c0f7ec3e9f9eb5bfaad492654361a8ae97014d26b76069975b66fca9085852945835ae2976122 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-RPI.img.xz) = e30c920feb978e72bbfc598986c6d3c01ed71a478dad77dd1fee04c29505ca68 o 13.0-BETA1 aarch64 PINE64: SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-PINE64.img.xz) = 752ee9097c499e6dc13937bb089fc491c5c3312063ec78b1b0783c63e6433a781a26296b170aa0885485ee407269d9289e248b76e20779c8b9b53aee8ddb1654 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-PINE64.img.xz) = 5a91d03a05146afb082b86b8f5b7301d557bb069844eb7667b5ee16f0cabe3db o 13.0-BETA1 aarch64 PINE64-LTS: SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-PINE64-LTS.img.xz) = d97c25d08cfcab189f50012b0c968033f2dbf76e78daac5b16a2bcb8e889d6d5e39222f2f23914e196c78591c145aed7daaafd193de92b2a88a8406ce1ca0afa SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-PINE64-LTS.img.xz) = 45338eb1d7544afe896adf1fb8b9a6e8a141759a2d8fcbbdb3af81515c7fe037 o 13.0-BETA1 aarch64 PINEBOOK: SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-PINEBOOK.img.xz) = 8c4353e1736d877ead58867542f292f1712e56538eb21c533cfc97a9462220a0274cd847d26dbef5195ac2bcc833330fb8830ce787e907c326062f5f35b90bc7 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-PINEBOOK.img.xz) = b4894dc8b778cd6a854c80d24d6d47996b125eaa569264b67b8200fd8195484d o 13.0-BETA1 aarch64 ROCK64: SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-ROCK64.img.xz) = 644f0a8d9f7200d3ac0b6db4faf0dbfc50613f12240961abcc897578e477b3a6dbb7d2a0cedb521885d65df07a2439cb26bd681cc228e38ef4daedba11f92e46 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-ROCK64.img.xz) = 034ef0131bd0f32d01304f96f76dcbf0ee680d9dc93e9297443081f5081b82b9 o 13.0-BETA1 aarch64 ROCKPRO64: SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64-ROCKPRO64.img.xz) = 4b38d7fe00c3a257faf3184c3db2c4e5e678f601cbbbcf1d48d55c4cc137c159d47c6efce3a75611c163df6b277f993e12a335b67baae335d80c4c7cea8b8aaf SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64-ROCKPRO64.img.xz) = debfd94cc48f249654fdb73a49ce562956654f593fc1c72f2120e11187932a0e o 13.0-BETA1 riscv64 GENERIC: SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-bootonly.iso) = 81818d20fe06dfe741deb82ca17529b73283ffde1391f08c5b9de8a3a7e22d0c3de46a32a9b860581e86affaaca230b913cdb908164d3fc0fec9e68fefcbb808 SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-bootonly.iso.xz) = 21e7e3650a2dff8aee827330fecf48a968c6474b6581ee0ac95105025284dbc0f209cf05c2210fbcd747805fb34ecf58d2eed933c74b9c4a93d95cb4be9e499a SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-disc1.iso) = bc8921a06d6991663dc27c5202449d4464f44a055fda85df1da1845be2bb6103e95416fbb5538bc1262c6ce32f0786508c89f34ac76fa45203f4a1978e354fc7 SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-disc1.iso.xz) = 0c2a3f213ea3d0e6bda422683b3931533bad1899562708cb08691f099231a391dad444c058e58c4ed0e036933edb4a9bc84622950fed8063abffc028efa5656e SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-memstick.img) = d95b573bb0c6ba2320d7ec397d6584f225219a7faa5860fa7623290672025aa11bca9de11503e866999474211f3be9901f8dc11f11529abc99eacd266fae0b63 SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-memstick.img.xz) = 429aa67a940db90642a839bd3dd16e9efb2a7f4bf192384538f9b1fb576c5c2e0d8a554f3b853004b6ea5bf54518aab5d10980e1b74c0ea9e768d4dafd13a5de SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-mini-memstick.img) = bc338557590df7aa6eb96e27eaadc9eb0bd53532e6f05b19c2780c826d70fe2d26167a0c49df567b80835f112d4767e189c5b5a14f364eafebc7b9310806b5af SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-mini-memstick.img.xz) = 94bd0271f78828d43400994db131cff6d10c03f3ccff399ee3a96e4dbee994709eb5c4622fef7c522a3f169df2c92500736b14e289868169465360bccd2a23ed SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-bootonly.iso) = dcd9e1cd7a1e8d6af7d8163d2ac408fc26ce17221ff4ca6b9b70870e043d7cba SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-bootonly.iso.xz) = af82e89ed6eaed2a5a991704a45393a1908dc90a02433b4802b72fe0087b5a7e SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-disc1.iso) = f41f0747ba0c095a6ce3be5fda27eb1c3285ce353e7f4bb3331297a0b842df7e SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-disc1.iso.xz) = c15fdc66d014abc47f8dd1953795f03d5601230f6f3a6a55362d37e2472e8bca SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-memstick.img) = d37fb196431368bd176f53d86b22bf6740ec20fb4d618bad72d91b3b4ec733cc SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-memstick.img.xz) = bddf105974ead3b64676908ca597c6cf221de827a6470249e45224018642f64b SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-mini-memstick.img) = a4d857f4aec0e521e9cd134c1d235c76ad026eeaed844684e634664c11d5ca56 SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-mini-memstick.img.xz) = 9bfe786bfbc82397dc166d1fc7452d117eb5c09c4abf3a0775b36ac3101f64fe o 13.0-BETA1 riscv64 GENERICSD: SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64-GENERICSD.img.xz) = 953be4b6236dbe049b83eb7db996f2a446140c50e269a245a537c362ccf5ede6ddb8a47e9fc91e578cbbf94942d6d0bccafed3fdc248b5b3905f65123ae83e6a SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64-GENERICSD.img.xz) = d90ebdc6cb56d65e9f7ebb8cce4bf3b7e2d7bc1ab97b2ebc43a75e71cb784564 == VM IMAGE CHECKSUMS == o 13.0-BETA1 amd64: SHA512 (FreeBSD-13.0-BETA1-amd64.qcow2.xz) = 1118319bb7cd71cff147deafefd0abfe4230ac02ed2040383fdf7c3bc8aa9678d0e5fa67671bbc68e60b2e1a543c19e08183e0ff82317534c68d61571eeceb24 SHA512 (FreeBSD-13.0-BETA1-amd64.raw.xz) = 5ad2a13b65066d79a8c2bc3d5b1054688a26bce8d4abdac677d6d63a5d8513e954e57e53ec72bd0aaeea5427d49d9edbf7df82f66a61d50d5b5e2da5eca26d39 SHA512 (FreeBSD-13.0-BETA1-amd64.vhd.xz) = 110bbd8d02efbfaa30fbc8ce24304d25ba7494886d044145acd076976d82f9dbb3f8dd86570df16b71f8eab9395f98f88eec69798d6e1942bb0fee9b9bade69e SHA512 (FreeBSD-13.0-BETA1-amd64.vmdk.xz) = 5d4d1eab20789cb983277b4a8e6dd62c0bb5c9d8284850ac656b1bfb570bd90ee86773a1e5c0499fed50bda68ef95e176a5fc9345528f974f810e97ef24e0eea SHA256 (FreeBSD-13.0-BETA1-amd64.qcow2.xz) = f83ff830a49c6c83bd1c23467be5fb9a6f1a366a76f6703499ab997b732626aa SHA256 (FreeBSD-13.0-BETA1-amd64.raw.xz) = 8e6c2be370b781bc158d100afa6a5b1d4c4abd8f5232b8882f6935bf35daf5b5 SHA256 (FreeBSD-13.0-BETA1-amd64.vhd.xz) = 818b9a407799f191bf9b27804d9a08708d32fdf11540ebf2af70d96a652f21f1 SHA256 (FreeBSD-13.0-BETA1-amd64.vmdk.xz) = 23b7b668f9dafa05bb13b913aa271595ee3f175a55d8b3ccbd86f3b846616ee2 o 13.0-BETA1 aarch64: SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64.qcow2.xz) = 844a6d62664f4464b2a6d2f6baae8aa8273854e4085f264c7ce87ba8148aadc45463892fca0ee4ac994169fbda7b0abd465ea1c5aa78139ed4200f512e4704fe SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64.raw.xz) = e1702079f80a811eeccadf6122451d0138120edaf2e6477d2d0c399a8f4bf20fde8a0532bcc87d2bb8522e777bd6dfdb2b4cec1b91afa52208d1917182a03f82 SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64.vhd.xz) = 15f4d14732c33050170f795d13992936cf4abf5053f81870bb8b90d6e4e7dd1b72086e2c6830fe7d05442ca493d7986bfb40f8cc3791f714a7bdbd6f93ed459c SHA512 (FreeBSD-13.0-BETA1-arm64-aarch64.vmdk.xz) = 8a0c7bb86912d5fa3fa3fc9df1cb996f96f06b96342cdf718c78efc2e6113e3156d95652ff96a08452b7d28577229cfe50e5ca23eed6a1b6db4b134faf4ba8f7 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64.qcow2.xz) = 755d755df5999f1327289cdd857b18c984548539eda560d067d68d58d331b8c9 SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64.raw.xz) = d4340c4eb7554a3fe8adb4c286f4e5b35571cef5031865c994754fa667b7fd6a SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64.vhd.xz) = a0798c7d1cbcb5cdc72b9cfffeb8167dae69f410594d7a89a0b535b3ab44167c SHA256 (FreeBSD-13.0-BETA1-arm64-aarch64.vmdk.xz) = 60f7d49df310ccde3051e6d2d4f9ff9fec035caca16aac57b30bed72b498484a o 13.0-BETA1 riscv64: SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64.qcow2.xz) = 461b97491838876e781a1a10d2441157a1308b10727d0a5e1b00c00cd3e00e56704c24d3881d7cb59180088c3710e4e52d87316690e8d2ee9505592d78b1a101 SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64.raw.xz) = 82ca0e735ff93ae5fbb4dec136090c301fb807f116688b87990070ec08164bfbbf119b8d287a895f6731fc5e949b3fa3b16c19612f768faea11cdaa53901827c SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64.vhd.xz) = 69f1aa83e82e59a87caa018d7121f8f21cc90e836ded7e49c242d66fedbbef1bcd6cb4608fe08d84646b1b3e55d2346c910046a499f82ecc375405af76ccec4a SHA512 (FreeBSD-13.0-BETA1-riscv-riscv64.vmdk.xz) = 6dfed31ea3ff6f1139961959451b05bd7099c156b84c7fa1601d9c0dd9a1dece341c1efcb254fd21f4ed5d0a48979ef8ef7878dfdefb1e699577b89cbccea574 SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64.qcow2.xz) = f64efde6eea6fd3be84e3cd2624a35ebed191f2ed277317e8ba69eb7eb1352db SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64.raw.xz) = b3d55a4ac0dbd2bfca5b002152acb555a7ef8270c0c8d85e712d5763d6f22e52 SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64.vhd.xz) = 54d052d49069599b3e924edbc33eee13973e634f198973d5b487a92995adbafc SHA256 (FreeBSD-13.0-BETA1-riscv-riscv64.vmdk.xz) = 97869d367af1e916fb645ac284d0d869774f9388b5aa0b857a5345e23d6113b8 o 13.0-BETA1 amd64 BASIC-CI: SHA512 (FreeBSD-13.0-BETA1-amd64-BASIC-CI.raw.xz) = a6dc7547cf522615f22e52b49f850ba940ccccc655002a0a2bee05268a9d753749076df419f4be9fff67f3083bd4c81ebefe62cf8c479dc6370d702a79df88cf SHA256 (FreeBSD-13.0-BETA1-amd64-BASIC-CI.raw.xz) = 05920bdbcba8eff6ac0e473a83559d7364d46ce1c2a4cf3d85b2132dac067632 Regards, Glen Love FreeBSD? Support this and future releases with a donation to the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmAe7/MACgkQAxRYpUeP 4pNH+g//VE4AgBE6qTeO6+uQc6BsSfqzQ0tQ+YxjOmhh+FA6TyA65uTwTCx9gLBZ t91Dw4IgIgHFAlPTprmfYtWqp3OTRFzKZH18aGk4W+ma0zukGuVZhyMK8hoTamtJ EcaZN+tG3v9Rqrnojy7PvLMMveb3y1E6w+cEhtako485hbAkneOOMVrcVpMbeKMN ValWvctcNZ6pBegMTHvwYsqRBroJi7q7xYSmQjX51++xsvSxBKwuUikXYwI42Q5t tmY/URhcpCEaAyF9NY7A1cwgzfY0dq7eQ26CfTI9x9oxoRd+y/ymZRyDgy0/LASp az97e5IdjHBfYPQMCRTNrWAcsRfaDSMwjkKzlycLkCfisf//vxDT2lER29nnFcjo BlaihHj84XB75Jhgw3f6VHRHp1Vb9BijNgrRalwrt0KASEDkXkoWip0vHTXdTgor vpraZx9kTICBuWwJN0Zn9TbIX+iKeLSCLGKYTvDCumn6iWl6NVJVBFWZJ1T+1C+H j3yPmVq6O6sD4hPcqGl51mM9ib/Ro518pE2Bc7UW1BhZ/cZGw8rMJnbKrA/TZBXR HYjU6D4AcMIpXs+sV9di1KIZzO8M0IHMHLuR9Esu+QhTHrkpRMZg2YJqRBZxb3Ev kfsvaaHsK4g3pRANNMsu3yZBy8iaNh5mlUnd3xIdFmFi/CS+G10= =kToP -----END PGP SIGNATURE-----