From owner-freebsd-current@freebsd.org Sun Oct 11 02:18:58 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 82DE643248A for ; Sun, 11 Oct 2020 02:18:58 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-to1can01on0603.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5d::603]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C858c2x3nz4X0w for ; Sun, 11 Oct 2020 02:18:56 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ngmQX71yzy7J91zUkBsHHSQhmCfFG6k33eNyFtWQF0GajYKo2l6TlJBYFH7DGM0jFNjtGTiIoRmjk/to2YBDQt4jz+k9LdFStNy91kyGrP32eBdakLjWIFsEXQRM5DGSAZ6FxE/wM5utH95b4q5TvBO+8yCXrNaZDp44qhPhELlEa0/WwBVHPxDnN4IPYUmUxQX6/Dsia5ivwyqIu5SCzzi+82hq/xMiPz10XwUc7pfZT/sFjAxJgdy62p0WF4TLDEceTJ2N7Uair3GJXOGW1ROKW0Fjwxw9zO4jwFOi8vGkb/XtVRqk8t5upyaAnf3iqMHDtvw1PuXiAsP8MQLM2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0YI7rI92fvRUU8mjUkuoXFDjPI6ZX9W1huggFBayCio=; b=F9yvcPfN6cdAW70mOoFrx5RSS7ChrvFqpoCO1hWXThi8PObWdn+15mvBiC1ZTfM/MaIcZ9CFda/RC3AEqF77/PNsvrvc36b2pK7YNoMtSmQksSwTex8ebB99s9XQ0+ttuQc9w6QsjzEj35h2RPMv0USzDgn0bcpIKWF/u4iIDlGK/nXwqJRbZqdGX5MxSqh12Dx/DQqvF8uns277toOdR/tfvkybBmxi7DuhFQeEmww9RgaDqzdePuSOi34zeurZ2FFyDpB6ArN4Y4t+ywSI1mMeS/Ox1bPX226q1EcgDXNOlLdx3NOl7+eEpOu37zQyhHg0tIvHHIA79ypegFThLg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none Received: from YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:24::27) by YTOPR0101MB0940.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:1c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.28; Sun, 11 Oct 2020 02:18:51 +0000 Received: from YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM ([fe80::687f:d85a:a0a3:bd20]) by YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM ([fe80::687f:d85a:a0a3:bd20%6]) with mapi id 15.20.3455.028; Sun, 11 Oct 2020 02:18:51 +0000 From: Rick Macklem To: "freebsd-current@freebsd.org" CC: Meowthink Subject: RFC: gssd needs /usr mounted to start up Thread-Topic: RFC: gssd needs /usr mounted to start up Thread-Index: AQHWn3MMVFQ2dhtCu0yNdRi2qRdLdw== Date: Sun, 11 Oct 2020 02:18:51 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 13daabaa-20af-467e-9fc6-08d86d8bfee3 x-ms-traffictypediagnostic: YTOPR0101MB0940: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: vZJrUphHq4D01um5cUCx+6NaX98T+/UlmL2FQGFspiRC6PxzqCmq+QUS5R/Ot3ZOfA1HSpW53MqTP3O8wTR++bbH0E0ObWrbJWuH5ym2/3ZZ3MUi13HUocKfzeJc4sv8wW2hiR9pYBaEO/GA6hSk2zZo1F1fV9bWjbYWEPi4gA49KQkjaUIJERi/PX/NQelAEQXUTUNq3gMgbPcLAylEtIa8hlIiukrAg08sufH/lHgUaNVbX9XpBqBYN3gaS2KhMlgmG9Y42JJltC74gDkYwmI1SePGX5d6ih8nsZugv25Ecft2ZbhRVhBwHg4dgAasARuhLDvX7WwKIkbeZtgjNg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(366004)(136003)(376002)(346002)(39850400004)(396003)(4326008)(2906002)(6916009)(55016002)(186003)(478600001)(9686003)(52536014)(33656002)(83380400001)(8936002)(316002)(786003)(66556008)(8676002)(64756008)(66446008)(86362001)(66946007)(76116006)(66476007)(6506007)(5660300002)(7696005)(71200400001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: dsziHGE9gmY61tB1lJD/AxElM1CEtLmg1iyW4ZFFbBTA525xwHykXZ1U1O1Y1xmS0vRV8nFzdeNJyjL27Gw1a6VrWadpyn+bc9T8eZuXwhURWn2tXuojlbEFUDRriaHydEccoMHYK0mdbuSpcGkdTUGFojdkI1Hc7jxeVXlsPTS320iIMTaL4s7PUZD4doxASro/PgOo925xaNGblLwgiyU9dKLHlMMFTXzRJ4ch5y/OaAvUh+wJHKgtEs6Gtd7V0mHnYDtzWkH9Jzq2wSWonnETI4gdAxDYdmb22yxXA41c6Ob++pRoMuwh6xt5HxK0Zo/oMAnAHn0+0ws9dE54ZxY/wWMCZ8XOCEiysL2W8K0XVI8bDRmMm2rwELLR32911n6h463utFfVy8u4ZkmzCsya9aqiNnVH5TzD4N01iJz3nhgFI1TE8hqhZ9ZbNLxJkjOiOPEQ5vfgWEJTQS2CgIwkpqHb+OhOgfgSgY14OWCCJXYnwXv5wA+V2owCBhmAFsgeCKcEt575evRsTYbddn8OqTVppl2oLp1yuIeoPbjm7KZstiQJ8zurbVn0EPYlRVcdy7uZRUWk/cvMWFQfvTpn9F720TMoLVt+YtyhMXzbvFYtJhW5YrUK/NlcmL+bgqh7OkoXTHJPjugreiASHzlNi8nP38NhnItBy4Kseit48U6jP9//ZWNGtn74RJcJ1q6ejUpWicZQ1/2Va5Xoxg== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YTBPR01MB3966.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 13daabaa-20af-467e-9fc6-08d86d8bfee3 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Oct 2020 02:18:51.5998 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: tQZMUzDE/+ZHQKtW4oNKzjrh32RkXlk+dZI7sp1FrEBamZx4AU64ZF1iV+IstL4uXOVDVQ7KZxs1Pu27rwtq2g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTOPR0101MB0940 X-Rspamd-Queue-Id: 4C858c2x3nz4X0w X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.17 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.013]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; NEURAL_HAM_LONG(-1.01)[-1.014]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.14)[-1.144]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; FREEMAIL_CC(0.00)[gmail.com]; MAILMAN_DEST(0.00)[freebsd-current]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2020 02:18:58 -0000 Meowthink reported a problem on freebsd-hackers@ where the=0A= gssd would not start up because /usr was not yet mounted.=0A= (I moved the discussion here, hoping to catch more comments.)=0A= =0A= He has a separately mounted /usr and, recently, gssd was failing=0A= to start since /usr was not yet mounted when /etc/rc.d/gssd was=0A= executed.=0A= Looking at /etc/rc.d/gssd, this is not surprising, since the REQUIRED=0A= line only lists "root" as a requirement.=0A= I can see a couple of things that can be done, but no obvious ideal=0A= solution:=0A= (A) - Add "mountcritlocal" to the REQUIRED line, which is what=0A= Meowthink has done.=0A= This seems harmless and works for the case of a local filesystem=0A= /usr, but does not work if /usr is an NFS mounted file system.=0A= =0A= (B) - Add both "mountcritlocal" and "mountcritremote" to the=0A= REQUIRED line.=0A= This would also fix the case of an NFS mounted /usr, but it also=0A= implies that all NFS entries in /etc/fstab that uses "sec=3Dkrb5[ip= ]"=0A= would also need the "late" option specified.=0A= =0A= I am thinking that (A) can be done and MFC'd, since it shouldn't=0A= break anything (or cause a POLA violation).=0A= Maybe (B) can be done for head/FreeBSD13 with an entry in the=0A= Release notes, indicating the need for "late" on NFS entries using=0A= "sec=3Dkrb5[ip]" in /etc/fstab. (It would result in a POLA violation if=0A= MFC'd, since "sec=3Dkrb5[ip]" entries in /etc/fstab would break until=0A= "late" is added.)=0A= =0A= I am interested in a solution for this, in part, because the daemons=0A= for NFS over TLS have the same problem.=0A= =0A= Any ideas/suggestions, rick=0A= ps: I thought of moving gssd to /sbin, but it uses several libraries,=0A= including Kerberos ones, that are in /usr/lib.=0A= =0A= From owner-freebsd-current@freebsd.org Sun Oct 11 10:21:39 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C9FF943F0C0 for ; Sun, 11 Oct 2020 10:21:39 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C8HsZ3T17z40tG for ; Sun, 11 Oct 2020 10:21:38 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: by mail-wr1-x432.google.com with SMTP id n18so15527996wrs.5 for ; Sun, 11 Oct 2020 03:21:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:references:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=jn6iTtaQLi4OvZAenPZX2YBAN0eDYdMLjqzygm5t184=; b=F6PErld3SgxdtnRP0yj190ow5PCfA/P/NeizK1OMsuv21Fc3swpdH9kgaD4rUCK2HW Hy1DWaI9nqC2r+FQJKnfIkirsRy8ltpWW5lxpcxTyt7GtvS0nJ6bB5/ZwGiC/oaMl58A aFpdmw+CyNBqaL82L6ARUYG4UMcHwCMhUw7TQvxljvrIBOUyo49LFT6lt3GIcuu9P+2g aZ5k7Ko3adOQAMo3R3M49PCjHNRO2+Kn/0XCizwXA2bJWkVOGMNrU/stCtHqpxC2PNYt apjFEVXUNV9Q3PedbzaDSC4NfRf4NyqzBroY7agpoKUKtZXDvXgOx7jv2rvraz9mmiWC 3Z/w== X-Gm-Message-State: AOAM532AzgwdH+Mt7ub/cle9L7CGSiRGMlmOO6rb5pq2lheFRNJcWs+X zj+opN1XoAseSYTTHrd9f5uIQwZHIIKIuQ== X-Google-Smtp-Source: ABdhPJxFQj0KndKkHjJUEF+tjxJ3lQRb68asZjuvpByT8akpYU323+0tuKANz6JN1VWTFdevmAQ/vA== X-Received: by 2002:adf:9c19:: with SMTP id f25mr9343463wrc.366.1602411695998; Sun, 11 Oct 2020 03:21:35 -0700 (PDT) Received: from [192.168.1.11] (79-66-147-78.dynamic.dsl.as9105.com. [79.66.147.78]) by smtp.gmail.com with ESMTPSA id u15sm18537088wml.21.2020.10.11.03.21.34 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 11 Oct 2020 03:21:34 -0700 (PDT) Subject: Re: zpool export: umount failed, device inexplicably busy From: Graham Perrin To: freebsd-current@freebsd.org References: Message-ID: <4125dad7-b1ea-bf20-4996-6a138ff202eb@gmail.com> Date: Sun, 11 Oct 2020 11:21:34 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4C8HsZ3T17z40tG X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.61 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.70)[-0.698]; RECEIVED_SPAMHAUS_PBL(0.00)[79.66.147.78:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.976]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.93)[-0.931]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::432:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2020 10:21:39 -0000 On 06/10/2020 21:29, Graham Perrin wrote: > lsof finds no open file. > > How else might I tell why the device is busy? > > Mobile hard disk drive, USB. The same problem this morning, ---- root@momh167-gjp4-8570p:~ # zpool export Transcend cannot unmount '/Volumes/t500': umount failed root@momh167-gjp4-8570p:~ # lsof /Volumes/t500/ lsof: WARNING: device cache mismatch: /dev/usb/0.5.2 lsof: WARNING: /root/.lsof_momh167-gjp4-8570p was updated. root@momh167-gjp4-8570p:~ # lsof /Volumes/t500/ root@momh167-gjp4-8570p:~ # usbconfig -d 0.5 dump_all_desc ugen0.5: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)   bLength = 0x0012   bDescriptorType = 0x0001   bcdUSB = 0x0300   bDeviceClass = 0x0009    bDeviceSubClass = 0x0000   bDeviceProtocol = 0x0003   bMaxPacketSize0 = 0x0009   idVendor = 0x0424   idProduct = 0x5434   bcdDevice = 0x3082   iManufacturer = 0x0002    iProduct = 0x0003    iSerialNumber = 0x0001  <1239567>   bNumConfigurations = 0x0001  Configuration index 0     bLength = 0x0009     bDescriptorType = 0x0002     wTotalLength = 0x001f     bNumInterfaces = 0x0001     bConfigurationValue = 0x0001     iConfiguration = 0x0000      bmAttributes = 0x00e0     bMaxPower = 0x0000     Interface 0       bLength = 0x0009       bDescriptorType = 0x0004       bInterfaceNumber = 0x0000       bAlternateSetting = 0x0000       bNumEndpoints = 0x0001       bInterfaceClass = 0x0009        bInterfaceSubClass = 0x0000       bInterfaceProtocol = 0x0000       iInterface = 0x0000       Endpoint 0         bLength = 0x0007         bDescriptorType = 0x0005         bEndpointAddress = 0x0081          bmAttributes = 0x0013          wMaxPacketSize = 0x0002         bInterval = 0x0008         bRefresh = 0x0000         bSynchAddress = 0x0000       Additional Descriptor       bLength = 0x06       bDescriptorType = 0x30       bDescriptorSubType = 0x00        RAW dump:        0x00 | 0x06, 0x30, 0x00, 0x00, 0x02, 0x00 root@momh167-gjp4-8570p:~ # zpool export Transcend cannot unmount '/Volumes/t500': umount failed root@momh167-gjp4-8570p:~ # umount /Volumes/t500 umount: unmount of /Volumes/t500 failed: Device busy root@momh167-gjp4-8570p:~ # lsof /Volumes/t500/ root@momh167-gjp4-8570p:~ # ---- When this last occurred, I could export the pool after signing out from KDE. From owner-freebsd-current@freebsd.org Sun Oct 11 10:29:31 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A31C743F30E for ; Sun, 11 Oct 2020 10:29:31 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C8J2f53cZz41HW for ; Sun, 11 Oct 2020 10:29:30 +0000 (UTC) (envelope-from ohartmann@walstatt.org) X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from hermann.fritz.box ([78.54.17.89]) by mail.gmx.com (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MxDou-1kGKFU1cYC-00xcWC for ; Sun, 11 Oct 2020 12:29:28 +0200 Date: Sun, 11 Oct 2020 12:29:20 +0200 From: "Hartmann, O." To: FreeBSD CURRENT Subject: r366623: lock order reversal in ZFS Message-ID: <20201011122920.60b91a18@hermann.fritz.box> Organization: walstatt.org MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/Ms0EgWPaOJULScusCc//Abz"; protocol="application/pgp-signature"; micalg=pgp-sha256 X-Provags-ID: V03:K1:xNyIXrWaP2bfr+59OWXreCUaPNBMQ9LoQ2l/UcWUYo3qfqEEwlJ 3EO2vDcbvJOaneh5++ZHTovdIBAckl/RcfGc7phbnd9fp/wHWAkeQYLYl8Mpe3x3eSbTmaC qmb+/nihiH0Lmzora3Lu0KyAQPjRNZGckLeKachnd2OwuvZ5p+oX0d/0+EAWnmg06yuzuM9 tmjMkcmNHS35W/6xOSZwQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:4u5wd+54F6o=:HQ29PPoLf3BqMn6+Zlh6y+ SJdvBIBCEaD02pMKU4MAnMknaHA7CYo67YFfvD0bIJr65I6CURDVmVeaSyWGzNf/N+5DSI9mv YExGenqRNCtRPA58nane3HbAVz2doYMrqICYRRGZyi0ubjuZsphuSYVcjj7G3d4GGg/6jmVFZ w8Lap7LKXXPG+D4cHY8wQth7LLD6AdAQZ+0cGwxuAsLTNKjATQFgomy8gPL0UnuUMyFYBd5o9 A9EuRcFIIhBfVj3DIV26NS30F2/xvFCmMLBvbs9DwG+nVosjY14s3PXQHK4SUDL63q6Jgqt0t SY0w2AKwVt0+UrVMO8Ijkl8difWHOSiri5TZwxaSkteVOVW9a0XeXviZlBmvgkt80ZX0nXMx3 9OV5Uc9e4EbUpAfvbFbjUDk1B1AlO6fmV7DecITbGB8fprg9Nhte877xjeGewZBeMOAlPjO8Y JmMGhSOqaIz/xnae35EkessxaS5JtU6La4YGS4Fi0Jt9jvicDnHeqWW/eAtvyCVgDr6/QPU2y QyRHp7SnuzgB5akcRvFMH6FsIHWaWZKhl2QT0oGgzSZOVvX9CEobrsMrWIslg0qDhy1xmc6XR wW9qr1QHTEcDoMPmrmy/SHmFhW6h3RS71Rfs7froVLUesmp0vd62NscsAVveBWq84+BTAsgRk s95yGW1MA+vtKVjz/uEBtaZNaSshaPMvu/CO/jh8BsMxuk3UiqmO0ym3PnBGfT2Sf5A9XhI8H aO4t0O+Jgx7f3MfktUW+aTkkfmojk9SgmZK2Tnqfdzzmi9yGReoBxeagfSjxGzkQQFtkp9nHa PLAhxEqoSW/xr9JF1ZioqMplIxXUl4Nmqgn5M8XIPG/X6e4nKq3rw52n+cFwC8nqO67hh2LCm YOQPoGqQavM+uO7PfdCogcMavC9iNsMDgS6UZN/eDzpiA2AClz7VVtqFAiOuEPt1naNUerncR Z0DTYRCfYdSL07ibpiJoNiD9Jmgcr4oQ/yKtF1ZJuNqj2cdXdRRpcHRd/MEfYvGKLITvt58/g MyX3yj9tnp4VAzVSEebb9JXbJvr7aoRbnf08mNhFQdS7BReq29sjXP5rBn7YkUU1P6+VDRlhR TYOlkAOxWU/NXTY6Jfn79TBOYe58ZVfOhsXlsB6XaLYCg/SaNIOiQZ4iXkqRU816wYezd/eOc ccYr1F9eSR0rOvCQRLtIxqPuo95smVp/cpzCKB0f8sTVlR5ZY9NQhQSeu2WfZvNK4LMl14Uek Xr2QE3LP9+0jQogon+SVNCbFqfnWFbqLj3csurQ== X-Rspamd-Queue-Id: 4C8J2f53cZz41HW X-Spamd-Bar: - X-Spamd-Result: default: False [-1.96 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RWL_MAILSPIKE_POSSIBLE(0.00)[212.227.17.22:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.04)[-0.035]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[78.54.17.89:received]; DMARC_NA(0.00)[walstatt.org]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; NEURAL_HAM_SHORT(-0.52)[-0.521]; NEURAL_HAM_MEDIUM(-0.90)[-0.902]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; MAILMAN_DEST(0.00)[freebsd-current]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.17.22:from] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Oct 2020 10:29:31 -0000 --Sig_/Ms0EgWPaOJULScusCc//Abz Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On a recent CURRENT bos acting as poudriere host, we face after a crash under heavy load and extensive swap usage now this lock order reversal warning on the console: lock order reversal: 1st 0xfffff8007bdd9e00 zfs (zfs, lockmgr) @ /usr/src/sys/kern/vfs_mount.c:1018 2nd 0xfffff8034abfc070 devfs (devfs, lockmgr) @ /usr/src/sys/kern/vfs_mount.c:1029 lock order zfs -> devfs attempted at: #0 0xffffffff80f3380d at witness_checkorder+0xebd #1 0xffffffff80e8ba4b at lockmgr_xlock+0xab #2 0xffffffff80fc3e04 at _vn_lock+0x54 #3 0xffffffff80fa0e34 at vfs_domount+0xde4 #4 0xffffffff80f9f598 at vfs_donmount+0x898 #5 0xffffffff80f9ecb9 at sys_nmount+0x69 #6 0xffffffff81461255 at amd64_syscall+0x135 #7 0xffffffff814335ee at fast_syscall_common+0xf8 System: FreeBSD 13.0-CURRENT #71 r366623: Sun Oct 11 08:59:17 CEST 2020 --Sig_/Ms0EgWPaOJULScusCc//Abz Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQSy8IBxAPDkqVBaTJ44N1ZZPba5RwUCX4LegAAKCRA4N1ZZPba5 R94nAQDOamlyZwm+nddju1WV87cuWK+j08x9x0ZxNJk4LF272wEA6o8vSABFXfbh of3edFBeER69ffg28qoUS5oSwpNLAgQ= =DYBM -----END PGP SIGNATURE----- --Sig_/Ms0EgWPaOJULScusCc//Abz-- From owner-freebsd-current@freebsd.org Mon Oct 12 09:22:32 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8B32643115A for ; Mon, 12 Oct 2020 09:22:32 +0000 (UTC) (envelope-from ltning@anduin.net) Received: from mail.modirum.com (mail.modirum.com [185.42.170.44]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C8tVv3vMqz3fNK for ; Mon, 12 Oct 2020 09:22:31 +0000 (UTC) (envelope-from ltning@anduin.net) Received: by mail.modirum.com with esmtpsa (TLS1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.94 (FreeBSD)) (envelope-from ) id 1kRu2G-000Di3-Gr for freebsd-current@freebsd.org; Mon, 12 Oct 2020 09:22:28 +0000 From: Eirik =?ISO-8859-1?Q?=D8verby?= To: freebsd-current@freebsd.org Subject: Re: Wake from sleep kinda broken-ish? (ThinkPad Carbon X1 6th gen) Date: Mon, 12 Oct 2020 11:22:27 +0200 Message-ID: <2162563.mfXeX5GmMH@carbon8.weirdr.net> In-Reply-To: References: <0b2f0f19-490b-4bb5-52b3-201e24ebeaee@anduin.net> <39ceba1e-3e9e-a861-ab71-d376969990c4@FreeBSD.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-SA-Authenticated: Yes X-Rspamd-Queue-Id: 4C8tVv3vMqz3fNK X-Spamd-Bar: - X-Spamd-Result: default: False [-1.40 / 15.00]; RCVD_TLS_ALL(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.89)[-0.888]; DMARC_NA(0.00)[anduin.net]; NEURAL_SPAM_SHORT(0.20)[0.202]; RCVD_IN_DNSWL_NONE(0.00)[185.42.170.44:from]; NEURAL_HAM_MEDIUM(-0.92)[-0.915]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:62248, ipnet:185.42.170.0/24, country:NO]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-current]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 09:22:32 -0000 On Wednesday, September 16, 2020 9:05:43 AM CEST Warner Losh wrote: > I too can report this for my Lenovo Yoga running code as of September 13, > but with manu's latest drm... It used to work fine, but my last build on > the system was from May. Most likely a new panic in that code path, but > I've not chased down further... So I got a gen8 to play with, and the list of grievances is long - but I have one observation that may be of interest: The gen8 would be usable (at least seemingly so) with a 13-kernel from lat 2019 or very early 2020. Then around the end of January - I've bisected it down to around Jan 24, give or take, it would start wedging _hard_ after a minute or two of heavy load (compiling, cat /dev/random, that sort of thing). It was a problem prior to that too but it was _much_ harder to trigger, at least based on my tests this weekend. The "solution" is to add hint.hwpstate_intel.0.disabled="1" to /boot/loader.conf. This obviously has disastrous impact on battery life. The emt module takes over, so power management is a lot more rudimentary (powerd now does nearly nothing, while powerd++ kills interactivity). Battery life is much shorter than on my gen6, and it gets hotter. BUT: This thing - gen8 - would get stuck in the acpi_beep before adding this to loader.conf. After adding the hint, I have not had a single failure when resuming. It's behaving much better than my gen6. Worth noting that I patched the ig4 driver to allow it to find the I2C device (just adding the PCI device IDs to the end of the lists already there), so iichid would work and give me a trackpad. Latest drm-devel-kmod works well, but install the xf86 intel driver and set "AccelMethod" "sna" in the appropriate xorg config file. /Eirik From owner-freebsd-current@freebsd.org Mon Oct 12 12:06:10 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 98D354361D0 for ; Mon, 12 Oct 2020 12:06:10 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C8y7k3YJlz47GW; Mon, 12 Oct 2020 12:06:10 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f0bbc003d7e076980bc4f6e.dip0.t-ipconnect.de [IPv6:2003:cd:5f0b:bc00:3d7e:769:80bc:4f6e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id D0E072BA02; Mon, 12 Oct 2020 12:06:09 +0000 (UTC) (envelope-from se@freebsd.org) To: Allan Jude , Matthew Macy , FreeBSD CURRENT From: Stefan Esser Subject: OpenZFS: L2ARC shrinking over time? Message-ID: <3788e4d8-df6b-4096-ce58-d931583609b4@freebsd.org> Date: Mon, 12 Oct 2020 14:06:07 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qBFwC29KAXWa1h0iNxXA2uULdfOExG8cV" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 12:06:10 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qBFwC29KAXWa1h0iNxXA2uULdfOExG8cV Content-Type: multipart/mixed; boundary="EVSe7Sem6P7TU3OMxURkpOfLYPjLzLAsZ"; protected-headers="v1" From: Stefan Esser To: Allan Jude , Matthew Macy , FreeBSD CURRENT Message-ID: <3788e4d8-df6b-4096-ce58-d931583609b4@freebsd.org> Subject: OpenZFS: L2ARC shrinking over time? --EVSe7Sem6P7TU3OMxURkpOfLYPjLzLAsZ Content-Type: multipart/mixed; boundary="------------CA03FD83979A02E25EE78596" Content-Language: en-US This is a multi-part message in MIME format. --------------CA03FD83979A02E25EE78596 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable After the switch-over to OpenZFS in -CURRENT I have observed that the=20 L2ARC shrinks over time (at a rate of 10 to 20 MB/day). My system uses a 1 TB NVME SSD partitioned as 64 GB of SWAP (generally unused) and 256 GB of ZFS cache (L2ARC) to speed up reads from a 3*6 TB raidz1. (L2ARC persistence is great, especially on a system that is used for development and rebooted into the latest -CURRENT about once per week!) After reboot, the full cache partition is available, but even measured only minutes apart the reported site of the L2ARC is declining. The following two values were obtained just 120 seconds apart: kstat.zfs.misc.arcstats.l2_asize: 273831726080 kstat.zfs.misc.arcstats.l2_asize: 273831644160 [After finishing the text of this mail I have checked the value of that variable another time - maybe 10 minutes have passed ... kstat.zfs.misc.arcstats.l2_asize: 273827724288 That corresponds with some 4 MB lost over maybe 10 minutes ...] I have first noticed this effect with the zfs-stats command updated to support the OpenZFS sysctl variables (committed to ports a few days ago). After 6 days of uptime the output of "uptime; zfs-stats -L" is: 12:31PM up 6 days, 7 mins, 2 users, load averages: 2.67, 0.73, 0.36 ------------------------------------------------------------------------ ZFS Subsystem Report Mon Oct 12 12:31:57 2020 ------------------------------------------------------------------------ L2 ARC Summary: (HEALTHY) Low Memory Aborts: 87 Free on Write: 5.81 k R/W Clashes: 0 Bad Checksums: 0 IO Errors: 0 L2 ARC Size: (Adaptive) 160.09 GiB Decompressed Data Size: 373.03 GiB Compression Factor: 2.33 Header Size: 0.12% 458.14 MiB L2 ARC Evicts: Lock Retries: 61 Upon Reading: 9 L2 ARC Breakdown: 12.66 m Hit Ratio: 75.69% 9.58 m Miss Ratio: 24.31% 3.08 m Feeds: 495.76 k L2 ARC Writes: Writes Sent: 100.00% 48.94 k ------------------------------------------------------------------------ After a reboot and with the persistent L2ARC now reported to be available again (and filled with the expected amount of data): 13:24 up 28 mins, 2 users, load averages: 0,09 0,05 0,01 ------------------------------------------------------------------------ ZFS Subsystem Report Mon Oct 12 13:24:56 2020 ------------------------------------------------------------------------ L2 ARC Summary: (HEALTHY) Low Memory Aborts: 0 Free on Write: 0 R/W Clashes: 0 Bad Checksums: 0 IO Errors: 0 L2 ARC Size: (Adaptive) 255.03 GiB Decompressed Data Size: 633.21 GiB Compression Factor: 2.48 Header Size: 0.14% 901.41 MiB L2 ARC Breakdown: 9.11 k Hit Ratio: 35.44% 3.23 k Miss Ratio: 64.56% 5.88 k Feeds: 1.57 k L2 ARC Writes: Writes Sent: 100.00% 205 ------------------------------------------------------------------------ I do not know whether this is just an accounting effect, or whether the usable size of the L2ARC is actually shrinking, but since there is data in the L2ARC after the reboot, I assume it is just an accounting error. But I think this should still be researched and fixed - there might be a wrap-around after several weeks of up-time, and if the size value is not only used for display purposes, this might lead to unexpected behavior. --------------CA03FD83979A02E25EE78596-- --EVSe7Sem6P7TU3OMxURkpOfLYPjLzLAsZ-- --qBFwC29KAXWa1h0iNxXA2uULdfOExG8cV Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAl+ERq8FAwAAAAAACgkQR+u171r99UQl Jgf/RnGPrA8FR++I67qcHPFdpqamtapGofcvnr4qj7yHTJUgtGG75luwmknRFTi7WYnQspdfYzJH 1dVwf8Bt/NpDCQaf3BMIwu+TNwuEGbzKXHBPhKdquSF0dZshOzg59nKQOxC7bIG9lSpEPzbeCu/8 5W4W7bJZdPZTy4B0otukkIXPTMDxZXnndyv1SC1y07+eX1UEjjTuuIeAUxo59dO0fRzd2xnu00dS eKgoooCk7BpF4wVw3KJfM73EZOnwZJtR6VinVYo88gJSmBUNVhLOJSrwP1m6B8NOkP7qUWwdshcv vpEKRG380PXPBXN4uI2k4MfEddqdnsAMNt+4cuEcaA== =86nr -----END PGP SIGNATURE----- --qBFwC29KAXWa1h0iNxXA2uULdfOExG8cV-- From owner-freebsd-current@freebsd.org Mon Oct 12 15:25:57 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1A6D2439FF9 for ; Mon, 12 Oct 2020 15:25:57 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C92ZD0BFMz4LJ4; Mon, 12 Oct 2020 15:25:55 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-qt1-x843.google.com with SMTP id t9so13014637qtp.9; Mon, 12 Oct 2020 08:25:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:references:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=2WwGaLZer7NKZBGu5BOwaOejYMI97GTaJqbRPaCxj28=; b=lu4pQL8lt3jtN2IOYZ7uBbEhggi1RRtIl4b7PpM0OWjIuaASamXlVf/23DbBkH82fp LDCfDf2bARNXbEUqigpIgCD7atAaryRDEA5jkMP+dUso8B+Gc8A/TLYhRZog9j79XESr AxItrP7QGgPifXFoD9llAOb47WmnCxpdn6KOrJhunUsSvCqIe6uzM8vOvhlb5/P3yDY0 59Y7t5pVETQmQD5KtTNCzdLd4ZajPDdCH+TCrl/DJhzQk8qdL4AVUIvto0inAkeR7jPy CwHJr1im3g2h3aVDLKJ6b2jS47CLxBfOCMUSlNsrZlEbmpCPXySHPrjozMMWF8TAKsDd eh7Q== X-Gm-Message-State: AOAM532BxnoUVqTyZ8SgdsowNvVvq3NcacYaCRsSbyYi0va3srquilKQ ZPirnEhxatDTrhO1Ci6+xwfN5AZht+CvJw== X-Google-Smtp-Source: ABdhPJyYa6bC03SAPtG6O9vXEdiT7/4YvKhpf8r9gwshiq3mSKDeXdGOkF/dT6yA+g+Yy0VLNA2DNg== X-Received: by 2002:ac8:a87:: with SMTP id d7mr10812928qti.336.1602516354983; Mon, 12 Oct 2020 08:25:54 -0700 (PDT) Received: from spectre.mavhome.dp.ua (104-55-12-234.lightspeed.knvltn.sbcglobal.net. [104.55.12.234]) by smtp.gmail.com with ESMTPSA id f3sm12684325qtp.57.2020.10.12.08.25.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Oct 2020 08:25:53 -0700 (PDT) Sender: Alexander Motin Subject: Re: GPF on boot with devmatch From: Alexander Motin To: Warner Losh Cc: Xin LI , FreeBSD Current , Warner Losh References: <02fa309e-9467-f741-8092-974bfc145c9a@FreeBSD.org> <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> Autocrypt: addr=mav@FreeBSD.org; prefer-encrypt=mutual; keydata= mQENBFOzxAwBCADkPrax0pI2W/ig0CK9nRJJwsHitAGEZ2HZiFEuti+6/4UVxj81yr4ak/4g 9bKUyC7rMEAp/ZHNhd+MFCPAAcHPvtovnfykqE/vuosCS3wlSLloix2iKVLks0CwbLHGAyne 46lTQW74Xl/33c3W1Z6d8jD9gVFT/xaVzZ0U9xdzOmsYAZaAj4ki0tuxO9F7L+ct9grRe7iP g8t9hai7BL4ee3VRwk2JXnKb7UvBiVITKYWKz1jRvZIrjPokgEcCLOSlv7x/1kjuFnj3xWZU 7HSFFT8J93epBbrSSCsYsppIk2fZH41kaaFXsMQfTPH8wkeM6qwrvOh4HiQM08R+9tThABEB AAG0IUFsZXhhbmRlciBNb3RpbiA8bWF2QEZyZWVCU0Qub3JnPokBVwQTAQoAQQIbAwULCQgH AwUVCgkICwUWAwIBAAIeAQIXgAIZARYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMKuBQkN McyiAAoJEIMYw5VbqyJ/tuUIAOG3ONOSNYqjK4eTZ1TVh9jdUBAhWk5nhDFnODN49Wj0AbYm 7aIqy8O1hnCDSZG5LttjSAo3UfXJZDKQM0BLb0gpRMBnAYqO6tdolLNqAbPGJBnGoPjsh24y 6KcbDaNnis+lD4GwPXwQM+92wZGhCUFElPV9NciZGVS65TNIgk7X+yEjjhD1MSWKKijZ1r9Z zIt4OzUTxxNOvzdlABZS88nNRdJkatOQJPmFdd1mpP6UzTNCiLUo1pIqOEtJgvVVDYq5WHY6 tciWWYdmZG/tIBexJmv2mV2OLVjXR6ZeKmntVH14H72/wRHJuYHQC+r5SVRcWWayrThsY6jZ Yr4+raS5AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6Z AXgDtmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8Flv mI/c40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt 3ytU8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZ R1EdEIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm5 9R8AEQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczM AAoJEIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLq A6xe6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHu uC5vgPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15Gc sS9YcQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9 TevwGsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCg lz65AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6ZAXgD tmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8FlvmI/c 40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt3ytU 8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZR1Ed EIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm59R8A EQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczMAAoJ EIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLqA6xe 6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHuuC5v gPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15GcsS9Y cQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9Tevw Gsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCglz4= Message-ID: <385234b0-0ca4-cec6-6a2e-6dec0a84b0f0@FreeBSD.org> Date: Mon, 12 Oct 2020 11:25:52 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.0 MIME-Version: 1.0 In-Reply-To: <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4C92ZD0BFMz4LJ4 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.68 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_HAM_LONG(-0.99)[-0.988]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.47)[-0.473]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::843:from]; NEURAL_HAM_MEDIUM(-1.02)[-1.019]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 15:25:57 -0000 On 05.10.2020 17:39, Alexander Motin wrote: > On 05.10.2020 17:20, Warner Losh wrote: >> On Mon, Oct 5, 2020 at 12:36 PM Alexander Motin > > wrote: >> >> I can add that we've received report about identical panic on FreeBSD >> releng/12.2 of r365436, AKA TrueNAS 12.0-RC1: >> https://jira.ixsystems.com/browse/NAS-107578 .  So it looks a) pretty >> rate (one report from thousands of early adopters and none in our lab), >> and b) it is in stable/12 too, not only head. >> >> Thanks! I'll see if I can recreate here....  But we're accessing the >> sysctl tree from devmatch to get some information, which should always >> be OK (the fact that it isn't suggests either a bug in some driver >> leaving bad pointers, or some race or both)...  It would be nice to know >> which nodes they were, or to have a kernel panic I can look at... > > All we have now in this case is a screenshot you may see in the ticket. > Also previously the same user on some earlier version of stable/12 > reported other very weird panics on process lock being dropped where it > can't be in some other sysctls inside kern.proc, so if we guess those > are related, I suspect there may be some kind of memory corruption > happening, but have no clue where. Unfortunately we have only textdumps > for those. So if Xin is able to reproduce it locally, it may be our > best chance to debug it, at least this specific issue. The TrueNAS user has confirmed that preloading the modules as Xin did workarounds the problem for him also. -- Alexander Motin From owner-freebsd-current@freebsd.org Mon Oct 12 15:27:31 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8364543A728 for ; Mon, 12 Oct 2020 15:27:31 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [199.48.133.146]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C92c16nPmz4LWb for ; Mon, 12 Oct 2020 15:27:29 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from disco.vangyzen.net (unknown [70.97.188.230]) by smtp.vangyzen.net (Postfix) with ESMTPSA id 4971D56477 for ; Mon, 12 Oct 2020 10:27:23 -0500 (CDT) Subject: Re: ZFS crash -- zvol_geom_bio_getattr called when volmode=dev To: freebsd-current@freebsd.org References: <1dfaf797-f23b-af97-d2d3-efa0f5bb88d9@vangyzen.net> <95636bbf-2367-f226-f462-d5581581c18f@vangyzen.net> <74ba295a-1f67-5109-0a21-9662eac0334b@vangyzen.net> From: Eric van Gyzen Message-ID: <9a140b74-de79-60d5-f5c8-b2a2676dbbea@vangyzen.net> Date: Mon, 12 Oct 2020 10:27:17 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.1 MIME-Version: 1.0 In-Reply-To: <74ba295a-1f67-5109-0a21-9662eac0334b@vangyzen.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4C92c16nPmz4LWb X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.02 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_SPAM_SHORT(0.10)[0.100]; FREEFALL_USER(0.00)[eric]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.92)[-0.919]; ARC_NA(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; DMARC_NA(0.00)[vangyzen.net]; NEURAL_HAM_MEDIUM(-0.90)[-0.901]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:36236, ipnet:199.48.132.0/22, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 15:27:31 -0000 On 10/9/20 7:54 PM, Eric van Gyzen wrote: > On 10/9/20 6:27 PM, Ryan Moeller wrote: >> >> On 10/9/20 6:22 PM, Alan Somers wrote: >>> This sounds like it might be a regression introduced by the OpenZFS >>> merge. >>> Have you compared vdev_geom.c in OpenZFS vs the old version? >>> -Alan >> >> >> I don't think vdev_geom.c is involved, we're taking a wrong path in >> zvol_os.c because >> it seems the volume is created using the default volmode and later >> changed to volmode=dev. > > Yes, you're on the right track.  I tried this several times on a VM and > it eventually hit the window: > > # zfs create -s -V 20G -o primarycache=none -o volmode=dev > head_root/testvol > zvol_create_minor_impl:1250[1]: Creating ZVOL head_root/testvol... > zvol_create_minor_impl:1371[1]: ZVOL head_root/testvol created. > > > Fatal trap 12: page fault while in kernel mode Let's track this in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250297 Eric From owner-freebsd-current@freebsd.org Mon Oct 12 18:13:34 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6A0A243DA89 for ; Mon, 12 Oct 2020 18:13:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C96Hd5Fr1z4WMN for ; Mon, 12 Oct 2020 18:13:33 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x836.google.com with SMTP id r8so14166328qtp.13 for ; Mon, 12 Oct 2020 11:13:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KBwo4i43GiWoJw5v9KV4HdS3PDa5YY/wJDQGjMAAkqU=; b=uCJLRpRtu5t0CFUFA6N+usgVa3Iy8FOKrEnIhjBCuG9MsaQF5y3EttU9q/KKqhn+N1 lxE2BKl9wXMeEqQXNNHNPcF+FyUskMp+DFpfR7w5hkDltdLwjIXFlS4BYIQGkIxvhthQ Y+8LQ3IHscP+nZIoLm54a5HGxGcypwecqlS2faniunlNy8FVQJOv6komoXTlK0TW5D77 9DHXyK0+wUYMXA2OTmAxj7szs4Hu1nM9ZM07aLxylCq5MVbIYFPI4Odyoz3NYN90UpLb f3dZLzJWCGmF1K8AD3UC6cq02yI2dAWC7m7ISKgXTOYwHUz0qxV50dzs4QWT81fdtIMS 09Wg== X-Gm-Message-State: AOAM533EfNMlh6ZAuyhUCOrJ5TUCEwlZEROaEQGe15WbOyLsNQbcGjhe Xy/NbowOfSQ1C3VBPQOV8JUjSoeMzMsG+w726L9cIw== X-Google-Smtp-Source: ABdhPJxrAajCEA67t9rddFjiTI5Mi22xEuZ5dIIwcIsN4CtIiHGMh8uwODMe0CkKwkcLJiVs0z4U5Q0XXEVIeEctmB4= X-Received: by 2002:ac8:3178:: with SMTP id h53mr11160960qtb.187.1602526412498; Mon, 12 Oct 2020 11:13:32 -0700 (PDT) MIME-Version: 1.0 References: <02fa309e-9467-f741-8092-974bfc145c9a@FreeBSD.org> <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> In-Reply-To: <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> From: Warner Losh Date: Mon, 12 Oct 2020 12:13:21 -0600 Message-ID: Subject: Re: GPF on boot with devmatch To: Alexander Motin Cc: Xin LI , FreeBSD Current , Warner Losh X-Rspamd-Queue-Id: 4C96Hd5Fr1z4WMN X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.33 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(0.00)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-0.65)[-0.649]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-0.70)[-0.698]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; URIBL_RED(3.50)[ixsystems.com:url]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; HAS_ANON_DOMAIN(0.10)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::836:from]; NEURAL_HAM_SHORT(-0.22)[-0.223]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-current]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 18:13:34 -0000 On Mon, Oct 5, 2020 at 3:39 PM Alexander Motin wrote: > On 05.10.2020 17:20, Warner Losh wrote: > > On Mon, Oct 5, 2020 at 12:36 PM Alexander Motin > > wrote: > > > > I can add that we've received report about identical panic on FreeBSD > > releng/12.2 of r365436, AKA TrueNAS 12.0-RC1: > > https://jira.ixsystems.com/browse/NAS-107578 . So it looks a) > pretty > > rate (one report from thousands of early adopters and none in our > lab), > > and b) it is in stable/12 too, not only head. > > > > Thanks! I'll see if I can recreate here.... But we're accessing the > > sysctl tree from devmatch to get some information, which should always > > be OK (the fact that it isn't suggests either a bug in some driver > > leaving bad pointers, or some race or both)... It would be nice to know > > which nodes they were, or to have a kernel panic I can look at... > > All we have now in this case is a screenshot you may see in the ticket. > Also previously the same user on some earlier version of stable/12 > reported other very weird panics on process lock being dropped where it > can't be in some other sysctls inside kern.proc, so if we guess those > are related, I suspect there may be some kind of memory corruption > happening, but have no clue where. Unfortunately we have only textdumps > for those. So if Xin is able to reproduce it locally, it may be our > best chance to debug it, at least this specific issue. > That's totally weird. Xin Li's traceback lead to code I just rewrote in current, while this code leads to code that's been there for a long time and hasn't been MFC'd. This suggests that Xin Li's backtrace isn't to be trusted, or there's two issues at play. Both are plausible. I've fixed a minor signedness bug and a possible one byte overflow that might have happened in the code I just rewrote. But I suspect this is due to something else related to how children are handled after we've raced. Maybe there's something special about how USB does things, because other buses will create the child early and the child list is stable. If USB's discovery code is adding something and is racing with devd's walking of the tree, that might explain it... It would be nice if there were some way to provoke the race on a system I could get a core from for deeper analysis.... Warner > -- > Alexander Motin > From owner-freebsd-current@freebsd.org Mon Oct 12 22:23:38 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B4A77429DB1; Mon, 12 Oct 2020 22:23:38 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9CrB4Cy9z3VXb; Mon, 12 Oct 2020 22:23:38 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1129) id 78F3BA636; Mon, 12 Oct 2020 22:23:38 +0000 (UTC) Date: Mon, 12 Oct 2020 22:23:38 +0000 From: Li-Wen Hsu To: freebsd-testing@freebsd.org Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: FreeBSD CI Weekly Report 2020-10-11 Message-ID: <20201012222338.GA87671@freefall.freebsd.org> Reply-To: freebsd-testing@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1602541418; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=ZNsyj0tFP9GddJCFGdcMnM/vCgccPlPSS9KOo3jddrk=; b=BYtBPNQw5UIU+fZGEljjzFpyoOCtsiLYWiCeKnbVcW7mqHCwxnOweSkiBMDd3oDfc3+Igk izTOiOrzOYewIfjEOct1DeL3qN6WgKhwrDdiJ7/Vu1I2WA6INioTrN20QNdmxMDqRK1oCB S0gDfFuf1hOq+Ut4JtcjCnKu5+snZUK0nNEK06KFFhvamOg3k3qUQFieaVbj/2BCDYCq1D 6TbMM/dkE1bn4ahewGH5O+rVgxj/E/mXgpX8xNxdmZrP0NRbm8XStJFHy/T4y3+QraGgFV 5N5AXMw7fVM/izCBllf+yJB/ZwHjUV8QH4rqjEpKvXXOvvDmjBEHysFmBKs0cg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1602541418; a=rsa-sha256; cv=none; b=uR2OSRtochWPM3Fylj8J45tCSzVjVZsMjBzhwqZTzSIjxgWP4tBU58RrU3fozZj+yAeRLK 5fMiyd73u5X1swehBAfgpC1PYUKgRrnyV9zBpr+CxwTw9XpHV7DB1mgJRW3ij3R47a/5Bf ye0y1dvHR8uRwRT71Ez/6KXINfPO35OAiywvy6NwuHJjkrWI6dxUaiKClPlZeC+d4EypMV 2VmMW6TivbaXh0yQmr1P/vebgytexqI03h/sFiA/COmX7MWb3uXfRlqNAMVMvpqyi5v92O S6jXtV0qo5dtUcciM68jiCVQZNnSjF/mSco3gAfCoyZGQ+BQQagXCnOfEmMqNA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 22:23:38 -0000 FreeBSD CI Weekly Report 2020-10-11 =================================== Here is a summary of the FreeBSD Continuous Integration results for the period from 2020-10-05 to 2020-10-11. During this period, we have: * 1809 builds (94.6% (+0.7) passed, 5.4% (-0.7) failed) of buildworld and buildkernel (GENERIC and LINT) were executed on aarch64, amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, sparc64 architectures for head, stable/12, stable/11 branches. * 213 test runs (70.5% (+51.9) passed, 28.6% (-52.8) unstable, 0.9% (+0.9) exception) were executed on amd64, i386, riscv64 architectures for head, stable/12, stable/11 branches. * 34 doc and www builds (100% passed) Test case status (on 2020-10-11 23:59): | Branch/Architecture | Total | Pass | Fail | Skipped | | ------------------- | ---------- | ---------- | ------ | -------- | | head/amd64 | 7995 (+96) | 7891 (+93) | 0 (-2) | 104 (+5) | | head/i386 | 7993 (+96) | 7869 (+93) | 0 (-2) | 124 (+5) | | 12-STABLE/amd64 | 7630 (0) | 7570 (0) | 0 (0) | 60 (0) | | 12-STABLE/i386 | 7628 (0) | 7563 (0) | 0 (0) | 65 (0) | | 11-STABLE/amd64 | 6912 (0) | 6858 (0) | 0 (0) | 54 (0) | | 11-STABLE/i386 | 6910 (0) | 6857 (+3) | 0 (0) | 53 (-3) | (The statistics from experimental jobs are omitted) If any of the issues found by CI are in your area of interest or expertise please investigate the PRs listed below. The latest web version of this report is available at https://hackmd.io/@FreeBSD-CI/report-20201011 and archive is available at https://hackmd.io/@FreeBSD-CI/ , any help is welcomed. ## News * Capsicum tests have been turned fine grained results. Now each test case reports individually: https://ci.freebsd.org/job/FreeBSD-head-amd64-test/lastCompletedBuild/testReport/junit/sys.capsicum/functional/ The test are executed with root by default now and some previously skipped test cases are being executed now. ## Failing test cases ### HEAD * sys.capsicum.functional.Capability__NoBypassDAC https://bugs.freebsd.org/250178 * sys.capsicum.functional.Pdfork__OtherUserForked https://bugs.freebsd.org/250179 ## Failing jobs * https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc6_build/ There are still mutiple errors when building with gcc6, error log available at https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc6_build/lastCompletedBuild/console See also: https://lists.freebsd.org/pipermail/svn-src-all/2020-September/202307.html ## Regressions * lib.libexecinfo.backtrace_test.backtrace_fmt_basic starts failing on amd64 after r360915 https://bugs.freebsd.org/246537 * lib.msun.ctrig_test.test_inf_inputs starts failing after llvm10 import https://bugs.freebsd.org/244732 Needs to check if llvm11 import fixes this. * Lock-order reversals triggered by tests under sys.net.if_lagg_test.* on i386 https://bugs.freebsd.org/244163 Discovered by newly endabled sys.net.* tests. ([r357857](https://svnweb.freebsd.org/changeset/base/357857)) * sys.net.if_lagg_test.lacp_linkstate_destroy_stress panics i386 kernel https://bugs.freebsd.org/244168 Discovered by newly endabled sys.net.* tests. ([r357857](https://svnweb.freebsd.org/changeset/base/357857)) Fix committed as https://svnweb.freebsd.org/changeset/base/364220 , needs more verification. * lib.libbe.be_create.* and sbin.bectl.bectl_test.* https://bugs.freebsd.org/249055 & https://bugs.freebsd.org/249229 OpenZFS' zfs.ko can't be loaded in non-64bit platforms ## Failing and Flaky tests (from experimental jobs) * https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/ * cddl.usr.sbin.dtrace.common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237641 * common.ip.t_dtrace_contrib.tst_ipv4localsctp_ksh * common.ip.t_dtrace_contrib.tst_localsctpstate_ksh * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/ * Total 681 tests, 524 success, 47 failures, 110 skipped, see https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/ for more details * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_ltp/ * Total 3749 tests, 2289 success, 647 failures, 813 skipped ## Disabled Tests * sys.fs.tmpfs.mount_test.large https://bugs.freebsd.org/212862 * sys.fs.tmpfs.link_test.kqueue https://bugs.freebsd.org/213662 * sys.kqueue.libkqueue.kqueue_test.main https://bugs.freebsd.org/233586 * sys.kern.ptrace_test.ptrace__PT_KILL_competing_stop https://bugs.freebsd.org/220841 * lib.libc.regex.exhaust_test.regcomp_too_big (i386 only) https://bugs.freebsd.org/237450 * sys.netinet.socket_afinet.socket_afinet_bind_zero https://bugs.freebsd.org/238781 * sys.netpfil.pf.names.names * sys.netpfil.pf.synproxy.synproxy https://bugs.freebsd.org/238870 Workaround committed and need to verify. * sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger https://bugs.freebsd.org/239292 * sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger https://bugs.freebsd.org/239397 * sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger https://bugs.freebsd.org/239399 * sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger https://bugs.freebsd.org/239425 * sys.sys.qmath_test.qdivq_s64q https://bugs.freebsd.org/240219 * sys.kern.ptrace_test.ptrace__getppid https://bugs.freebsd.org/240510 * lib.libc.sys.stat_test.stat_socket https://bugs.freebsd.org/240621 * lib.libarchive.functional_test.test_write_filter_zstd https://bugs.freebsd.org/240683 * lib.libcasper.services.cap_dns.dns_test.main lib.libcasper.services.cap_net.net_test.* https://bugs.freebsd.org/241435 * local.kyua.* (31 cases) & local.lutok.* (3 cases) on 11-i386 https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/2278/testReport/ * sys.kern.ptrace_test.ptrace__procdesc_reparent_wait_child https://bugs.freebsd.org/243605 * sys.kern.ptrace_test.ptrace__parent_wait_after_attach https://bugs.freebsd.org/244055 * sys.kern.ptrace_test.ptrace__parent_exits_before_child https://bugs.freebsd.org/244056 * sys.net.if_lagg_test.witness (i386) https://bugs.freebsd.org/244163 * PipePdfork.WildcardWait in sys.capsicum.capsicum-test.main https://bugs.freebsd.org/244165 * sys.net.if_lagg_test.lacp_linkstate_destroy_stress (i386) https://bugs.freebsd.org/244168 * sys.netinet6.frag6.frag6_07.frag6_07 https://bugs.freebsd.org/244170 * sys.netinet.fibs_test.udp_dontroute6 https://bugs.freebsd.org/244172 * sys.netpfil.pf.nat.exhaust https://bugs.freebsd.org/244703 Workaround committed and need to verify. * sys.geom.class.gate.ggate_test.ggated (i386) https://bugs.freebsd.org/244737 * sys.kern.sysv_test.msg https://bugs.freebsd.org/233649 * sys.capsicum.functional.ForkedOpenatTest_WithFlagInCapabilityMode___ * sys.capsicum.functional.OpenatTest__WithFlag https://bugs.freebsd.org/249960 * sys.capsicum.functional.Capability__NoBypassDAC https://bugs.freebsd.org/250178 * sys.capsicum.functional.Pdfork__OtherUserForked https://bugs.freebsd.org/250179 ## Issues ### Cause build fails * https://bugs.freebsd.org/233769 Possible build race: ld: error: unable to find library -lgcc_s ### Cause kernel panics * https://bugs.freebsd.org/238870 sys.netpfil.pf.names.names and sys.netpfil.pf.synproxy.synproxy cause panic ### Open * https://bugs.freebsd.org/237641 Flakey test case: common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237656 "Freed UMA keg (rtentry) was not empty (18 items). Lost 1 pages of memory." seen when running sys/netipsec tests * https://bugs.freebsd.org/238781 sys.netinet.socket_afinet.socket_afinet_bind_zero does not work when mac_portacl(4) loaded * https://bugs.freebsd.org/239292 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger * https://bugs.freebsd.org/239397 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger * https://bugs.freebsd.org/239399 Flakey test case: sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger * https://bugs.freebsd.org/239425 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger * https://bugs.freebsd.org/241662 Flakey test case: lib.libarchive.functional_test.test_fuzz_iso9660 * https://bugs.freebsd.org/246443 sys.net.if_clone_test.epair_stress sometimes exceeds timeout limit but not caught by kyua * https://bugs.freebsd.org/247510 sys.net.if_lagg_test.status_stress panics kernel on i386 ### Others * [Tickets related to testing@](https://preview.tinyurl.com/y9maauwg) From owner-freebsd-current@freebsd.org Mon Oct 12 22:43:01 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5E4DB42A6AF for ; Mon, 12 Oct 2020 22:43:01 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "anubis.delphij.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9DGX1dMjz3Wrm; Mon, 12 Oct 2020 22:42:59 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from odin.corp.delphij.net (unknown [IPv6:2601:646:8601:f4a:a0bc:5958:aee5:d913]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id 3E3573C9FC; Mon, 12 Oct 2020 15:42:53 -0700 (PDT) Reply-To: d@delphij.net Subject: Re: GPF on boot with devmatch To: Warner Losh , Alexander Motin Cc: Xin LI , FreeBSD Current , Warner Losh References: <02fa309e-9467-f741-8092-974bfc145c9a@FreeBSD.org> <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> From: Xin Li Message-ID: <0ab2fc9a-3f60-c375-03ee-1f10c32acc2f@delphij.net> Date: Mon, 12 Oct 2020 15:42:52 -0700 User-Agent: Thunderbird MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="4h3QLtY5N89prGwiclIaknmUEBbV7jjfY" X-Rspamd-Queue-Id: 4C9DGX1dMjz3Wrm X-Spamd-Bar: / X-Spamd-Result: default: False [0.81 / 15.00]; HAS_REPLYTO(0.00)[d@delphij.net]; RCVD_VIA_SMTP_AUTH(0.00)[]; XM_UA_NO_VERSION(0.01)[]; R_SPF_ALLOW(0.00)[+mx]; HAS_ATTACHMENT(0.00)[]; URIBL_RED(3.50)[ixsystems.com:url]; RCPT_COUNT_FIVE(0.00)[5]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[delphij.net:+]; DMARC_POLICY_ALLOW(0.00)[delphij.net,reject]; HAS_ANON_DOMAIN(0.10)[]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:6939, ipnet:64.62.128.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.72)[-0.722]; R_DKIM_ALLOW(0.00)[delphij.net:s=m7e2]; FREEFALL_USER(0.00)[delphij]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.89)[0.886]; NEURAL_HAM_LONG(-0.86)[-0.862]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; REPLYTO_DOM_EQ_FROM_DOM(0.00)[]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 22:43:01 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --4h3QLtY5N89prGwiclIaknmUEBbV7jjfY Content-Type: multipart/mixed; boundary="oKbvK48qPVSnnoqk5JOc2KVAOQJnS4ZYK"; protected-headers="v1" From: Xin Li Reply-To: d@delphij.net To: Warner Losh , Alexander Motin Cc: Xin LI , FreeBSD Current , Warner Losh Message-ID: <0ab2fc9a-3f60-c375-03ee-1f10c32acc2f@delphij.net> Subject: Re: GPF on boot with devmatch References: <02fa309e-9467-f741-8092-974bfc145c9a@FreeBSD.org> <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> In-Reply-To: --oKbvK48qPVSnnoqk5JOc2KVAOQJnS4ZYK Content-Type: multipart/mixed; boundary="------------ED3A605B99096EC90529902D" Content-Language: en-US This is a multi-part message in MIME format. --------------ED3A605B99096EC90529902D Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 10/12/20 11:13, Warner Losh wrote: >=20 >=20 > On Mon, Oct 5, 2020 at 3:39 PM Alexander Motin > wrote: >=20 > On 05.10.2020 17:20, Warner Losh wrote: > > On Mon, Oct 5, 2020 at 12:36 PM Alexander Motin > > >> wrote: > > > >=C2=A0 =C2=A0 =C2=A0I can add that we've received report about ide= ntical panic on > FreeBSD > >=C2=A0 =C2=A0 =C2=A0releng/12.2 of r365436, AKA TrueNAS 12.0-RC1: > >=C2=A0 =C2=A0 =C2=A0https://jira.ixsystems.com/browse/NAS-107578 > .=C2=A0 So it looks = a) pretty > >=C2=A0 =C2=A0 =C2=A0rate (one report from thousands of early adopt= ers and none in > our lab), > >=C2=A0 =C2=A0 =C2=A0and b) it is in stable/12 too, not only head. > > > > Thanks! I'll see if I can recreate here....=C2=A0 But we're acces= sing the > > sysctl tree from devmatch to get some information, which should a= lways > > be OK (the fact that it isn't suggests=C2=A0either a bug in some = driver > > leaving bad pointers, or some race or both)...=C2=A0 It would be = nice > to know > > which nodes they were, or to have a kernel panic I can look at...= >=20 > All we have now in this case is a screenshot you may see in the tic= ket. > =C2=A0Also previously the same user on some earlier version of stab= le/12 > reported other very weird panics on process lock being dropped wher= e it > can't be in some other sysctls inside kern.proc, so if we guess tho= se > are related, I suspect there may be some kind of memory corruption > happening, but have no clue where.=C2=A0 Unfortunately we have only= textdumps > for those.=C2=A0 So if Xin is able to reproduce it locally, it may = be our > best chance to debug it, at least this specific issue. >=20 >=20 > That's totally weird.=C2=A0 >=20 > Xin Li's=C2=A0traceback lead to code I just rewrote in current, while t= his > code leads to code that's been there for a long time and hasn't been > MFC'd. This suggests that Xin Li's=C2=A0backtrace isn't to be trusted, = or > there's two issues at play. Both are plausible. I've fixed a minor > signedness bug and a possible one byte overflow that might have happene= d > in the code I just rewrote. But I suspect this is due to something else= > related to how children are handled after we've raced. Maybe there's > something special about how USB does things, because other buses will > create the child early and the child list is stable. If USB's discovery= > code is adding something and is racing with devd's=C2=A0walking of the = tree, > that might explain it...=C2=A0 It would be nice if there were some way = to > provoke the race on a system I could get a core from for deeper analysi= s.... There might be some other players; I just don't have a lot of time recently to shoot it down; the system is somewhat critical for my internal network so I can't afford to have long downtimes (it's fine if I have controllable downtimes, e.g. if you want me to deliberately panic the system and get some debugging data, please feel free to ask as long as I can continue to boot at the end of experiment :)) =46rom what I was observing, it seems to be some kind of race condition between the USB stack and sysctl tree; however, the race might be delicate as I never successfully provoked the panic on my laptop, which also runs -CURRENT). If you want to add some instruments to the code, please let me know and I'll get the tree patched to try to catch it. Cheers, > Warner > =C2=A0 >=20 > --=20 > Alexander Motin >=20 --------------ED3A605B99096EC90529902D-- --oKbvK48qPVSnnoqk5JOc2KVAOQJnS4ZYK-- --4h3QLtY5N89prGwiclIaknmUEBbV7jjfY Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEEceNg5NEMZIki80nQQHl/fJX0g08FAl+E2+wFAwAAAAAACgkQQHl/fJX0g0/3 SRAAhOAgJXlvZyWqzg9ZFi9mtul7GpLmMSDSh3osBpufBKF40AvZyXyHBRslkG04EiQ6NZ3/8i0+ JG9mE+Zl7dpYGjcD8ucbOsnK2DyrHubDL91gccUYWig7i2fSmO+Sa1Ow/vEZZ0oSRSoqJ0+QfVoS e96mPLWYeo4yOA/ANmK47YfC0t6NB9Sc4NDo3pvmCORE9yBE3UKDGVUXMIC4nr9jy/h9vSosH6We 4HtEKG1txrckIv8bN7J+kPAADvLzgnKuCaHr7iSDPm+ak3sbbabGnSY9TRkDdh3BvZ3z58rrTsui TSZTa/+LJ4KLK9NhjKsKNzp2CJQL/iJQ+ZiweT1tZ0pXaFJUGBlpgSuA7Iwt4yrMIF7UAicRXLYq IwCaCLLctb79ixHxHPuhR19sthCpVbMiFOdad73SBfvhQ78RPfknr9Wljjr9nYcaFQ7qMMSivlwW wFRXBRLkyXvw6XLlmX+l1UcfoZjmM+GaQ3qoTriYnStJlim63UumsSpeBSvNlBmhJtTUEJj2V6aL mz1jd8pgkVU6Hv9gJErxNv3EuLHGeC1SkHfTEK+rduuXDtpN005Onm//9cGWetnZ61hKZXbbGq6L 4CT22HAGJtBnmcXBZEAiHihHJmRv5qQB883yQsC7EQebHOd99FTb57GhYfB70c4j6JWYrc2p9BUq sKw= =DRvH -----END PGP SIGNATURE----- --4h3QLtY5N89prGwiclIaknmUEBbV7jjfY-- From owner-freebsd-current@freebsd.org Tue Oct 13 11:27:52 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B572B43E01B for ; Tue, 13 Oct 2020 11:27:52 +0000 (UTC) (envelope-from shigeru@os-hackers.jp) Received: from mailssl04.asahi-net.or.jp (mailssl04.asahi-net.or.jp [202.224.55.63]) by mx1.freebsd.org (Postfix) with ESMTP id 4C9YF36npMz4Zhh for ; Tue, 13 Oct 2020 11:27:51 +0000 (UTC) (envelope-from shigeru@os-hackers.jp) Received: from localhost (w142149.ppp.asahi-net.or.jp [121.1.142.149]) (Authenticated sender: WJ8S-YMMT) by mailssl04.asahi-net.or.jp (Postfix) with ESMTPSA id 0CBE24005A for ; Tue, 13 Oct 2020 20:21:55 +0900 (JST) Date: Tue, 13 Oct 2020 20:21:48 +0900 (JST) Message-Id: <20201013.202148.2154404302288800973.shigeru@os-hackers.jp> To: freebsd-current@freebsd.org Subject: zfs panic when 'make buildworld buildkernel' From: YAMAMOTO Shigeru X-Mailer: Mew version 6.8 on Emacs 27.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4C9YF36npMz4Zhh X-Spamd-Bar: +++++ X-Spamd-Result: default: False [5.86 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,body]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.00)[0.001]; DMARC_NA(0.00)[os-hackers.jp]; VIOLATED_DIRECT_SPF(3.50)[]; NEURAL_HAM_SHORT(-0.02)[-0.021]; NEURAL_SPAM_LONG(0.88)[0.881]; MID_CONTAINS_FROM(1.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:4685, ipnet:202.224.32.0/19, country:JP]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2020 11:27:52 -0000 Hi, I try to 'make buildworld buildkernel' at full ZFS environment. But I can't finished buildworld/buildkernel without panic. Anyone have same trouble? uname -a: ``` FreeBSD jenkins-02.current.os-hackers.jp 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r366657: Tue Oct 13 13:07:15 JST 2020 root@jenkins-02.current.os-hackers.jp:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 ``` panic message: ``` login: panic: VERIFY(tid) failed cpuid = 2 time = 1602582381 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe002abaa9f0 vpanic() at vpanic+0x182/frame 0xfffffe002abaaa40 spl_panic() at spl_panic+0x3a/frame 0xfffffe002abaaaa0 taskq_dispatch() at taskq_dispatch+0xe8/frame 0xfffffe002abaaae0 arc_prune_async() at arc_prune_async+0x3f/frame 0xfffffe002abaab00 arc_evict_cb() at arc_evict_cb+0x1f6/frame 0xfffffe002abaab60 zthr_procedure() at zthr_procedure+0x8f/frame 0xfffffe002abaabb0 fork_exit() at fork_exit+0x80/frame 0xfffffe002abaabf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe002abaabf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic [ thread pid 19 tid 100070 ] Stopped at kdb_enter+0x37: movq $0,0x10b0116(%rip) db> ``` Latest stable kernel, I have, is: ``` FreeBSD jenkins-02.current.os-hackers.jp 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r363746: Sat Aug 1 14:25:06 JST 2020 root@freebsd-00.current.os-hackers.jp:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 ``` Thanks, --- YAMAMOTO Shigeru From owner-freebsd-current@freebsd.org Tue Oct 13 12:09:45 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CD0C6440166 for ; Tue, 13 Oct 2020 12:09:45 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9Z9P195tz4dsl for ; Tue, 13 Oct 2020 12:09:44 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-wm1-x343.google.com with SMTP id d3so21502417wma.4 for ; Tue, 13 Oct 2020 05:09:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=4gECDIowWhxqYjPzjTS1jy6nII03camEIZxC4XK4tf0=; b=QgKsVBIHGO1LphL+nfmNBKd4ZUTQUuUnvkI/SbbKOY5/TA+90T3o2RpDbzQtEZUTs3 J8YSnyiiTpvqVlXhcQFJXB6MtzzKoh1UkE/5wt4EOF8iC0OgE1EIDfiDFMtzwmytn16z 5xHgW9j316975rg1bUUblDM+p2nX5Y6D53q4hOWslxrA9wpXV3MUVDrU8pX1hK0VtLP0 bGqvQ98wqQxkjETNR+Ppw20ZgOrY7ejoiipvZDIWMFz9zjqRftfHSybHZLqxLodfOCA4 axx15ucq9pjqf5UW2PyGRb5oC+wEvDzApNaSB7Nic03H03Jv3MsJ1BjQ7FBDi3PKb1as EoLA== X-Gm-Message-State: AOAM530zjhl7tSxs72IkKndYs5TRt7cxXBjfW0MYwldD3r9ccmNGZMHP uQ9V6dQm8HQqN0qgECQxzgnCNUHt5mu6O8k3l685jZZDoEU= X-Google-Smtp-Source: ABdhPJxrYHoam1qvkQ8jZ9abyq7GEMTf9wNrQ9Z3EdJTGcrMC8pEzczzXtT4po631ANT2rF4y+3bbuJVLIBenO62tAc= X-Received: by 2002:a7b:c20d:: with SMTP id x13mr16230447wmi.83.1602590982331; Tue, 13 Oct 2020 05:09:42 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:adf:c448:0:0:0:0:0 with HTTP; Tue, 13 Oct 2020 05:09:41 -0700 (PDT) In-Reply-To: <20201013.202148.2154404302288800973.shigeru@os-hackers.jp> References: <20201013.202148.2154404302288800973.shigeru@os-hackers.jp> From: Mateusz Guzik Date: Tue, 13 Oct 2020 14:09:41 +0200 Message-ID: Subject: Re: zfs panic when 'make buildworld buildkernel' To: YAMAMOTO Shigeru Cc: freebsd-current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4C9Z9P195tz4dsl X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.81 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.969]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_HAM_LONG(-0.98)[-0.976]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::343:from]; NEURAL_HAM_SHORT(-0.87)[-0.869]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2020 12:09:45 -0000 On 10/13/20, YAMAMOTO Shigeru wrote: > > Hi, > > I try to 'make buildworld buildkernel' at full ZFS environment. > But I can't finished buildworld/buildkernel without panic. > Anyone have same trouble? > > uname -a: > ``` > FreeBSD jenkins-02.current.os-hackers.jp 13.0-CURRENT FreeBSD 13.0-CURRENT > #0 r366657: Tue Oct 13 13:07:15 JST 2020 > root@jenkins-02.current.os-hackers.jp:/usr/obj/usr/src/amd64.amd64/sys/GENERIC > amd64 > ``` > > panic message: > ``` > login: panic: VERIFY(tid) failed > > cpuid = 2 > time = 1602582381 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe002abaa9f0 > vpanic() at vpanic+0x182/frame 0xfffffe002abaaa40 > spl_panic() at spl_panic+0x3a/frame 0xfffffe002abaaaa0 > taskq_dispatch() at taskq_dispatch+0xe8/frame 0xfffffe002abaaae0 > arc_prune_async() at arc_prune_async+0x3f/frame 0xfffffe002abaab00 > arc_evict_cb() at arc_evict_cb+0x1f6/frame 0xfffffe002abaab60 > zthr_procedure() at zthr_procedure+0x8f/frame 0xfffffe002abaabb0 > fork_exit() at fork_exit+0x80/frame 0xfffffe002abaabf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe002abaabf0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > KDB: enter: panic > [ thread pid 19 tid 100070 ] > Stopped at kdb_enter+0x37: movq $0,0x10b0116(%rip) > db> > ``` > The issue is pretty apparent: taskqid_t tqid = atomic_fetchadd_int(&tqidnext, 1); this eventually wraps to 0 and then you get the crash. Probably the thing to do is ot bump it to 64 bit and 0-check on other platforms. -- Mateusz Guzik From owner-freebsd-current@freebsd.org Tue Oct 13 12:15:23 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2C6074409CD for ; Tue, 13 Oct 2020 12:15:23 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wm1-x341.google.com (mail-wm1-x341.google.com [IPv6:2a00:1450:4864:20::341]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9ZHt1cQVz4fT7 for ; Tue, 13 Oct 2020 12:15:22 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-wm1-x341.google.com with SMTP id b127so8711880wmb.3 for ; Tue, 13 Oct 2020 05:15:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ltRL3MW9HseMYVfdDGKkNIRO4KjV8UlncHuty/aSyO4=; b=Rk33FcWWHMI3bhNCPQsBhC4vE12peUbuB/k2Ziy0HJW5LirGtP+AXCPFXaPorOtEal sJyyRcz9Xkhelj3KuqrJeGrzBzxuxPQarbr4lWT5xQ8OcKBX6vnvHLOF3W2Pum7/kTqk Ivtm44OmPQcbHbeqcxPv83WVLKF+FtGgnZHsgtsN8F8m7xubwxdQP+gQJiPaFmXcvuNo b4kR8iswN32+WRMy0RDperlmd7abLTuQrEebI98aJwEthrQY9p3d+EErpQtg1jZ941ON 9Iz5iDnY8ATim0/R8H+h5AOeAalECqgT+ls7bwLPvD7OPljodKg4fo0DcFTMUP2SlnFm Xkrw== X-Gm-Message-State: AOAM5327LKtkJu5/scJU1DkHUjncjAfl0lTPWoQP6eo91gTBtBQIsYa+ Fb0I1u4ty8DNJsMxx6Ibt5FBABPbfcGWeVHGvSE= X-Google-Smtp-Source: ABdhPJx07lkmBwszLR7wmNvz/eGq/tB/F9L0mHCCMLR5H1bQoOsUpjB1Wi2/ovyBdqDGYfjYU993/gZHjA97y+1KFjc= X-Received: by 2002:a7b:c451:: with SMTP id l17mr15807813wmi.127.1602591320576; Tue, 13 Oct 2020 05:15:20 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:adf:c448:0:0:0:0:0 with HTTP; Tue, 13 Oct 2020 05:15:19 -0700 (PDT) In-Reply-To: References: <20201013.202148.2154404302288800973.shigeru@os-hackers.jp> From: Mateusz Guzik Date: Tue, 13 Oct 2020 14:15:19 +0200 Message-ID: Subject: Re: zfs panic when 'make buildworld buildkernel' To: YAMAMOTO Shigeru Cc: freebsd-current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4C9ZHt1cQVz4fT7 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.80 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-0.97)[-0.968]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_HAM_LONG(-0.98)[-0.975]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::341:from]; NEURAL_HAM_SHORT(-0.86)[-0.856]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2020 12:15:23 -0000 On 10/13/20, Mateusz Guzik wrote: > On 10/13/20, YAMAMOTO Shigeru wrote: >> >> Hi, >> >> I try to 'make buildworld buildkernel' at full ZFS environment. >> But I can't finished buildworld/buildkernel without panic. >> Anyone have same trouble? >> >> uname -a: >> ``` >> FreeBSD jenkins-02.current.os-hackers.jp 13.0-CURRENT FreeBSD >> 13.0-CURRENT >> #0 r366657: Tue Oct 13 13:07:15 JST 2020 >> root@jenkins-02.current.os-hackers.jp:/usr/obj/usr/src/amd64.amd64/sys/GENERIC >> amd64 >> ``` >> >> panic message: >> ``` >> login: panic: VERIFY(tid) failed >> >> cpuid = 2 >> time = 1602582381 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe002abaa9f0 >> vpanic() at vpanic+0x182/frame 0xfffffe002abaaa40 >> spl_panic() at spl_panic+0x3a/frame 0xfffffe002abaaaa0 >> taskq_dispatch() at taskq_dispatch+0xe8/frame 0xfffffe002abaaae0 >> arc_prune_async() at arc_prune_async+0x3f/frame 0xfffffe002abaab00 >> arc_evict_cb() at arc_evict_cb+0x1f6/frame 0xfffffe002abaab60 >> zthr_procedure() at zthr_procedure+0x8f/frame 0xfffffe002abaabb0 >> fork_exit() at fork_exit+0x80/frame 0xfffffe002abaabf0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe002abaabf0 >> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >> KDB: enter: panic >> [ thread pid 19 tid 100070 ] >> Stopped at kdb_enter+0x37: movq $0,0x10b0116(%rip) >> db> >> ``` >> > > The issue is pretty apparent: > > taskqid_t tqid = atomic_fetchadd_int(&tqidnext, 1); > > this eventually wraps to 0 and then you get the crash. > > Probably the thing to do is ot bump it to 64 bit and 0-check on other > platforms. > This should do it for the time being: diff --git a/sys/contrib/openzfs/module/os/freebsd/spl/spl_taskq.c b/sys/contrib/openzfs/module/os/freebsd/spl/spl_taskq.c index 1050816cd968..a8e53aba3915 100644 --- a/sys/contrib/openzfs/module/os/freebsd/spl/spl_taskq.c +++ b/sys/contrib/openzfs/module/os/freebsd/spl/spl_taskq.c @@ -67,7 +67,7 @@ static unsigned long tqenthash; static unsigned long tqenthashlock; static struct sx *tqenthashtbl_lock; -static uint32_t tqidnext = 1; +static uint32_t tqidnext; #define TQIDHASH(tqid) (&tqenthashtbl[(tqid) & tqenthash]) #define TQIDHASHLOCK(tqid) (&tqenthashtbl_lock[((tqid) & tqenthashlock)]) @@ -90,7 +90,6 @@ system_taskq_init(void *arg) M_TASKQ, M_WAITOK | M_ZERO); for (i = 0; i < tqenthashlock + 1; i++) sx_init_flags(&tqenthashtbl_lock[i], "tqenthash", SX_DUPOK); - tqidnext = 1; taskq_zone = uma_zcreate("taskq_zone", sizeof (taskq_ent_t), NULL, NULL, NULL, NULL, UMA_ALIGN_CACHE, 0); @@ -137,10 +136,23 @@ taskq_lookup(taskqid_t tqid) return (ent); } +static taskqid_t +__taskq_nextgen(void) +{ + taskqid_t tqid; + + for (;;) { + tqid = atomic_fetchadd_int(&tqidnext, 1) + 1; + if (__predict_true(tqid != 0)) + break; + } + return (tqid); +} + static taskqid_t taskq_insert(taskq_ent_t *ent) { - taskqid_t tqid = atomic_fetchadd_int(&tqidnext, 1); + taskqid_t tqid = __taskq_nextgen(); ent->tqent_id = tqid; ent->tqent_registered = B_TRUE; @@ -345,9 +357,9 @@ taskq_dispatch(taskq_t *tq, task_func_t func, void *arg, uint_t flags) task->tqent_cancelled = B_FALSE; task->tqent_type = NORMAL_TASK; tid = taskq_insert(task); + VERIFY(tid); TASK_INIT(&task->tqent_task, prio, taskq_run, task); taskqueue_enqueue(tq->tq_queue, &task->tqent_task); - VERIFY(tid); return (tid); } -- Mateusz Guzik From owner-freebsd-current@freebsd.org Thu Oct 15 04:17:55 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 43BB8430B39 for ; Thu, 15 Oct 2020 04:17:55 +0000 (UTC) (envelope-from shigeru@os-hackers.jp) Received: from mailssl04.asahi-net.or.jp (mailssl04.asahi-net.or.jp [202.224.55.63]) by mx1.freebsd.org (Postfix) with ESMTP id 4CBbc22KxCz3fPS for ; Thu, 15 Oct 2020 04:17:54 +0000 (UTC) (envelope-from shigeru@os-hackers.jp) Received: from localhost (w142149.ppp.asahi-net.or.jp [121.1.142.149]) (Authenticated sender: WJ8S-YMMT) by mailssl04.asahi-net.or.jp (Postfix) with ESMTPSA id BF9874005A; Thu, 15 Oct 2020 13:10:47 +0900 (JST) Date: Thu, 15 Oct 2020 13:10:44 +0900 (JST) Message-Id: <20201015.131044.865831816285441904.shigeru@os-hackers.jp> To: mjguzik@gmail.com Cc: freebsd-current@freebsd.org Subject: Re: zfs panic when 'make buildworld buildkernel' From: YAMAMOTO Shigeru In-Reply-To: References: <20201013.202148.2154404302288800973.shigeru@os-hackers.jp> X-Mailer: Mew version 6.8 on Emacs 27.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CBbc22KxCz3fPS X-Spamd-Bar: +++++ X-Spamd-Result: default: False [5.09 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,body]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[os-hackers.jp]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_HAM_MEDIUM(-0.02)[-0.024]; TO_MATCH_ENVRCPT_SOME(0.00)[]; VIOLATED_DIRECT_SPF(3.50)[]; NEURAL_HAM_SHORT(-0.15)[-0.147]; RCPT_COUNT_TWO(0.00)[2]; MID_CONTAINS_FROM(1.00)[]; NEURAL_SPAM_LONG(0.26)[0.261]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:4685, ipnet:202.224.32.0/19, country:JP]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2020 04:17:55 -0000 Thank you for your help. after https://svnweb.freebsd.org/base?view=revision&revision=366685 I update kernel and try to 'make buildworld buildkernel'. 1st time, I can finished 'make buildworld buildkernel'. 2nd time, panic ``` [root@jenkins-02 ~]# uname -a FreeBSD jenkins-02.current.os-hackers.jp 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r366693: Wed Oct 14 21:04:42 JST 2020 root@jenkins-02.current.os-hackers.jp:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 [root@jenkins-02 ~]# ``` ``` flags () lock type zfs: UNLOCKED panic: No vop_fplookup_vexec(0xfffff80057f061f0, 0xfffffe004b286688) cpuid = 1 time = 1602733349 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe004b286570 vpanic() at vpanic+0x182/frame 0xfffffe004b2865c0 panic() at panic+0x43/frame 0xfffffe004b286620 VOP_FPLOOKUP_VEXEC_APV() at VOP_FPLOOKUP_VEXEC_APV+0x125/frame 0xfffffe004b286640 cache_fplookup() at cache_fplookup+0x437/frame 0xfffffe004b286750 namei() at namei+0x149/frame 0xfffffe004b286810 vn_open_cred() at vn_open_cred+0x336/frame 0xfffffe004b286970 kern_openat() at kern_openat+0x25a/frame 0xfffffe004b286ad0 amd64_syscall() at amd64_syscall+0x135/frame 0xfffffe004b286bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe004b286bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x805909caa, rsp = 0x7fffffff71a8, rbp = 0x7fffffff7220 --- KDB: enter: panic [ thread pid 84431 tid 100664 ] Stopped at kdb_enter+0x37: movq $0,0x10b0116(%rip) db> ``` ``` db> trace Tracing pid 84431 tid 100664 td 0xfffffe004b1e8000 kdb_enter() at kdb_enter+0x37/frame 0xfffffe004b286570 vpanic() at vpanic+0x19e/frame 0xfffffe004b2865c0 panic() at panic+0x43/frame 0xfffffe004b286620 VOP_FPLOOKUP_VEXEC_APV() at VOP_FPLOOKUP_VEXEC_APV+0x125/frame 0xfffffe004b286640 cache_fplookup() at cache_fplookup+0x437/frame 0xfffffe004b286750 namei() at namei+0x149/frame 0xfffffe004b286810 vn_open_cred() at vn_open_cred+0x336/frame 0xfffffe004b286970 kern_openat() at kern_openat+0x25a/frame 0xfffffe004b286ad0 amd64_syscall() at amd64_syscall+0x135/frame 0xfffffe004b286bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe004b286bf0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x805909caa, rsp = 0x7fffffff71a8, rbp = 0x7fffffff7220 --- db> ``` Thanks, --- YAMAMOTO Shigeru From owner-freebsd-current@freebsd.org Thu Oct 15 04:50:24 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C3BD1432424 for ; Thu, 15 Oct 2020 04:50:24 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CBcKW5yYDz3gv2 for ; Thu, 15 Oct 2020 04:50:23 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-wm1-x335.google.com with SMTP id d81so1682245wmc.1 for ; Wed, 14 Oct 2020 21:50:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=808BnjG8ZdGyNZCdmfwdgwZv8s7TZgOaQmoKuulNihg=; b=ABITbcC6mX57oapTA4qmhEzqFWGuQoSTcrlJ+MyAXraI9u29vquRdPvuzE+RPx2+oa 9t5BPqPBvO+goQjd0Rt+OSoTpbjCemZCP4NKc2Ojnpjn2o+sQq5Qq4V65NQsz6lnl5Hc 5y2J51cLcSjV9BD1JgFnM9hRgl0Tcunkv+i9t1YwHyyNuYOa8kJBeC2EMNiut8Dq0kaE BiBAymdW8JTwvEHIOU/4WCWtsvs+Kfomim4I0efIdrUmMgTYCNfqZpUY5RFraps084BC a0ETVHtTVRYIRyiy68Duu9oGq1L2jYhsi82U8vt4T3OF/ymjgDqn3I4KO9ORxEJmrHe9 uSZA== X-Gm-Message-State: AOAM531TQB/sP5yykXtlWALcZrEXisPzHk5tqg4jwNxTb8YuMjNx+9X6 8yEuWiWzrdm4DnWdJbSRP7xESPaEi0vQO1cn4W+u8urJlac= X-Google-Smtp-Source: ABdhPJy2V0TOGX8hm5RArbqnz85/nMnVCzTqlw9mSvGeHb7fJXrNHmzWHC1HPxRr4AvrWjdq57+/i9LXGYTTlJwBBf4= X-Received: by 2002:a1c:2905:: with SMTP id p5mr1959413wmp.187.1602737422165; Wed, 14 Oct 2020 21:50:22 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:adf:c448:0:0:0:0:0 with HTTP; Wed, 14 Oct 2020 21:50:21 -0700 (PDT) In-Reply-To: <20201015.131044.865831816285441904.shigeru@os-hackers.jp> References: <20201013.202148.2154404302288800973.shigeru@os-hackers.jp> <20201015.131044.865831816285441904.shigeru@os-hackers.jp> From: Mateusz Guzik Date: Thu, 15 Oct 2020 06:50:21 +0200 Message-ID: Subject: Re: zfs panic when 'make buildworld buildkernel' To: YAMAMOTO Shigeru Cc: freebsd-current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4CBcKW5yYDz3gv2 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.22 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_HAM_LONG(-1.01)[-1.014]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::335:from]; NEURAL_HAM_SHORT(-0.21)[-0.211]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2020 04:50:24 -0000 Please try https://svnweb.freebsd.org/changeset/base/366717 On 10/15/20, YAMAMOTO Shigeru wrote: > > Thank you for your help. > > after > https://svnweb.freebsd.org/base?view=revision&revision=366685 > > I update kernel and try to 'make buildworld buildkernel'. > > 1st time, I can finished 'make buildworld buildkernel'. > > 2nd time, panic > > ``` > [root@jenkins-02 ~]# uname -a > FreeBSD jenkins-02.current.os-hackers.jp 13.0-CURRENT FreeBSD 13.0-CURRENT > #0 r366693: Wed Oct 14 21:04:42 JST 2020 > root@jenkins-02.current.os-hackers.jp:/usr/obj/usr/src/amd64.amd64/sys/GENERIC > amd64 > [root@jenkins-02 ~]# > ``` > > ``` > flags () > lock type zfs: UNLOCKED > panic: No vop_fplookup_vexec(0xfffff80057f061f0, 0xfffffe004b286688) > cpuid = 1 > time = 1602733349 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe004b286570 > vpanic() at vpanic+0x182/frame 0xfffffe004b2865c0 > panic() at panic+0x43/frame 0xfffffe004b286620 > VOP_FPLOOKUP_VEXEC_APV() at VOP_FPLOOKUP_VEXEC_APV+0x125/frame > 0xfffffe004b286640 > cache_fplookup() at cache_fplookup+0x437/frame 0xfffffe004b286750 > namei() at namei+0x149/frame 0xfffffe004b286810 > vn_open_cred() at vn_open_cred+0x336/frame 0xfffffe004b286970 > kern_openat() at kern_openat+0x25a/frame 0xfffffe004b286ad0 > amd64_syscall() at amd64_syscall+0x135/frame 0xfffffe004b286bf0 > fast_syscall_common() at fast_syscall_common+0xf8/frame > 0xfffffe004b286bf0 > --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x805909caa, rsp = > 0x7fffffff71a8, rbp = 0x7fffffff7220 --- > KDB: enter: panic > [ thread pid 84431 tid 100664 ] > Stopped at kdb_enter+0x37: movq $0,0x10b0116(%rip) > db> > ``` > > ``` > db> trace > Tracing pid 84431 tid 100664 td 0xfffffe004b1e8000 > kdb_enter() at kdb_enter+0x37/frame 0xfffffe004b286570 > vpanic() at vpanic+0x19e/frame 0xfffffe004b2865c0 > panic() at panic+0x43/frame 0xfffffe004b286620 > VOP_FPLOOKUP_VEXEC_APV() at VOP_FPLOOKUP_VEXEC_APV+0x125/frame > 0xfffffe004b286640 > cache_fplookup() at cache_fplookup+0x437/frame 0xfffffe004b286750 > namei() at namei+0x149/frame 0xfffffe004b286810 > vn_open_cred() at vn_open_cred+0x336/frame 0xfffffe004b286970 > kern_openat() at kern_openat+0x25a/frame 0xfffffe004b286ad0 > amd64_syscall() at amd64_syscall+0x135/frame 0xfffffe004b286bf0 > fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe004b286bf0 > --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x805909caa, rsp = > 0x7fffffff71a8, rbp = 0x7fffffff7220 --- > db> > ``` > > Thanks, > --- > YAMAMOTO Shigeru > -- Mateusz Guzik From owner-freebsd-current@freebsd.org Thu Oct 15 06:56:41 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 556BC43510E for ; Thu, 15 Oct 2020 06:56:41 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CBg7C4khSz451H for ; Thu, 15 Oct 2020 06:56:39 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.16.1/8.16.1) with ESMTPS id 09F6uWR0072276 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 14 Oct 2020 23:56:32 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.16.1/8.16.1/Submit) id 09F6uWbW072275 for freebsd-current@freebsd.org; Wed, 14 Oct 2020 23:56:32 -0700 (PDT) (envelope-from sgk) Date: Wed, 14 Oct 2020 23:56:32 -0700 From: Steve Kargl To: freebsd-current@freebsd.org Subject: kernel panic and fun debugging Message-ID: <20201015065632.GA72261@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 4CBg7C4khSz451H X-Spamd-Bar: + X-Spamd-Result: default: False [1.06 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.41)[0.405]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_SPAM_MEDIUM(0.49)[0.486]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_LONG(0.17)[0.166]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:73, ipnet:128.95.0.0/16, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current]; DMARC_POLICY_SOFTFAIL(0.10)[washington.edu : No valid SPF, No valid DKIM, none] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2020 06:56:41 -0000 Just had a kernel panic. Best info I give you is % uname -a FreeBSD mobile 13.0-CURRENT FreeBSD 13.0-CURRENT #1 r366176M: Sat Sep 26 10:35:23 PDT 2020 kargl@mobile:/usr/obj/usr/src/i386.i386/sys/MOBILE i386 % kgdb gdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 ... Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug... /usr/ports/devel/gdb/work-py37/gdb-9.2/gdb/inferior.c:283: internal-error: struct inferior *find_inferior_pid(int): Assertion `pid != 0' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. Whoops. -- Steve From owner-freebsd-current@freebsd.org Thu Oct 15 09:59:03 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1AC304398AC for ; Thu, 15 Oct 2020 09:59:03 +0000 (UTC) (envelope-from jakob@alvermark.net) Received: from out.alvermark.net (out.alvermark.net [185.34.136.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CBl9d59S7z4HSb for ; Thu, 15 Oct 2020 09:59:01 +0000 (UTC) (envelope-from jakob@alvermark.net) Received: from c-f649235c.06-431-73746f70.bbcust.telenor.se ([92.35.73.246] helo=mail.alvermark.net) by out.alvermark.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1kT02A-000NUJ-6X for freebsd-current@freebsd.org; Thu, 15 Oct 2020 11:58:54 +0200 Received: from office.as33885.net ([84.55.65.101] helo=[192.168.50.206]) by mail.alvermark.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1kT029-000GDg-O3 for freebsd-current@freebsd.org; Thu, 15 Oct 2020 11:58:53 +0200 To: freebsd-current From: Jakob Alvermark Subject: r366657: Add zstd support to the boot loader. Message-ID: Date: Thu, 15 Oct 2020 11:58:53 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 4CBl9d59S7z4HSb X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.26 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[alvermark.net:s=x]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.34.136.138]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[alvermark.net: no valid DMARC record]; NEURAL_HAM_LONG(-1.01)[-1.015]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-0.85)[-0.848]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[alvermark.net:+]; NEURAL_SPAM_SHORT(0.10)[0.100]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:34971, ipnet:185.34.136.0/24, country:IT]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2020 09:59:03 -0000 Hi, Does this mean I can 'zpool upgrade' now? Jakob From owner-freebsd-current@freebsd.org Fri Oct 16 09:53:58 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DA86E42E25A; Fri, 16 Oct 2020 09:53:58 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CCM1K6KvVz4hBs; Fri, 16 Oct 2020 09:53:57 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: by mail-wr1-x435.google.com with SMTP id h7so2045537wre.4; Fri, 16 Oct 2020 02:53:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:reply-to:subject:to:cc:references :message-id:date:user-agent:mime-version:in-reply-to :content-language; bh=CuqZdMzKnDhTNM9Ltncr6lMmS0lZ8LFVBEreqayUbyM=; b=SlUQjmDHWPxO30oZqHiBvxr30lJTwB1vJCM6e2fQrnORL9iKWccy6/wztjrJV8lAPW f+MIE6+rAcPJ1Chzr4MXw4G7TBxEk3jzRwoG2M4rX41YRH2J8EQZlbASvTRtqyf2E301 0fI9h5NsbQnk5EvC2owO4q0ylnVD1PHewLBdQOHFUwD5XLguzBhtkshuyBhUWSJt64Lr uF6duAGV4ihRiY08kNCHuPnFd1/CFXcIYZQksReRAajTtTLQhm+zgiW4JAqR0NGsSOG9 JnZAySE9sU80svUAlfhzCZgCDqqvP/FbNmRyOiGVYFrXqUYEgTX/U9QFjris+B7UZDAo hygQ== X-Gm-Message-State: AOAM5321b8Z6N88AQvvGSfz5mHGuENDJ7Hdx7XOZEIw0fdTF8OPccmsv /4Fl6xcySTWLHDBSuGbp2M308wOAzRg= X-Google-Smtp-Source: ABdhPJyeDiij3O+qGbYLwt3o0u60WL0nR3dSBIsxjvUqIWfXHLY9fd/yr1adStJZDdcFcVqpsLIqBQ== X-Received: by 2002:a5d:52ca:: with SMTP id r10mr2786310wrv.195.1602842036145; Fri, 16 Oct 2020 02:53:56 -0700 (PDT) Received: from [88.208.79.100] (halouny.humusoft.cz. [88.208.79.100]) by smtp.gmail.com with ESMTPSA id t124sm2173731wmg.31.2020.10.16.02.53.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 16 Oct 2020 02:53:55 -0700 (PDT) Sender: Michal Meloun From: Michal Meloun X-Google-Original-From: Michal Meloun Reply-To: mmel@freebsd.org Subject: Re: panic: non-current pmap 0xffffa00020eab8f0 on Rpi3 To: Mark Johnston , bob prohaska Cc: freebsd-current@freebsd.org, freebsd-arm@freebsd.org References: <20201006021029.GA13260@www.zefox.net> <20201006133743.GA96285@raichu> Message-ID: Date: Fri, 16 Oct 2020 11:53:56 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.3.1 MIME-Version: 1.0 In-Reply-To: <20201006133743.GA96285@raichu> Content-Type: multipart/mixed; boundary="------------47B922A2C808702819A29B19" Content-Language: en-US X-Rspamd-Queue-Id: 4CCM1K6KvVz4hBs X-Spamd-Bar: / X-Spamd-Result: default: False [0.33 / 15.00]; HAS_REPLYTO(0.00)[mmel@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; MIME_BASE64_TEXT_BOGUS(1.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; MIME_BASE64_TEXT(0.10)[]; CTYPE_MIXED_BOGUS(1.00)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.001]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.22)[0.221]; NEURAL_HAM_LONG(-0.99)[-0.991]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MID_RHS_MATCH_TO(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::435:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm,freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2020 09:53:58 -0000 This is a multi-part message in MIME format. --------------47B922A2C808702819A29B19 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 06.10.2020 15:37, Mark Johnston wrote: > On Mon, Oct 05, 2020 at 07:10:29PM -0700, bob prohaska wrote: >> Still seeing non-current pmap panics on the Pi3, this time a B+ running >> 13.0-CURRENT (GENERIC-MMCCAM) #0 71e02448ffb-c271826(master) >> during a -j4 buildworld. The backtrace reports >> >> panic: non-current pmap 0xffffa00020eab8f0 > > Could you show the output of "show procvm" from the debugger? I see same panic too, in my case its very rare - typical scenario is rebuild of kf5 ports (~250, 2 days of full load). Any idea how to debug this? Michal --------------47B922A2C808702819A29B19 Content-Type: text/plain; charset=UTF-8; name="pmap_panic.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="pmap_panic.txt" cGFuaWM6IG5vbi1jdXJyZW50IHBtYXAgMHhmZmZmYTAwMDEyZTkyYTgwCmNwdWlkID0gMgp0 aW1lID0gMTYwMjgzMTI2NQpLREI6IHN0YWNrIGJhY2t0cmFjZToKZGJfdHJhY2Vfc2VsZigp IGF0IGRiX2ZldGNoX2tzeW10YWIrMHgxNDgKICAgICAgICAgcGMgPSAweGZmZmYwMDAwMDA1 ZjY1NWMgIGxyID0gMHhmZmZmMDAwMDAwMTQ4ODA0CiAgICAgICAgIHNwID0gMHhmZmZmMDAw MDZkYzQxMjAwICBmcCA9IDB4ZmZmZjAwMDA2ZGM0MTQwMAoKZGJfZmV0Y2hfa3N5bXRhYigp IGF0IGtkYl9iYWNrdHJhY2UrMHgzOAogICAgICAgICBwYyA9IDB4ZmZmZjAwMDAwMDE0ODgw NCAgbHIgPSAweGZmZmYwMDAwMDAzNmQzYzAKICAgICAgICAgc3AgPSAweGZmZmYwMDAwNmRj NDE0MTAgIGZwID0gMHhmZmZmMDAwMDZkYzQxNGQwCgprZGJfYmFja3RyYWNlKCkgYXQgdnBh bmljKzB4MTljCiAgICAgICAgIHBjID0gMHhmZmZmMDAwMDAwMzZkM2MwICBsciA9IDB4ZmZm ZjAwMDAwMDMyYjMyMAogICAgICAgICBzcCA9IDB4ZmZmZjAwMDA2ZGM0MTRlMCAgZnAgPSAw eGZmZmYwMDAwNmRjNDE1MzAKCnZwYW5pYygpIGF0IHBhbmljKzB4NDQKICAgICAgICAgcGMg PSAweGZmZmYwMDAwMDAzMmIzMjAgIGxyID0gMHhmZmZmMDAwMDAwMzJhZjNjCiAgICAgICAg IHNwID0gMHhmZmZmMDAwMDZkYzQxNTQwICBmcCA9IDB4ZmZmZjAwMDA2ZGM0MTVmMAoKcGFu aWMoKSBhdCBwbWFwX3JlbW92ZV9wYWdlcysweDVjMAogICAgICAgICBwYyA9IDB4ZmZmZjAw MDAwMDMyYWYzYyAgbHIgPSAweGZmZmYwMDAwMDA2MGJmMGMKICAgICAgICAgc3AgPSAweGZm ZmYwMDAwNmRjNDE2MDAgIGZwID0gMHhmZmZmMDAwMDZkYzQxNjYwCgpwbWFwX3JlbW92ZV9w YWdlcygpIGF0IHZtc3BhY2VfZXhpdCsweGIwCiAgICAgICAgIHBjID0gMHhmZmZmMDAwMDAw NjBiZjBjICBsciA9IDB4ZmZmZjAwMDAwMDViOTg4YwogICAgICAgICBzcCA9IDB4ZmZmZjAw MDA2ZGM0MTY3MCAgZnAgPSAweGZmZmYwMDAwNmRjNDE2ZDAKCnZtc3BhY2VfZXhpdCgpIGF0 IGV4aXQxKzB4NGZjCiAgICAgICAgIHBjID0gMHhmZmZmMDAwMDAwNWI5ODhjICBsciA9IDB4 ZmZmZjAwMDAwMDJlM2ZjYwogICAgICAgICBzcCA9IDB4ZmZmZjAwMDA2ZGM0MTZlMCAgZnAg PSAweGZmZmYwMDAwNmRjNDE3NDAKCmV4aXQxKCkgYXQgc3lzX3N5c19leGl0KzB4MTAKICAg ICAgICAgcGMgPSAweGZmZmYwMDAwMDAyZTNmY2MgIGxyID0gMHhmZmZmMDAwMDAwMmUzYWNj CiAgICAgICAgIHNwID0gMHhmZmZmMDAwMDZkYzQxNzUwICBmcCA9IDB4ZmZmZjAwMDA2ZGM0 MTdhMAoKc3lzX3N5c19leGl0KCkgYXQgdW5oYW5kbGVkX2V4Y2VwdGlvbisweDU3YwogICAg ICAgICBwYyA9IDB4ZmZmZjAwMDAwMDJlM2FjYyAgbHIgPSAweGZmZmYwMDAwMDA2MTI1OGMK ICAgICAgICAgc3AgPSAweGZmZmYwMDAwNmRjNDE3YjAgIGZwID0gMHhmZmZmMDAwMDZkYzQx N2MwCgp1bmhhbmRsZWRfZXhjZXB0aW9uKCkgYXQgZG9fZWwwX3N5bmMrMHgzMjQKICAgICAg ICAgcGMgPSAweGZmZmYwMDAwMDA2MTI1OGMgIGxyID0gMHhmZmZmMDAwMDAwNjExZjY0CiAg ICAgICAgIHNwID0gMHhmZmZmMDAwMDZkYzQxN2QwICBmcCA9IDB4ZmZmZjAwMDA2ZGM0MTgw MAoKZG9fZWwwX3N5bmMoKSBhdCBkb19lbDBfc3luYysweDEyOAogICAgICAgICBwYyA9IDB4 ZmZmZjAwMDAwMDYxMWY2NCAgbHIgPSAweGZmZmYwMDAwMDA2MTFkNjgKICAgICAgICAgc3Ag PSAweGZmZmYwMDAwNmRjNDE4MTAgIGZwID0gMHhmZmZmMDAwMDZkYzQxODMwCgpkb19lbDBf c3luYygpIGF0IGhhbmRsZV9lbDBfc3luYysweDkwCiAgICAgICAgIHBjID0gMHhmZmZmMDAw MDAwNjExZDY4ICBsciA9IDB4ZmZmZjAwMDAwMDVmOGEyNAogICAgICAgICBzcCA9IDB4ZmZm ZjAwMDA2ZGM0MTg0MCAgZnAgPSAweGZmZmYwMDAwNmRjNDE5ODAKCmhhbmRsZV9lbDBfc3lu YygpIGF0IDB4NDA3YTVlZDAKICAgICAgICAgcGMgPSAweGZmZmYwMDAwMDA1ZjhhMjQgIGxy ID0gMHgwMDAwMDAwMDQwN2E1ZWQwCiAgICAgICAgIHNwID0gMHhmZmZmMDAwMDZkYzQxOTkw ICBmcCA9IDB4MDAwMGZmZmZmZmZmYmNiMAoKS0RCOiBlbnRlcjogcGFuaWMKWyB0aHJlYWQg cGlkIDIyNjAzIHRpZCAxMDAwOTkgXQpTdG9wcGVkIGF0ICAgICAgMHg0MDc3ZGYzMDogICAg IHVuZGVmaW5lZCAgICAgICA1NDAwMDA0MgpkYj4gc2hvdyBhbGwgcGNwdQpDdXJyZW50IENQ VTogMgoKY3B1aWQgICAgICAgID0gMApkeW5hbWljIHBjcHUgPSAweDY2Yzg4MApjdXJ0aHJl YWQgICAgPSAweGZmZmZhMDAwYjM5ZWIwMDA6IHBpZCAyMjYwNCB0aWQgMTAwMTM5IGNyaXRu ZXN0IDEgIm1zZ2ZtdCIKY3VycGNiICAgICAgID0gMHhmZmZmMDAwMDZkZDA5YWEwCmZwY3Vy dGhyZWFkICA9IDB4ZmZmZmEwMDBiMzllYjAwMDogcGlkIDIyNjA0ICJtc2dmbXQiCmlkbGV0 aHJlYWQgICA9IDB4ZmZmZmEwMDAwMTJhZDVhMDogdGlkIDEwMDAwMyAiaWRsZTogY3B1MCIK c3BpbiBsb2NrcyBoZWxkOgoKY3B1aWQgICAgICAgID0gMQpkeW5hbWljIHBjcHUgPSAweDQ1 YmZjODgwCmN1cnRocmVhZCAgICA9IDB4ZmZmZmEwMDBkYTQxODVhMDogcGlkIDIyNjA1IHRp ZCAxMDExNTUgY3JpdG5lc3QgMSAiY21ha2UiCmN1cnBjYiAgICAgICA9IDB4ZmZmZjAwMDA2 ZTAyY2FhMApmcGN1cnRocmVhZCAgPSAweGZmZmZhMDAwZGE0MTg1YTA6IHBpZCAyMjYwNSAi Y21ha2UiCmlkbGV0aHJlYWQgICA9IDB4ZmZmZmEwMDAwMTJhZDAwMDogdGlkIDEwMDAwNCAi aWRsZTogY3B1MSIKc3BpbiBsb2NrcyBoZWxkOgoKY3B1aWQgICAgICAgID0gMgpkeW5hbWlj IHBjcHUgPSAweDQ1YzAwODgwCmN1cnRocmVhZCAgICA9IDB4ZmZmZmEwMDAxMmM0YzAwMDog cGlkIDIyNjAzIHRpZCAxMDAwOTkgY3JpdG5lc3QgMSAibXNnZm10IgpjdXJwY2IgICAgICAg PSAweGZmZmYwMDAwNmRjNDFhYTAKZnBjdXJ0aHJlYWQgID0gMHhmZmZmYTAwMDEyYzRjNWEw OiBwaWQgMjIzMDcgImNtYWtlIgppZGxldGhyZWFkICAgPSAweGZmZmZhMDAwMDEyYWE1YTA6 IHRpZCAxMDAwMDUgImlkbGU6IGNwdTIiCnNwaW4gbG9ja3MgaGVsZDoKCmNwdWlkICAgICAg ICA9IDMKZHluYW1pYyBwY3B1ID0gMHg0NWMwNDg4MApjdXJ0aHJlYWQgICAgPSAweGZmZmZh MDAwMDc4OGQ1YTA6IHBpZCA3IHRpZCAxMDAwMzkgY3JpdG5lc3QgMSAiZG9uZXEwIgpjdXJw Y2IgICAgICAgPSAweGZmZmYwMDAwNDAzNzJhYTAKZnBjdXJ0aHJlYWQgID0gMHhmZmZmYTAw MDEyYzRjMDAwOiBwaWQgMjI2MDMgIm1zZ2ZtdCIKaWRsZXRocmVhZCAgID0gMHhmZmZmYTAw MDAxMmFhMDAwOiB0aWQgMTAwMDA2ICJpZGxlOiBjcHUzIgpzcGluIGxvY2tzIGhlbGQ6Cgpk Yj4gc2hvdyBwcm9jdm0KcCA9IDB4ZmZmZmEwMDAxMmU1ZDAwMCwgdm1zcGFjZSA9IDB4ZmZm ZmEwMDAxMmU5Mjk2MCwgbWFwID0gMHhmZmZmYTAwMDEyZTkyOTYwLCBwbWFwID0gMHhmZmZm YTAwMDEyZTkyYTgwClRhc2sgbWFwIDB4ZmZmZmEwMDAxMmU5Mjk2MDogcG1hcD0weGZmZmZh MDAwMTJlOTJhODAsIG5lbnRyaWVzPTg4LCB2ZXJzaW9uPTE3MAogIG1hcCBlbnRyeSAweGZm ZmZhMDAwMjM1M2I2YzA6IHN0YXJ0PTB4MjAwMDAwLCBlbmQ9MHgyMDgwMDAsIGVmbGFncz0w eGMwYywKICAgcHJvdD0xLzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDBiYTFjZWM2MCwgb2Zm c2V0PTB4MCwgY29weSAobmVlZGVkKQogICAgT2JqZWN0IDB4ZmZmZmEwMDBiYTFjZWM2MDog dHlwZT0yLCBzaXplPTB4MTMsIHJlcz0xOSwgcmVmPTQsIGZsYWdzPTB4MTAwMCBydWlkIC0x IGNoYXJnZSAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCgwKT0oMCkrMHgwCiAgbWFw IGVudHJ5IDB4ZmZmZmEwMDA0NTdjMzE4MDogc3RhcnQ9MHgyMTcwMDAsIGVuZD0weDIyMjAw MCwgZWZsYWdzPTB4YzBjLAogICBwcm90PTUvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMGJh MWNlYzYwLCBvZmZzZXQ9MHg3MDAwLCBjb3B5IChuZWVkZWQpCiAgbWFwIGVudHJ5IDB4ZmZm ZmEwMDBiM2NhYTYwMDogc3RhcnQ9MHgyMzEwMDAsIGVuZD0weDIzMjAwMCwgZWZsYWdzPTAs CiAgIHByb3Q9MS83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwNGIzNzZjNjAsIG9mZnNldD0w eDAsIG9iaiBydWlkIDAgY2hhcmdlIDEwMDAKICAgIE9iamVjdCAweGZmZmZhMDAwNGIzNzZj NjA6IHR5cGU9MCwgc2l6ZT0weDEsIHJlcz0xLCByZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1aWQg MCBjaGFyZ2UgMTAwMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3QoMCk9KDApKzB4MAog IG1hcCBlbnRyeSAweGZmZmZhMDAwYjNjZTU3MjA6IHN0YXJ0PTB4MjQxMDAwLCBlbmQ9MHgy NDMwMDAsIGVmbGFncz0wLAogICBwcm90PTMvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMGIz YTlhNTI4LCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJnZSAyMDAwCiAgICBPYmplY3Qg MHhmZmZmYTAwMGIzYTlhNTI4OiB0eXBlPTAsIHNpemU9MHgyLCByZXM9MiwgcmVmPTEsIGZs YWdzPTB4MzAxMCBydWlkIDAgY2hhcmdlIDIwMDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2Jq ZWN0KDApPSgwKSsweDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDEyZGJlZTQwOiBzdGFydD0w eDQwMjQxMDAwLCBlbmQ9MHg0MDI0NjAwMCwgZWZsYWdzPTB4YzBjLAogICBwcm90PTEvNy9j b3B5LCBvYmplY3Q9MHhmZmZmYTAwMGQzNmM5MDAwLCBvZmZzZXQ9MHgwLCBjb3B5IChuZWVk ZWQpCiAgICBPYmplY3QgMHhmZmZmYTAwMGQzNmM5MDAwOiB0eXBlPTIsIHNpemU9MHgxZCwg cmVzPTI3LCByZWY9MjYsIGZsYWdzPTB4MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3Jl Zj0wLCBiYWNraW5nX29iamVjdCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDAy MzMwZDJhMDogc3RhcnQ9MHg0MDI1NTAwMCwgZW5kPTB4NDAyNmMwMDAsIGVmbGFncz0weGMw YywKICAgcHJvdD01LzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDBkMzZjOTAwMCwgb2Zmc2V0 PTB4NDAwMCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwMzY4Nzg4YTA6 IHN0YXJ0PTB4NDAyN2IwMDAsIGVuZD0weDQwMjdjMDAwLCBlZmxhZ3M9MHg4MDQsCiAgIHBy b3Q9MS83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwYjdmZTJiNTgsIG9mZnNldD0weDAsIG9i aiBydWlkIDAgY2hhcmdlIDEwMDAsIGNvcHkgKGRvbmUpCiAgICBPYmplY3QgMHhmZmZmYTAw MGI3ZmUyYjU4OiB0eXBlPTAsIHNpemU9MHgxLCByZXM9MSwgcmVmPTEsIGZsYWdzPTB4MzAx MCBydWlkIDAgY2hhcmdlIDEwMDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0KDI2KT0o MHhmZmZmYTAwMGQzNmM5MDAwKSsweDFhMDAwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDA0OTE1 NjMwMDogc3RhcnQ9MHg0MDI4YjAwMCwgZW5kPTB4NDAyYTMwMDAsIGVmbGFncz0wLAogICBw cm90PTMvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMDIzMWU4NzM4LCBvZmZzZXQ9MHgwLCBv YmogcnVpZCAwIGNoYXJnZSAxODAwMAogICAgT2JqZWN0IDB4ZmZmZmEwMDAyMzFlODczODog dHlwZT0wLCBzaXplPTB4MTgsIHJlcz0yMywgcmVmPTEsIGZsYWdzPTB4MzAxMCBydWlkIDAg Y2hhcmdlIDE4MDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCgwKT0oMCkrMHgwCiAg bWFwIGVudHJ5IDB4ZmZmZmEwMDAxMmFkOTVhMDogc3RhcnQ9MHg0MDJhZTAwMCwgZW5kPTB4 NDAyYzMwMDAsIGVmbGFncz0weDQwYywKICAgcHJvdD0xLzcvY29weSwgb2JqZWN0PTB4ZmZm ZmEwMDA3OTc5NjYzMCwgb2Zmc2V0PTB4MCwgY29weSAobmVlZGVkKQogICAgT2JqZWN0IDB4 ZmZmZmEwMDA3OTc5NjYzMDogdHlwZT0yLCBzaXplPTB4NGQsIHJlcz01MywgcmVmPTgsIGZs YWdzPTB4MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVj dCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDA4NWJmNmQ4MDogc3RhcnQ9MHg0 MDJjMzAwMCwgZW5kPTB4NDAyZDIwMDAsIGVmbGFncz0weDEwMDBjLAogICBwcm90PTAvMC9j b3B5LCBvYmplY3Q9MCwgb2Zmc2V0PTB4MTUwMDAsIGNvcHkgKG5lZWRlZCkKICBtYXAgZW50 cnkgMHhmZmZmYTAwMDM4NWJkMzYwOiBzdGFydD0weDQwMmQyMDAwLCBlbmQ9MHg0MDMwOTAw MCwgZWZsYWdzPTB4NDBjLAogICBwcm90PTUvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMDc5 Nzk2NjMwLCBvZmZzZXQ9MHgxNDAwMCwgY29weSAobmVlZGVkKQogICAgT2JqZWN0IDB4ZmZm ZmEwMDA3OTc5NjYzMDogdHlwZT0yLCBzaXplPTB4NGQsIHJlcz01MywgcmVmPTgsIGZsYWdz PTB4MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCgw KT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDAxMmY1NGNjMDogc3RhcnQ9MHg0MDMw OTAwMCwgZW5kPTB4NDAzMTgwMDAsIGVmbGFncz0weDEwMDBjLAogICBwcm90PTAvMC9jb3B5 LCBvYmplY3Q9MCwgb2Zmc2V0PTB4NWIwMDAsIGNvcHkgKG5lZWRlZCkKICBtYXAgZW50cnkg MHhmZmZmYTAwMDcwYTM1ODQwOiBzdGFydD0weDQwMzE4MDAwLCBlbmQ9MHg0MDMxOTAwMCwg ZWZsYWdzPTB4NCwKICAgcHJvdD0xLzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDA3MGZjYmU3 MCwgb2Zmc2V0PTB4MCwgb2JqIHJ1aWQgMCBjaGFyZ2UgMTAwMCwgY29weSAoZG9uZSkKICAg IE9iamVjdCAweGZmZmZhMDAwNzBmY2JlNzA6IHR5cGU9MCwgc2l6ZT0weDEsIHJlcz0xLCBy ZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1aWQgMCBjaGFyZ2UgMTAwMAogICAgIHNyZWY9MCwgYmFj a2luZ19vYmplY3QoOCk9KDB4ZmZmZmEwMDA3OTc5NjYzMCkrMHg0YTAwMAogIG1hcCBlbnRy eSAweGZmZmZhMDAwMzY4NzZhZTA6IHN0YXJ0PTB4NDAzMTkwMDAsIGVuZD0weDQwMzI4MDAw LCBlZmxhZ3M9MHgxMDAwYywKICAgcHJvdD0wLzAvY29weSwgb2JqZWN0PTAsIG9mZnNldD0w eDZiMDAwLCBjb3B5IChuZWVkZWQpCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDBiN2I2Yzc4MDog c3RhcnQ9MHg0MDMyODAwMCwgZW5kPTB4NDAzMmEwMDAsIGVmbGFncz0weDQsCiAgIHByb3Q9 My83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwNDU2OGNlNzAsIG9mZnNldD0weDAsIG9iaiBy dWlkIDAgY2hhcmdlIDIwMDAsIGNvcHkgKGRvbmUpCiAgICBPYmplY3QgMHhmZmZmYTAwMDQ1 NjhjZTcwOiB0eXBlPTAsIHNpemU9MHgyLCByZXM9MiwgcmVmPTEsIGZsYWdzPTB4MzAxMCBy dWlkIDAgY2hhcmdlIDIwMDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0KDgpPSgweGZm ZmZhMDAwNzk3OTY2MzApKzB4NGEwMDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDg1YjE3OWMw OiBzdGFydD0weDQwMzJhMDAwLCBlbmQ9MHg0MDMyYjAwMCwgZWZsYWdzPTAsCiAgIHByb3Q9 My83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwYjNmMTM4NDAsIG9mZnNldD0weDAsIG9iaiBy dWlkIDAgY2hhcmdlIDEwMDAKICAgIE9iamVjdCAweGZmZmZhMDAwYjNmMTM4NDA6IHR5cGU9 MCwgc2l6ZT0weDEsIHJlcz0xLCByZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1aWQgMCBjaGFyZ2Ug MTAwMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3QoMCk9KDApKzB4MAogIG1hcCBlbnRy eSAweGZmZmZhMDAwMjMxYTlmNjA6IHN0YXJ0PTB4NDAzMmIwMDAsIGVuZD0weDQwNDAxMDAw LCBlZmxhZ3M9MHg0MGMsCiAgIHByb3Q9MS83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwZGEx YzlhNTAsIG9mZnNldD0weDAsIGNvcHkgKG5lZWRlZCkKICAgIE9iamVjdCAweGZmZmZhMDAw ZGExYzlhNTA6IHR5cGU9Miwgc2l6ZT0weDE3MywgcmVzPTExMiwgcmVmPTgsIGZsYWdzPTB4 MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCgwKT0o MCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDBkYzVkZDg0MDogc3RhcnQ9MHg0MDQwMTAw MCwgZW5kPTB4NDA0MTAwMDAsIGVmbGFncz0weDEwMDBjLAogICBwcm90PTAvMC9jb3B5LCBv YmplY3Q9MCwgb2Zmc2V0PTB4ZDYwMDAsIGNvcHkgKG5lZWRlZCkKICBtYXAgZW50cnkgMHhm ZmZmYTAwMDg1YjE3ODQwOiBzdGFydD0weDQwNDEwMDAwLCBlbmQ9MHg0MDRhYjAwMCwgZWZs YWdzPTB4NDBjLAogICBwcm90PTUvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMGRhMWM5YTUw LCBvZmZzZXQ9MHhkNTAwMCwgY29weSAobmVlZGVkKQogICAgT2JqZWN0IDB4ZmZmZmEwMDBk YTFjOWE1MDogdHlwZT0yLCBzaXplPTB4MTczLCByZXM9MTEyLCByZWY9OCwgZmxhZ3M9MHgx MDAwIHJ1aWQgLTEgY2hhcmdlIDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0KDApPSgw KSsweDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDIzMzBkNjAwOiBzdGFydD0weDQwNGFiMDAw LCBlbmQ9MHg0MDRiYTAwMCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9i amVjdD0wLCBvZmZzZXQ9MHgxODAwMDAsIGNvcHkgKG5lZWRlZCkKICBtYXAgZW50cnkgMHhm ZmZmYTAwMDEyYWQ5OTYwOiBzdGFydD0weDQwNGJhMDAwLCBlbmQ9MHg0MDRiYjAwMCwgZWZs YWdzPTB4NCwKICAgcHJvdD0xLzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDA0OTRiMWQ2OCwg b2Zmc2V0PTB4MCwgb2JqIHJ1aWQgMCBjaGFyZ2UgMTAwMCwgY29weSAoZG9uZSkKICAgIE9i amVjdCAweGZmZmZhMDAwNDk0YjFkNjg6IHR5cGU9MCwgc2l6ZT0weDEsIHJlcz0xLCByZWY9 MSwgZmxhZ3M9MHgzMDEwIHJ1aWQgMCBjaGFyZ2UgMTAwMAogICAgIHNyZWY9MCwgYmFja2lu Z19vYmplY3QoOCk9KDB4ZmZmZmEwMDBkYTFjOWE1MCkrMHgxNmYwMDAKICBtYXAgZW50cnkg MHhmZmZmYTAwMGI3OTZkYzAwOiBzdGFydD0weDQwNGJiMDAwLCBlbmQ9MHg0MDRjYTAwMCwg ZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9MHgx OTAwMDAsIGNvcHkgKG5lZWRlZCkKICBtYXAgZW50cnkgMHhmZmZmYTAwMGUxYmJkNDIwOiBz dGFydD0weDQwNGNhMDAwLCBlbmQ9MHg0MDRjZDAwMCwgZWZsYWdzPTB4NCwKICAgcHJvdD0z LzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDBkYTc2NmI1OCwgb2Zmc2V0PTB4MCwgb2JqIHJ1 aWQgMCBjaGFyZ2UgMzAwMCwgY29weSAoZG9uZSkKICAgIE9iamVjdCAweGZmZmZhMDAwZGE3 NjZiNTg6IHR5cGU9MCwgc2l6ZT0weDMsIHJlcz0zLCByZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1 aWQgMCBjaGFyZ2UgMzAwMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3QoOCk9KDB4ZmZm ZmEwMDBkYTFjOWE1MCkrMHgxNmYwMDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDEyZjA5MDAw OiBzdGFydD0weDQwNGNkMDAwLCBlbmQ9MHg0MDRkMDAwMCwgZWZsYWdzPTAsCiAgIHByb3Q9 My83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwNGIwNDVlNzAsIG9mZnNldD0weDAsIG9iaiBy dWlkIDAgY2hhcmdlIDMwMDAKICAgIE9iamVjdCAweGZmZmZhMDAwNGIwNDVlNzA6IHR5cGU9 MCwgc2l6ZT0weDMsIHJlcz0zLCByZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1aWQgMCBjaGFyZ2Ug MzAwMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3QoMCk9KDApKzB4MAogIG1hcCBlbnRy eSAweGZmZmZhMDAwMzY4NzczNjA6IHN0YXJ0PTB4NDA0ZDAwMDAsIGVuZD0weDQwNGY0MDAw LCBlZmxhZ3M9MHg0MGMsCiAgIHByb3Q9MS83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwNDkz ZmFhNTAsIG9mZnNldD0weDAsIGNvcHkgKG5lZWRlZCkKICAgIE9iamVjdCAweGZmZmZhMDAw NDkzZmFhNTA6IHR5cGU9Miwgc2l6ZT0weDllLCByZXM9NDAsIHJlZj04LCBmbGFncz0weDEw MDAgcnVpZCAtMSBjaGFyZ2UgMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3QoMCk9KDAp KzB4MAogIG1hcCBlbnRyeSAweGZmZmZhMDAwMTJlNDczYzA6IHN0YXJ0PTB4NDA0ZjQwMDAs IGVuZD0weDQwNTAzMDAwLCBlZmxhZ3M9MHgxMDAwYywKICAgcHJvdD0wLzAvY29weSwgb2Jq ZWN0PTAsIG9mZnNldD0weDI0MDAwLCBjb3B5IChuZWVkZWQpCiAgbWFwIGVudHJ5IDB4ZmZm ZmEwMDBlNDFmYzdlMDogc3RhcnQ9MHg0MDUwMzAwMCwgZW5kPTB4NDA1N2EwMDAsIGVmbGFn cz0weDQwYywKICAgcHJvdD01LzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDA0OTNmYWE1MCwg b2Zmc2V0PTB4MjMwMDAsIGNvcHkgKG5lZWRlZCkKICAgIE9iamVjdCAweGZmZmZhMDAwNDkz ZmFhNTA6IHR5cGU9Miwgc2l6ZT0weDllLCByZXM9NDAsIHJlZj04LCBmbGFncz0weDEwMDAg cnVpZCAtMSBjaGFyZ2UgMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3QoMCk9KDApKzB4 MAogIG1hcCBlbnRyeSAweGZmZmZhMDAwMzI0NWFhODA6IHN0YXJ0PTB4NDA1N2EwMDAsIGVu ZD0weDQwNTg5MDAwLCBlZmxhZ3M9MHgxMDAwYywKICAgcHJvdD0wLzAvY29weSwgb2JqZWN0 PTAsIG9mZnNldD0weGFhMDAwLCBjb3B5IChuZWVkZWQpCiAgbWFwIGVudHJ5IDB4ZmZmZmEw MDA0OTEyOTk2MDogc3RhcnQ9MHg0MDU4OTAwMCwgZW5kPTB4NDA1OGQwMDAsIGVmbGFncz0w eDQsCiAgIHByb3Q9MS83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwMzg1OWM0MjAsIG9mZnNl dD0weDAsIG9iaiBydWlkIDAgY2hhcmdlIDQwMDAsIGNvcHkgKGRvbmUpCiAgICBPYmplY3Qg MHhmZmZmYTAwMDM4NTljNDIwOiB0eXBlPTAsIHNpemU9MHg0LCByZXM9NCwgcmVmPTEsIGZs YWdzPTB4MzAxMCBydWlkIDAgY2hhcmdlIDQwMDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2Jq ZWN0KDgpPSgweGZmZmZhMDAwNDkzZmFhNTApKzB4OTkwMDAKICBtYXAgZW50cnkgMHhmZmZm YTAwMDMyNDVjZjYwOiBzdGFydD0weDQwNThkMDAwLCBlbmQ9MHg0MDU5YzAwMCwgZWZsYWdz PTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9MHhiZDAwMCwg Y29weSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwZGM1ZGQ2MDA6IHN0YXJ0PTB4 NDA1OWMwMDAsIGVuZD0weDQwNTllMDAwLCBlZmxhZ3M9MHg0LAogICBwcm90PTMvNy9jb3B5 LCBvYmplY3Q9MHhmZmZmYTAwMDQ1MjM4YTUwLCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNo YXJnZSAyMDAwLCBjb3B5IChkb25lKQogICAgT2JqZWN0IDB4ZmZmZmEwMDA0NTIzOGE1MDog dHlwZT0wLCBzaXplPTB4MiwgcmVzPTIsIHJlZj0xLCBmbGFncz0weDMwMTAgcnVpZCAwIGNo YXJnZSAyMDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCg4KT0oMHhmZmZmYTAwMDQ5 M2ZhYTUwKSsweDljMDAwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDA3MGFkMDM2MDogc3RhcnQ9 MHg0MDU5ZTAwMCwgZW5kPTB4NDA1OWYwMDAsIGVmbGFncz0wLAogICBwcm90PTMvNy9jb3B5 LCBydWlkIDAsIG9iamVjdD0wLCBvZmZzZXQ9MHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDBi M2NlNWYwMDogc3RhcnQ9MHg0MDU5ZjAwMCwgZW5kPTB4NDA1YTMwMDAsIGVmbGFncz0weDQw YywKICAgcHJvdD0xLzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDBiM2FlMDEwOCwgb2Zmc2V0 PTB4MCwgY29weSAobmVlZGVkKQogICAgT2JqZWN0IDB4ZmZmZmEwMDBiM2FlMDEwODogdHlw ZT0yLCBzaXplPTB4ZSwgcmVzPTE0LCByZWY9OCwgZmxhZ3M9MHgxMDAwIHJ1aWQgLTEgY2hh cmdlIDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0KDApPSgwKSsweDAKICBtYXAgZW50 cnkgMHhmZmZmYTAwMDM4NWE3MjQwOiBzdGFydD0weDQwNWEzMDAwLCBlbmQ9MHg0MDViMjAw MCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9 MHg0MDAwLCBjb3B5IChuZWVkZWQpCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDAzODY0ZWQ4MDog c3RhcnQ9MHg0MDViMjAwMCwgZW5kPTB4NDA1YjgwMDAsIGVmbGFncz0weDQwYywKICAgcHJv dD01LzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDBiM2FlMDEwOCwgb2Zmc2V0PTB4MzAwMCwg Y29weSAobmVlZGVkKQogICAgT2JqZWN0IDB4ZmZmZmEwMDBiM2FlMDEwODogdHlwZT0yLCBz aXplPTB4ZSwgcmVzPTE0LCByZWY9OCwgZmxhZ3M9MHgxMDAwIHJ1aWQgLTEgY2hhcmdlIDAK ICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0KDApPSgwKSsweDAKICBtYXAgZW50cnkgMHhm ZmZmYTAwMDQ5MTU3MjQwOiBzdGFydD0weDQwNWI4MDAwLCBlbmQ9MHg0MDVjNzAwMCwgZWZs YWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9MHgxOTAw MCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwYjNjZTViNDA6IHN0YXJ0 PTB4NDA1YzcwMDAsIGVuZD0weDQwNWM4MDAwLCBlZmxhZ3M9MHg0LAogICBwcm90PTEvNy9j b3B5LCBvYmplY3Q9MHhmZmZmYTAwMDQ5N2QyYzYwLCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAw IGNoYXJnZSAxMDAwLCBjb3B5IChkb25lKQogICAgT2JqZWN0IDB4ZmZmZmEwMDA0OTdkMmM2 MDogdHlwZT0wLCBzaXplPTB4MSwgcmVzPTEsIHJlZj0xLCBmbGFncz0weDMwMTAgcnVpZCAw IGNoYXJnZSAxMDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCg4KT0oMHhmZmZmYTAw MGIzYWUwMTA4KSsweDgwMDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDM4NjRlNmMwOiBzdGFy dD0weDQwNWM4MDAwLCBlbmQ9MHg0MDVkNzAwMCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9 MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9MHgyOTAwMCwgY29weSAobmVlZGVkKQogIG1h cCBlbnRyeSAweGZmZmZhMDAwMzY4Yzg1NDA6IHN0YXJ0PTB4NDA1ZDcwMDAsIGVuZD0weDQw NWQ4MDAwLCBlZmxhZ3M9MHg0LAogICBwcm90PTMvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAw MDQ1MjBjNjMwLCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJnZSAxMDAwLCBjb3B5IChk b25lKQogICAgT2JqZWN0IDB4ZmZmZmEwMDA0NTIwYzYzMDogdHlwZT0wLCBzaXplPTB4MSwg cmVzPTEsIHJlZj0xLCBmbGFncz0weDMwMTAgcnVpZCAwIGNoYXJnZSAxMDAwCiAgICAgc3Jl Zj0wLCBiYWNraW5nX29iamVjdCg4KT0oMHhmZmZmYTAwMGIzYWUwMTA4KSsweDgwMDAKICBt YXAgZW50cnkgMHhmZmZmYTAwMDg1YzFhNjYwOiBzdGFydD0weDQwNWQ4MDAwLCBlbmQ9MHg0 MDVkYTAwMCwgZWZsYWdzPTAsCiAgIHByb3Q9My83L2NvcHksIG9iamVjdD0weGZmZmZhMDAw ZTQzYWE5NDgsIG9mZnNldD0weDAsIG9iaiBydWlkIDAgY2hhcmdlIDIwMDAKICAgIE9iamVj dCAweGZmZmZhMDAwZTQzYWE5NDg6IHR5cGU9MCwgc2l6ZT0weDIsIHJlcz0xLCByZWY9MSwg ZmxhZ3M9MHgzMDEwIHJ1aWQgMCBjaGFyZ2UgMjAwMAogICAgIHNyZWY9MCwgYmFja2luZ19v YmplY3QoMCk9KDApKzB4MAogIG1hcCBlbnRyeSAweGZmZmZhMDAwMzg1YTkyNDA6IHN0YXJ0 PTB4NDA1ZGEwMDAsIGVuZD0weDQwNWVkMDAwLCBlZmxhZ3M9MHg0MGMsCiAgIHByb3Q9MS83 L2NvcHksIG9iamVjdD0weGZmZmZhMDAwYjdlNWQ5NDgsIG9mZnNldD0weDAsIGNvcHkgKG5l ZWRlZCkKICAgIE9iamVjdCAweGZmZmZhMDAwYjdlNWQ5NDg6IHR5cGU9Miwgc2l6ZT0weDQy LCByZXM9NDQsIHJlZj0xOCwgZmxhZ3M9MHgxMDAwIHJ1aWQgLTEgY2hhcmdlIDAKICAgICBz cmVmPTAsIGJhY2tpbmdfb2JqZWN0KDApPSgwKSsweDAKICBtYXAgZW50cnkgMHhmZmZmYTAw MDQ5MTI5MWUwOiBzdGFydD0weDQwNWVkMDAwLCBlbmQ9MHg0MDVmYzAwMCwgZWZsYWdzPTB4 MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9MHgxMzAwMCwgY29w eSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwMTJkYmVmMDA6IHN0YXJ0PTB4NDA1 ZmMwMDAsIGVuZD0weDQwNjI3MDAwLCBlZmxhZ3M9MHg0MGMsCiAgIHByb3Q9NS83L2NvcHks IG9iamVjdD0weGZmZmZhMDAwYjdlNWQ5NDgsIG9mZnNldD0weDEyMDAwLCBjb3B5IChuZWVk ZWQpCiAgICBPYmplY3QgMHhmZmZmYTAwMGI3ZTVkOTQ4OiB0eXBlPTIsIHNpemU9MHg0Miwg cmVzPTQ0LCByZWY9MTgsIGZsYWdzPTB4MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3Jl Zj0wLCBiYWNraW5nX29iamVjdCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDBh NjgxNzZjMDogc3RhcnQ9MHg0MDYyNzAwMCwgZW5kPTB4NDA2MzYwMDAsIGVmbGFncz0weDEw MDBjLAogICBwcm90PTAvMC9jb3B5LCBvYmplY3Q9MCwgb2Zmc2V0PTB4NGQwMDAsIGNvcHkg KG5lZWRlZCkKICBtYXAgZW50cnkgMHhmZmZmYTAwMDM2OGNiNWEwOiBzdGFydD0weDQwNjM2 MDAwLCBlbmQ9MHg0MDYzNzAwMCwgZWZsYWdzPTB4NCwKICAgcHJvdD0xLzcvY29weSwgb2Jq ZWN0PTB4ZmZmZmEwMDAyMzQ1Y2U3MCwgb2Zmc2V0PTB4MCwgb2JqIHJ1aWQgMCBjaGFyZ2Ug MTAwMCwgY29weSAoZG9uZSkKICAgIE9iamVjdCAweGZmZmZhMDAwMjM0NWNlNzA6IHR5cGU9 MCwgc2l6ZT0weDEsIHJlcz0xLCByZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1aWQgMCBjaGFyZ2Ug MTAwMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3QoMTgpPSgweGZmZmZhMDAwYjdlNWQ5 NDgpKzB4M2MwMDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDEyZjA5YTgwOiBzdGFydD0weDQw NjM3MDAwLCBlbmQ9MHg0MDY0NjAwMCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2Nv cHksIG9iamVjdD0wLCBvZmZzZXQ9MHg1ZDAwMCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRy eSAweGZmZmZhMDAwODVmOGI3ODA6IHN0YXJ0PTB4NDA2NDYwMDAsIGVuZD0weDQwNjQ4MDAw LCBlZmxhZ3M9MHg0LAogICBwcm90PTMvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMDQ5MjY2 MjEwLCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJnZSAyMDAwLCBjb3B5IChkb25lKQog ICAgT2JqZWN0IDB4ZmZmZmEwMDA0OTI2NjIxMDogdHlwZT0wLCBzaXplPTB4MiwgcmVzPTIs IHJlZj0xLCBmbGFncz0weDMwMTAgcnVpZCAwIGNoYXJnZSAyMDAwCiAgICAgc3JlZj0wLCBi YWNraW5nX29iamVjdCgxOCk9KDB4ZmZmZmEwMDBiN2U1ZDk0OCkrMHgzYzAwMAogIG1hcCBl bnRyeSAweGZmZmZhMDAwZGM1ZGQ5YzA6IHN0YXJ0PTB4NDA2NDgwMDAsIGVuZD0weDQwNmMx MDAwLCBlZmxhZ3M9MHg0MGMsCiAgIHByb3Q9MS83L2NvcHksIG9iamVjdD0weGZmZmZhMDAw NDQ1MDA2MzAsIG9mZnNldD0weDAsIGNvcHkgKG5lZWRlZCkKICAgIE9iamVjdCAweGZmZmZh MDAwNDQ1MDA2MzA6IHR5cGU9Miwgc2l6ZT0weDFlYSwgcmVzPTM1MiwgcmVmPTM0LCBmbGFn cz0weDEwMDAgcnVpZCAtMSBjaGFyZ2UgMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3Qo MCk9KDApKzB4MAogIG1hcCBlbnRyeSAweGZmZmZhMDAwNzBhZDBiYTA6IHN0YXJ0PTB4NDA2 YzEwMDAsIGVuZD0weDQwNmQwMDAwLCBlZmxhZ3M9MHgxMDAwYywKICAgcHJvdD0wLzAvY29w eSwgb2JqZWN0PTAsIG9mZnNldD0weDc5MDAwLCBjb3B5IChuZWVkZWQpCiAgbWFwIGVudHJ5 IDB4ZmZmZmEwMDA4NWU5NjFlMDogc3RhcnQ9MHg0MDZkMDAwMCwgZW5kPTB4NDA4MjUwMDAs IGVmbGFncz0weDQwYywKICAgcHJvdD01LzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDA0NDUw MDYzMCwgb2Zmc2V0PTB4NzgwMDAsIGNvcHkgKG5lZWRlZCkKICAgIE9iamVjdCAweGZmZmZh MDAwNDQ1MDA2MzA6IHR5cGU9Miwgc2l6ZT0weDFlYSwgcmVzPTM1MiwgcmVmPTM0LCBmbGFn cz0weDEwMDAgcnVpZCAtMSBjaGFyZ2UgMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3Qo MCk9KDApKzB4MAogIG1hcCBlbnRyeSAweGZmZmZhMDAwMTJkYmUzYzA6IHN0YXJ0PTB4NDA4 MjUwMDAsIGVuZD0weDQwODM0MDAwLCBlZmxhZ3M9MHgxMDAwYywKICAgcHJvdD0wLzAvY29w eSwgb2JqZWN0PTAsIG9mZnNldD0weDFkZDAwMCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRy eSAweGZmZmZhMDAwODVmOGE2MDA6IHN0YXJ0PTB4NDA4MzQwMDAsIGVuZD0weDQwODNlMDAw LCBlZmxhZ3M9MHg0LAogICBwcm90PTEvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMDRiNGIx MTA4LCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJnZSBhMDAwLCBjb3B5IChkb25lKQog ICAgT2JqZWN0IDB4ZmZmZmEwMDA0YjRiMTEwODogdHlwZT0wLCBzaXplPTB4YSwgcmVzPTks IHJlZj0xLCBmbGFncz0weDMwMTAgcnVpZCAwIGNoYXJnZSBhMDAwCiAgICAgc3JlZj0wLCBi YWNraW5nX29iamVjdCgzNCk9KDB4ZmZmZmEwMDA0NDUwMDYzMCkrMHgxY2MwMDAKICBtYXAg ZW50cnkgMHhmZmZmYTAwMDg1ZjAwMTgwOiBzdGFydD0weDQwODNlMDAwLCBlbmQ9MHg0MDg0 ZDAwMCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBvZmZz ZXQ9MHgxZjYwMDAsIGNvcHkgKG5lZWRlZCkKICBtYXAgZW50cnkgMHhmZmZmYTAwMDMyNDVh OTAwOiBzdGFydD0weDQwODRkMDAwLCBlbmQ9MHg0MDg1MzAwMCwgZWZsYWdzPTB4NCwKICAg cHJvdD0zLzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDA3MGRmNTk0OCwgb2Zmc2V0PTB4MCwg b2JqIHJ1aWQgMCBjaGFyZ2UgNjAwMCwgY29weSAoZG9uZSkKICAgIE9iamVjdCAweGZmZmZh MDAwNzBkZjU5NDg6IHR5cGU9MCwgc2l6ZT0weDYsIHJlcz02LCByZWY9MSwgZmxhZ3M9MHgz MDEwIHJ1aWQgMCBjaGFyZ2UgNjAwMAogICAgIHNyZWY9MCwgYmFja2luZ19vYmplY3QoMzQp PSgweGZmZmZhMDAwNDQ1MDA2MzApKzB4MWQ1MDAwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDAx MmVkOWQyMDogc3RhcnQ9MHg0MDg1MzAwMCwgZW5kPTB4NDBhODEwMDAsIGVmbGFncz0wLAog ICBwcm90PTMvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMGRhNjZlYjU4LCBvZmZzZXQ9MHgw LCBvYmogcnVpZCAwIGNoYXJnZSAyMmUwMDAKICAgIE9iamVjdCAweGZmZmZhMDAwZGE2NmVi NTg6IHR5cGU9MCwgc2l6ZT0weDIyZSwgcmVzPTIyLCByZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1 aWQgMCBjaGFyZ2UgMjJlMDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCgwKT0oMCkr MHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDAzODY0ZTMwMDogc3RhcnQ9MHg0MGE4MTAwMCwg ZW5kPTB4NDBhYWEwMDAsIGVmbGFncz0weDQwYywKICAgcHJvdD0xLzcvY29weSwgb2JqZWN0 PTB4ZmZmZmEwMDA0YjUyNTczOCwgb2Zmc2V0PTB4MCwgY29weSAobmVlZGVkKQogICAgT2Jq ZWN0IDB4ZmZmZmEwMDA0YjUyNTczODogdHlwZT0yLCBzaXplPTB4NWQsIHJlcz00NCwgcmVm PTgsIGZsYWdzPTB4MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3JlZj0wLCBiYWNraW5n X29iamVjdCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDBiNzk2ZGY2MDogc3Rh cnQ9MHg0MGFhYTAwMCwgZW5kPTB4NDBhYjkwMDAsIGVmbGFncz0weDEwMDBjLAogICBwcm90 PTAvMC9jb3B5LCBvYmplY3Q9MCwgb2Zmc2V0PTB4MjkwMDAsIGNvcHkgKG5lZWRlZCkKICBt YXAgZW50cnkgMHhmZmZmYTAwMDEyYWQ2N2UwOiBzdGFydD0weDQwYWI5MDAwLCBlbmQ9MHg0 MGFlOTAwMCwgZWZsYWdzPTB4NDBjLAogICBwcm90PTUvNy9jb3B5LCBvYmplY3Q9MHhmZmZm YTAwMDRiNTI1NzM4LCBvZmZzZXQ9MHgyODAwMCwgY29weSAobmVlZGVkKQogICAgT2JqZWN0 IDB4ZmZmZmEwMDA0YjUyNTczODogdHlwZT0yLCBzaXplPTB4NWQsIHJlcz00NCwgcmVmPTgs IGZsYWdzPTB4MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29i amVjdCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDAxMmY4MWU0MDogc3RhcnQ9 MHg0MGFlOTAwMCwgZW5kPTB4NDBhZjgwMDAsIGVmbGFncz0weDEwMDBjLAogICBwcm90PTAv MC9jb3B5LCBvYmplY3Q9MCwgb2Zmc2V0PTB4NjgwMDAsIGNvcHkgKG5lZWRlZCkKICBtYXAg ZW50cnkgMHhmZmZmYTAwMGNiYTZmZWEwOiBzdGFydD0weDQwYWY4MDAwLCBlbmQ9MHg0MGFm YzAwMCwgZWZsYWdzPTB4NCwKICAgcHJvdD0xLzcvY29weSwgb2JqZWN0PTB4ZmZmZmEwMDBk YmFjMTk0OCwgb2Zmc2V0PTB4MCwgb2JqIHJ1aWQgMCBjaGFyZ2UgNDAwMCwgY29weSAoZG9u ZSkKICAgIE9iamVjdCAweGZmZmZhMDAwZGJhYzE5NDg6IHR5cGU9MCwgc2l6ZT0weDQsIHJl cz00LCByZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1aWQgMCBjaGFyZ2UgNDAwMAogICAgIHNyZWY9 MCwgYmFja2luZ19vYmplY3QoOCk9KDB4ZmZmZmEwMDA0YjUyNTczOCkrMHg1NzAwMAogIG1h cCBlbnRyeSAweGZmZmZhMDAwNDU3NGFmMDA6IHN0YXJ0PTB4NDBhZmMwMDAsIGVuZD0weDQw YjBiMDAwLCBlZmxhZ3M9MHgxMDAwYywKICAgcHJvdD0wLzAvY29weSwgb2JqZWN0PTAsIG9m ZnNldD0weDdiMDAwLCBjb3B5IChuZWVkZWQpCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDAxMmFk OTI0MDogc3RhcnQ9MHg0MGIwYjAwMCwgZW5kPTB4NDBiMGQwMDAsIGVmbGFncz0weDQsCiAg IHByb3Q9My83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwNDQ0YzJhNTAsIG9mZnNldD0weDAs IG9iaiBydWlkIDAgY2hhcmdlIDIwMDAsIGNvcHkgKGRvbmUpCiAgICBPYmplY3QgMHhmZmZm YTAwMDQ0NGMyYTUwOiB0eXBlPTAsIHNpemU9MHgyLCByZXM9MiwgcmVmPTEsIGZsYWdzPTB4 MzAxMCBydWlkIDAgY2hhcmdlIDIwMDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0KDgp PSgweGZmZmZhMDAwNGI1MjU3MzgpKzB4NWEwMDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDM4 NjRlOWMwOiBzdGFydD0weDQwYjBkMDAwLCBlbmQ9MHg0MGIzZTAwMCwgZWZsYWdzPTAsCiAg IHByb3Q9My83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwODVhMzRlNzAsIG9mZnNldD0weDAs IG9iaiBydWlkIDAgY2hhcmdlIDMxMDAwCiAgICBPYmplY3QgMHhmZmZmYTAwMDg1YTM0ZTcw OiB0eXBlPTAsIHNpemU9MHgzMSwgcmVzPTIwLCByZWY9MSwgZmxhZ3M9MHgzMDEwIHJ1aWQg MCBjaGFyZ2UgMzEwMDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0KDApPSgwKSsweDAK ICBtYXAgZW50cnkgMHhmZmZmYTAwMGI3OTZkN2UwOiBzdGFydD0weDQwYjNlMDAwLCBlbmQ9 MHg0MGIzZjAwMCwgZWZsYWdzPTB4NDBjLAogICBwcm90PTEvNy9jb3B5LCBvYmplY3Q9MHhm ZmZmYTAwMDQ0NDc0MTA4LCBvZmZzZXQ9MHgwLCBjb3B5IChuZWVkZWQpCiAgICBPYmplY3Qg MHhmZmZmYTAwMDQ0NDc0MTA4OiB0eXBlPTIsIHNpemU9MHgzLCByZXM9MywgcmVmPTQsIGZs YWdzPTB4MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVj dCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDAxMmU1MDNjMDogc3RhcnQ9MHg0 MGIzZjAwMCwgZW5kPTB4NDBiNGUwMDAsIGVmbGFncz0weDEwMDBjLAogICBwcm90PTAvMC9j b3B5LCBvYmplY3Q9MCwgb2Zmc2V0PTB4MTAwMCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRy eSAweGZmZmZhMDAwMjMzMGMxZTA6IHN0YXJ0PTB4NDBiNGUwMDAsIGVuZD0weDQwYjUwMDAw LCBlZmxhZ3M9MHg0MGMsCiAgIHByb3Q9NS83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwNDQ0 NzQxMDgsIG9mZnNldD0weDAsIGNvcHkgKG5lZWRlZCkKICAgIE9iamVjdCAweGZmZmZhMDAw NDQ0NzQxMDg6IHR5cGU9Miwgc2l6ZT0weDMsIHJlcz0zLCByZWY9NCwgZmxhZ3M9MHgxMDAw IHJ1aWQgLTEgY2hhcmdlIDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0KDApPSgwKSsw eDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDg1YzFhZDgwOiBzdGFydD0weDQwYjUwMDAwLCBl bmQ9MHg0MGI1ZjAwMCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVj dD0wLCBvZmZzZXQ9MHgxMjAwMCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZh MDAwODVmNDIxZTA6IHN0YXJ0PTB4NDBiNWYwMDAsIGVuZD0weDQwYjYwMDAwLCBlZmxhZ3M9 MHg0LAogICBwcm90PTEvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMDg1Y2I0OTQ4LCBvZmZz ZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJnZSAxMDAwLCBjb3B5IChkb25lKQogICAgT2JqZWN0 IDB4ZmZmZmEwMDA4NWNiNDk0ODogdHlwZT0wLCBzaXplPTB4MSwgcmVzPTEsIHJlZj0xLCBm bGFncz0weDMwMTAgcnVpZCAwIGNoYXJnZSAxMDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29i amVjdCg0KT0oMHhmZmZmYTAwMDQ0NDc0MTA4KSsweDEwMDAKICBtYXAgZW50cnkgMHhmZmZm YTAwMDg1ZjQyY2MwOiBzdGFydD0weDQwYjYwMDAwLCBlbmQ9MHg0MGI2ZjAwMCwgZWZsYWdz PTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9MHgyMjAwMCwg Y29weSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwMTJmMGQ0MjA6IHN0YXJ0PTB4 NDBiNmYwMDAsIGVuZD0weDQwYjcwMDAwLCBlZmxhZ3M9MHg0LAogICBwcm90PTMvNy9jb3B5 LCBvYmplY3Q9MHhmZmZmYTAwMDc5N2Y0ZTcwLCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNo YXJnZSAxMDAwLCBjb3B5IChkb25lKQogICAgT2JqZWN0IDB4ZmZmZmEwMDA3OTdmNGU3MDog dHlwZT0wLCBzaXplPTB4MSwgcmVzPTEsIHJlZj0xLCBmbGFncz0weDMwMTAgcnVpZCAwIGNo YXJnZSAxMDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCg0KT0oMHhmZmZmYTAwMDQ0 NDc0MTA4KSsweDEwMDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDcwYWQwNDIwOiBzdGFydD0w eDQwYjcwMDAwLCBlbmQ9MHg0MGI3MTAwMCwgZWZsYWdzPTB4NDBjLAogICBwcm90PTEvNy9j b3B5LCBvYmplY3Q9MHhmZmZmYTAwMDQ5MDQ3NTI4LCBvZmZzZXQ9MHgwLCBjb3B5IChuZWVk ZWQpCiAgICBPYmplY3QgMHhmZmZmYTAwMDQ5MDQ3NTI4OiB0eXBlPTIsIHNpemU9MHgyLCBy ZXM9MiwgcmVmPTQsIGZsYWdzPTB4MTAwMCBydWlkIC0xIGNoYXJnZSAwCiAgICAgc3JlZj0w LCBiYWNraW5nX29iamVjdCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDA4NWYw MDg0MDogc3RhcnQ9MHg0MGI3MTAwMCwgZW5kPTB4NDBiODAwMDAsIGVmbGFncz0weDEwMDBj LAogICBwcm90PTAvMC9jb3B5LCBvYmplY3Q9MCwgb2Zmc2V0PTB4MTAwMCwgY29weSAobmVl ZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwMTJlNDc0MjA6IHN0YXJ0PTB4NDBiODAwMDAs IGVuZD0weDQwYjgyMDAwLCBlZmxhZ3M9MHg0MGMsCiAgIHByb3Q9NS83L2NvcHksIG9iamVj dD0weGZmZmZhMDAwNDkwNDc1MjgsIG9mZnNldD0weDAsIGNvcHkgKG5lZWRlZCkKICAgIE9i amVjdCAweGZmZmZhMDAwNDkwNDc1Mjg6IHR5cGU9Miwgc2l6ZT0weDIsIHJlcz0yLCByZWY9 NCwgZmxhZ3M9MHgxMDAwIHJ1aWQgLTEgY2hhcmdlIDAKICAgICBzcmVmPTAsIGJhY2tpbmdf b2JqZWN0KDApPSgwKSsweDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDIzMzBkNzIwOiBzdGFy dD0weDQwYjgyMDAwLCBlbmQ9MHg0MGI5MTAwMCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9 MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9MHgxMjAwMCwgY29weSAobmVlZGVkKQogIG1h cCBlbnRyeSAweGZmZmZhMDAwMTJmNTQ5YzA6IHN0YXJ0PTB4NDBiOTEwMDAsIGVuZD0weDQw YjkyMDAwLCBlZmxhZ3M9MHg0LAogICBwcm90PTEvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAw MGRhM2E5ZDY4LCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJnZSAxMDAwLCBjb3B5IChk b25lKQogICAgT2JqZWN0IDB4ZmZmZmEwMDBkYTNhOWQ2ODogdHlwZT0wLCBzaXplPTB4MSwg cmVzPTEsIHJlZj0xLCBmbGFncz0weDMwMTAgcnVpZCAwIGNoYXJnZSAxMDAwCiAgICAgc3Jl Zj0wLCBiYWNraW5nX29iamVjdCg0KT0oMHhmZmZmYTAwMDQ5MDQ3NTI4KSsweDEwMDAKICBt YXAgZW50cnkgMHhmZmZmYTAwMGU0MWZjYjQwOiBzdGFydD0weDQwYjkyMDAwLCBlbmQ9MHg0 MGJhMTAwMCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBv ZmZzZXQ9MHgyMjAwMCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwZGM1 ZGQzNjA6IHN0YXJ0PTB4NDBiYTEwMDAsIGVuZD0weDQwYmEyMDAwLCBlZmxhZ3M9MHg0LAog ICBwcm90PTMvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMGU0MmNlNTI4LCBvZmZzZXQ9MHgw LCBvYmogcnVpZCAwIGNoYXJnZSAxMDAwLCBjb3B5IChkb25lKQogICAgT2JqZWN0IDB4ZmZm ZmEwMDBlNDJjZTUyODogdHlwZT0wLCBzaXplPTB4MSwgcmVzPTEsIHJlZj0xLCBmbGFncz0w eDMwMTAgcnVpZCAwIGNoYXJnZSAxMDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCg0 KT0oMHhmZmZmYTAwMDQ5MDQ3NTI4KSsweDEwMDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDg1 ZGRmZGUwOiBzdGFydD0weDQwYmEyMDAwLCBlbmQ9MHg0MGJhMzAwMCwgZWZsYWdzPTB4NDBj LAogICBwcm90PTEvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMDg1ZmRkMTA4LCBvZmZzZXQ9 MHgwLCBjb3B5IChuZWVkZWQpCiAgICBPYmplY3QgMHhmZmZmYTAwMDg1ZmRkMTA4OiB0eXBl PTIsIHNpemU9MHgyLCByZXM9MiwgcmVmPTQsIGZsYWdzPTB4MTAwMCBydWlkIC0xIGNoYXJn ZSAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5 IDB4ZmZmZmEwMDBiM2NlNWRlMDogc3RhcnQ9MHg0MGJhMzAwMCwgZW5kPTB4NDBiYjIwMDAs IGVmbGFncz0weDEwMDBjLAogICBwcm90PTAvMC9jb3B5LCBvYmplY3Q9MCwgb2Zmc2V0PTB4 MTAwMCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwMzY4NzgxZTA6IHN0 YXJ0PTB4NDBiYjIwMDAsIGVuZD0weDQwYmIzMDAwLCBlZmxhZ3M9MHg0MGMsCiAgIHByb3Q9 NS83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwODVmZGQxMDgsIG9mZnNldD0weDAsIGNvcHkg KG5lZWRlZCkKICAgIE9iamVjdCAweGZmZmZhMDAwODVmZGQxMDg6IHR5cGU9Miwgc2l6ZT0w eDIsIHJlcz0yLCByZWY9NCwgZmxhZ3M9MHgxMDAwIHJ1aWQgLTEgY2hhcmdlIDAKICAgICBz cmVmPTAsIGJhY2tpbmdfb2JqZWN0KDApPSgwKSsweDAKICBtYXAgZW50cnkgMHhmZmZmYTAw MDM4NjRlYWUwOiBzdGFydD0weDQwYmIzMDAwLCBlbmQ9MHg0MGJjMjAwMCwgZWZsYWdzPTB4 MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHksIG9iamVjdD0wLCBvZmZzZXQ9MHgxMTAwMCwgY29w eSAobmVlZGVkKQogIG1hcCBlbnRyeSAweGZmZmZhMDAwMzY4Y2Y0ODA6IHN0YXJ0PTB4NDBi YzIwMDAsIGVuZD0weDQwYmMzMDAwLCBlZmxhZ3M9MHg0LAogICBwcm90PTEvNy9jb3B5LCBv YmplY3Q9MHhmZmZmYTAwMDI3MzQxNDIwLCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJn ZSAxMDAwLCBjb3B5IChkb25lKQogICAgT2JqZWN0IDB4ZmZmZmEwMDAyNzM0MTQyMDogdHlw ZT0wLCBzaXplPTB4MSwgcmVzPTEsIHJlZj0xLCBmbGFncz0weDMwMTAgcnVpZCAwIGNoYXJn ZSAxMDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29iamVjdCg0KT0oMHhmZmZmYTAwMDg1ZmRk MTA4KSsweDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDMyNDVhYzAwOiBzdGFydD0weDQwYmMz MDAwLCBlbmQ9MHg0MGJkMjAwMCwgZWZsYWdzPTB4MTAwMGMsCiAgIHByb3Q9MC8wL2NvcHks IG9iamVjdD0wLCBvZmZzZXQ9MHgyMTAwMCwgY29weSAobmVlZGVkKQogIG1hcCBlbnRyeSAw eGZmZmZhMDAwNzBhMzUwYzA6IHN0YXJ0PTB4NDBiZDIwMDAsIGVuZD0weDQwYmQzMDAwLCBl ZmxhZ3M9MHg0LAogICBwcm90PTMvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMGRhNmZkNDIw LCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJnZSAxMDAwLCBjb3B5IChkb25lKQogICAg T2JqZWN0IDB4ZmZmZmEwMDBkYTZmZDQyMDogdHlwZT0wLCBzaXplPTB4MSwgcmVzPTEsIHJl Zj0xLCBmbGFncz0weDMwMTAgcnVpZCAwIGNoYXJnZSAxMDAwCiAgICAgc3JlZj0wLCBiYWNr aW5nX29iamVjdCg0KT0oMHhmZmZmYTAwMDg1ZmRkMTA4KSsweDAKICBtYXAgZW50cnkgMHhm ZmZmYTAwMDM4Mzk3NGUwOiBzdGFydD0weDQwYzAwMDAwLCBlbmQ9MHg0MTQwMDAwMCwgZWZs YWdzPTAsCiAgIHByb3Q9My83L2NvcHksIG9iamVjdD0weGZmZmZhMDAwYjc4MTljNjAsIG9m ZnNldD0weDAsIG9iaiBydWlkIDAgY2hhcmdlIDgwMDAwMAogICAgT2JqZWN0IDB4ZmZmZmEw MDBiNzgxOWM2MDogdHlwZT0wLCBzaXplPTB4ODAwLCByZXM9NzAsIHJlZj0xLCBmbGFncz0w eDMwMTAgcnVpZCAwIGNoYXJnZSA4MDAwMDAKICAgICBzcmVmPTAsIGJhY2tpbmdfb2JqZWN0 KDApPSgwKSsweDAKICBtYXAgZW50cnkgMHhmZmZmYTAwMDIzMWE5MzAwOiBzdGFydD0weGZm ZmZiZmZmZjAwMCwgZW5kPTB4ZmZmZmZmZmRmMDAwLCBlZmxhZ3M9MHgzMDAwMCwKICAgcHJv dD0wLzAvY29weSwgb2JqZWN0PTAsIG9mZnNldD0weDAKICBtYXAgZW50cnkgMHhmZmZmYTAw MGUxYmJkMTgwOiBzdGFydD0weGZmZmZmZmZkZjAwMCwgZW5kPTB4ZmZmZmZmZmZmMDAwLCBl ZmxhZ3M9MHgxMDAwLAogICBwcm90PTMvNy9jb3B5LCBvYmplY3Q9MHhmZmZmYTAwMGIzODAx MjEwLCBvZmZzZXQ9MHgwLCBvYmogcnVpZCAwIGNoYXJnZSAyMDAwMAogICAgT2JqZWN0IDB4 ZmZmZmEwMDBiMzgwMTIxMDogdHlwZT0wLCBzaXplPTB4MjAsIHJlcz0xMiwgcmVmPTEsIGZs YWdzPTB4MzAxMCBydWlkIDAgY2hhcmdlIDIwMDAwCiAgICAgc3JlZj0wLCBiYWNraW5nX29i amVjdCgwKT0oMCkrMHgwCiAgbWFwIGVudHJ5IDB4ZmZmZmEwMDA1MjY1ZDQ4MDogc3RhcnQ9 MHhmZmZmZmZmZmYwMDAsIGVuZD0weDEwMDAwMDAwMDAwMDAsIGVmbGFncz0wLAogICBwcm90 PTUvNS9zaGFyZSwgb2JqZWN0PTB4ZmZmZmEwMDAwMTA1OWQ2OCwgb2Zmc2V0PTB4MAogICAg T2JqZWN0IDB4ZmZmZmEwMDAwMTA1OWQ2ODogdHlwZT00LCBzaXplPTB4MSwgcmVzPTEsIHJl Zj0xOSwgZmxhZ3M9MHg2IHJ1aWQgLTEgY2hhcmdlIDAKICAgICBzcmVmPTAsIGJhY2tpbmdf b2JqZWN0KDApPSgwKSsweDAKZGI+ --------------47B922A2C808702819A29B19-- From owner-freebsd-current@freebsd.org Sat Oct 17 05:54:43 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0DD9842A935 for ; Sat, 17 Oct 2020 05:54:43 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CCsfn6kxlz4fvY for ; Sat, 17 Oct 2020 05:54:41 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: by mail-wr1-x42f.google.com with SMTP id t9so5717657wrq.11 for ; Fri, 16 Oct 2020 22:54:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding:content-language; bh=jlnRY+JNKGD2VPSQ26Sdk4DICxFbVNh0S7VRBAG8goE=; b=l2L93z0oZFLWsDL2KrRof37bV1zRvRWBw+TAOCpsurGlrSENYOMWVVRhbO0vqpKlE8 fVIwOfGAfO99Hk2ji/9dbNXLx8EaydXVVU0/PhEi94t48R2Nq+n0ZhtiSfJDCbLmkXlN 3RFgdYtCjCbDvhzoEnAsqy4b5ZmBN+vKMtr8jaQ2IUhUOZ0fnwihF3JGAhS0NFcspCc2 CWJlJDAqSqr7vxmF1Rr24lX62tqhv1W5QzfkLFcVehxqAcWBrPIn01jW9pF60ay/lE0K vb4c88J/P2syhBJjcjh3YvLzmlzqHSg1JpnKkU+9b8BzeXnu1h9eRRkAvYjY1nSQVWwW 9MzA== X-Gm-Message-State: AOAM533EqabPJftin4lRjqmlJu45MkVILGM+TOYUTVf1Pgz2EakFU4Ek WXNWYu3h/Mn4s9XzLqHawSXxN8WZS3dMhA== X-Google-Smtp-Source: ABdhPJwqrBbP0cxAXbJCM8xaexu/Pi/r4+KZq05LQ5kQVTillanuQ/aTo6TNWvmNzuk3LkgLV62XwA== X-Received: by 2002:adf:f20f:: with SMTP id p15mr8394901wro.339.1602914078737; Fri, 16 Oct 2020 22:54:38 -0700 (PDT) Received: from [192.168.1.11] (79-66-147-78.dynamic.dsl.as9105.com. [79.66.147.78]) by smtp.gmail.com with ESMTPSA id u2sm5811749wme.1.2020.10.16.22.54.37 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 16 Oct 2020 22:54:37 -0700 (PDT) To: freebsd-current@freebsd.org From: Graham Perrin Subject: OpenZFS: using an encrypted dataset without a prompt for its passphrase Message-ID: Date: Sat, 17 Oct 2020 06:54:37 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4CCsfn6kxlz4fvY X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.14 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.20)[-0.204]; RECEIVED_SPAMHAUS_PBL(0.00)[79.66.147.78:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.964]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.97)[-0.968]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::42f:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 05:54:43 -0000 root@momh167-gjp4-8570p:~ # zfs get all Transcend/VirtualBox | grep -e creation -e key -e crypt Transcend/VirtualBox  creation              Wed Sep  2 19:02 2020     - Transcend/VirtualBox  encryption aes-256-gcm               - Transcend/VirtualBox  keylocation prompt                    local Transcend/VirtualBox  keyformat passphrase                - Transcend/VirtualBox  encryptionroot Transcend/VirtualBox      - Transcend/VirtualBox  keystatus unavailable               - root@momh167-gjp4-8570p:~ # I was prompted in early September but since then, no prompts. I can export and import the pool (Transcend) without entering the passphrase. Is this intended behaviour and if so: how does the pool – or the computer to which I connect the device (a mobile hard disk drive) – know that entry of the phrase is unnecessary? From owner-freebsd-current@freebsd.org Sat Oct 17 06:08:17 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A113B42B460; Sat, 17 Oct 2020 06:08:17 +0000 (UTC) (envelope-from yklaxds@gmail.com) Received: from mail-il1-x12d.google.com (mail-il1-x12d.google.com [IPv6:2607:f8b0:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CCsyS4FQyz3S6m; Sat, 17 Oct 2020 06:08:16 +0000 (UTC) (envelope-from yklaxds@gmail.com) Received: by mail-il1-x12d.google.com with SMTP id j13so5123033ilc.4; Fri, 16 Oct 2020 23:08:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=/q1YxJ5amQii+E4S9TtZWs/+KjruKn7WR8rdNqDbZL0=; b=bQGAhe8GssGMsSOQL1eiQTagmOA/b1RRzGwUcAriX/dakSQIQLOlACXUVGxk5NCbJ4 QCP/ws+qj3wgQos8wCyiFdHVrsfHS0LNyEBpZyMSR75Qfhso/zGmdQ2w3g8+Mc4OClQG ErZRV4IL/EnPU/BlxD4OIc0d4kNNfHF0K36etJB0K61p7WL/8/+zGdz5Uf4sTbMUS6q3 ES3Cqv2grmsSXKQhQOcNBQQBHKULQJ+9F/pePqdPSuMXSNhOX5d4VG/jdFp/tHFU8bSz xFmiAHzyaIionAH+yTjpjWUA4qDm/mOLmsrXfBaSYHAYwjAOUctNu/mtATxGNXeJvnmJ Uvew== X-Gm-Message-State: AOAM532Ty8DvKbB7SL3S7hjWp3/uNgvJpuLWyA7XdcdM0roZrJLylwY6 gnbmzFC5WZQ4F0QzwgMjFsdL9RXPsyWFoe07AOXf860g4B0fMQ== X-Google-Smtp-Source: ABdhPJwH1HhJpmuc7fIoQ8p6SQNZbNZ+MtlSK2qcUVdCd5KP1TK/6LWqdgBUQf527Hoj7WbUzXOTAJ0mcoPagvK5a2o= X-Received: by 2002:a92:cd02:: with SMTP id z2mr401119iln.247.1602914894156; Fri, 16 Oct 2020 23:08:14 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a92:a10e:0:0:0:0:0 with HTTP; Fri, 16 Oct 2020 23:08:13 -0700 (PDT) From: ykla Date: Sat, 17 Oct 2020 14:08:13 +0800 Message-ID: Subject: Status of SDIO To: freebsd-current@freebsd.org, freebsd-arm@freebsd.org X-Rspamd-Queue-Id: 4CCsyS4FQyz3S6m X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-0.98)[-0.983]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.002]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::12d:from]; NEURAL_HAM_SHORT(-0.51)[-0.511]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-arm]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 06:08:17 -0000 SDIO is used in raspberry pi. Rpi WiFi network needs SDIO drivers. But WIKI that about SDIO is not update for a long time. https://wiki.freebsd.org/SDIO So what's status of SDIO development? And maybe WIFI works well in raspberry pi? From owner-freebsd-current@freebsd.org Sat Oct 17 07:40:39 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 22C8742E3C2 for ; Sat, 17 Oct 2020 07:40:39 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CCw1305vFz3XLb for ; Sat, 17 Oct 2020 07:40:39 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Received: from Ryans-MBP.attlocal.net (unknown [IPv6:2600:1700:358a:c660:69bf:cad1:99bc:22ec]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: freqlabs/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id C68E81F8C9 for ; Sat, 17 Oct 2020 07:40:38 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Subject: Re: OpenZFS: using an encrypted dataset without a prompt for its passphrase To: freebsd-current@freebsd.org References: From: Ryan Moeller Message-ID: <4fb31ed5-2281-13cf-e45e-28dae27f26b3@FreeBSD.org> Date: Sat, 17 Oct 2020 03:40:38 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 07:40:39 -0000 On 10/17/20 1:54 AM, Graham Perrin wrote: > root@momh167-gjp4-8570p:~ # zfs get all Transcend/VirtualBox | grep -e > creation -e key -e crypt > Transcend/VirtualBox  creation              Wed Sep  2 19:02 2020     - > Transcend/VirtualBox  encryption aes-256-gcm               - > Transcend/VirtualBox  keylocation prompt                    local > Transcend/VirtualBox  keyformat passphrase                - > Transcend/VirtualBox  encryptionroot Transcend/VirtualBox      - > Transcend/VirtualBox  keystatus unavailable               - > root@momh167-gjp4-8570p:~ # > > I was prompted in early September but since then, no prompts. > > I can export and import the pool (Transcend) without entering the > passphrase. > > Is this intended behaviour and if so: how does the pool – or the > computer to which I connect the device (a mobile hard disk drive) – > know that entry of the phrase is unnecessary? This is intentional. The pool can be imported but the filesystem is not mounted until the key is loaded. See zfs-load-key(8) -Ryan > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Sat Oct 17 09:55:19 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5286F431E32 for ; Sat, 17 Oct 2020 09:55:19 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CCz0Q2xFsz3flD for ; Sat, 17 Oct 2020 09:55:18 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: by mail-wr1-x434.google.com with SMTP id j7so2060133wrt.9 for ; Sat, 17 Oct 2020 02:55:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=zTryvrtkjSqW6qo4nSuhP7YR69jmhf2UmDhWradByy8=; b=ga5jdY1c73CAv+0fEuK7ACBklGxDuptjZRUQstXJQNwspE35Ba+cayDai+qnKiK7Ft 6iLEI7asFkeoZD4c5BArMYN7pgmJQI7wBzRcCIGCPW5S+8rIBdN93Y0VoH9fjckAKwA4 o1IF4oL3g02HfAjfL4hG6f79If/7vik4t0rPabhmbfz5xuXJX7AXhs4au811jI7k8C3A iGKZuCYozXEpw7yNZMOaUin4YrZz9o/m4eKA0mVx2HZgprMrKSp98EwF/DOCGREVeOKe wT1E5ZkhTcsQSla3Mq0VWXS91oxAvy1v+4ZnCV6oPOS15y7UsohD+0PbvqjiOjZf6dIH ZwDA== X-Gm-Message-State: AOAM531TYn88auVEWqpwqIsp9D8ph8uxiZKj70gAMWZjnoHAyHhQsA5P bsO9LxCF/Jr7+RzSZvJ2MJNAtKL9MzpzTw== X-Google-Smtp-Source: ABdhPJyMHJ7fhj/rcjIFJzPD7rntipmwuE3yLX2bc2oIybLpFFmo9yULAMdvDTkI3CO7A8SM8i5jVA== X-Received: by 2002:a5d:4e48:: with SMTP id r8mr536315wrt.141.1602928515621; Sat, 17 Oct 2020 02:55:15 -0700 (PDT) Received: from [192.168.1.11] (79-66-147-78.dynamic.dsl.as9105.com. [79.66.147.78]) by smtp.gmail.com with ESMTPSA id p4sm7941003wrf.67.2020.10.17.02.55.14 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 17 Oct 2020 02:55:14 -0700 (PDT) Subject: Re: OpenZFS: using an encrypted dataset without a prompt for its passphrase To: freebsd-current@freebsd.org References: <4fb31ed5-2281-13cf-e45e-28dae27f26b3@FreeBSD.org> From: Graham Perrin Message-ID: Date: Sat, 17 Oct 2020 10:55:14 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <4fb31ed5-2281-13cf-e45e-28dae27f26b3@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 4CCz0Q2xFsz3flD X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.13 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.18)[-0.175]; RECEIVED_SPAMHAUS_PBL(0.00)[79.66.147.78:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.961]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.989]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::434:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 09:55:19 -0000 On 17/10/2020 08:40, Ryan Moeller wrote: > This is intentional. The pool can be imported but the filesystem is > not mounted until the key is loaded. Thanks, the file system mounts without me entering a passphrase; is this intentional? From owner-freebsd-current@freebsd.org Sat Oct 17 11:35:20 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E090C43423E for ; Sat, 17 Oct 2020 11:35:20 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CD1Cr5fyFz41nm for ; Sat, 17 Oct 2020 11:35:20 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Received: from Ryans-MBP.attlocal.net (unknown [IPv6:2600:1700:358a:c660:4841:5ee9:47c1:72c7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: freqlabs/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 91DF92150C for ; Sat, 17 Oct 2020 11:35:20 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Subject: Re: OpenZFS: using an encrypted dataset without a prompt for its passphrase To: freebsd-current@freebsd.org References: <4fb31ed5-2281-13cf-e45e-28dae27f26b3@FreeBSD.org> From: Ryan Moeller Message-ID: Date: Sat, 17 Oct 2020 07:35:19 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 11:35:20 -0000 On 10/17/20 5:55 AM, Graham Perrin wrote: > On 17/10/2020 08:40, Ryan Moeller wrote: >> This is intentional. The pool can be imported but the filesystem is >> not mounted until the key is loaded. > > Thanks, the file system mounts without me entering a passphrase; is > this intentional? > It shouldn't be possible. # zfs mount storage/crypt cannot mount 'storage/crypt': encryption key not loaded > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Sat Oct 17 12:53:51 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EF86043741A; Sat, 17 Oct 2020 12:53:51 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CD2yQ6d23z45xR; Sat, 17 Oct 2020 12:53:50 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 2A60E8D4A179; Sat, 17 Oct 2020 12:53:43 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id B2532E70813; Sat, 17 Oct 2020 12:53:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id mM6lGoTUb5L3; Sat, 17 Oct 2020 12:53:41 +0000 (UTC) Received: from [169.254.148.91] (unknown [IPv6:fde9:577b:c1a9:4902:280e:7b69:92fc:710f]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 80E15E707B1; Sat, 17 Oct 2020 12:53:41 +0000 (UTC) From: "Bjoern A. Zeeb" To: ykla Cc: freebsd-current@freebsd.org, freebsd-arm@freebsd.org Subject: Re: Status of SDIO Date: Sat, 17 Oct 2020 12:53:40 +0000 X-Mailer: MailMate (2.0BETAr6151) Message-ID: <78BF5EEB-D138-43EE-A14D-25A7BFFC91EB@lists.zabbadoz.net> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-Rspamd-Queue-Id: 4CD2yQ6d23z45xR X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.11 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.201.62.131]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zabbadoz.net]; NEURAL_HAM_LONG(-0.96)[-0.959]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.90)[-0.896]; NEURAL_HAM_MEDIUM(-0.95)[-0.953]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm,freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 12:53:52 -0000 On 17 Oct 2020, at 6:08, ykla wrote: > SDIO is used in raspberry pi. Rpi WiFi network needs SDIO drivers. > But > WIKI that about SDIO is not update for a long time. > https://wiki.freebsd.org/SDIO > So what's status of SDIO development? And maybe WIFI works well in > raspberry pi? SDIO went in and works with MMCCAM. The brcmfmac driver I am still working on in my free time. Apparently that is also the only driver anyone ever seems interest in so far as no others have emerged either. /bz From owner-freebsd-current@freebsd.org Sat Oct 17 13:02:52 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EE31F437945 for ; Sat, 17 Oct 2020 13:02:52 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CD38q6psJz46bw for ; Sat, 17 Oct 2020 13:02:51 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: by mail-wr1-x429.google.com with SMTP id n6so6420957wrm.13 for ; Sat, 17 Oct 2020 06:02:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=9t0KTduQ+cDPrZfBN667HD74BtlbmXcXEIO/rA4Nb1s=; b=lXzrCjbR7b5gyO5+wKv+VFYCEIy/50Tts81PU4FeBWztwSrAUESzDdG2j+eHnDY3pf /W5xHMsH9z2I9EiHzbO/p2iCD5HYnw94h30YyM6a05unJAZjGKDpkH4HcurqtA+3dlTf pJFK6YyrEL4X6SlLYXczloE2sOfkAeDWjEBXSUc+ZyzqO879xhZRL8kZmu2jd6eekRht PJMLY3/eUN//rSVn8xhE2WMxor8DR7vJKKlPo+t1sM4S0tr3UK26NcLRzLHgwMntkb3c 7ZVz4IircCkaOXGXN1NJlJ/4WpTSbDF2lqvMamyd77h8U6i2iftNAlsGSYW6VdBVc25V xq+A== X-Gm-Message-State: AOAM533wkrqUkrqQ4EGjfy2ECvYicR3S4y009pcmT7lTuyLTMsWlH/hA hATsPmJzwy6g2MvxmJuT1v0dOeioA/wImQ== X-Google-Smtp-Source: ABdhPJx5Jn3cCMHgyLKGRa6m8FbF1He+rAPaB+VDQAlhsY9vukm7cTePmyLa0tU2tHbAIoWob7S/CA== X-Received: by 2002:a5d:4144:: with SMTP id c4mr9804636wrq.311.1602939768969; Sat, 17 Oct 2020 06:02:48 -0700 (PDT) Received: from [192.168.1.11] (79-66-147-78.dynamic.dsl.as9105.com. [79.66.147.78]) by smtp.gmail.com with ESMTPSA id m1sm7133129wmm.34.2020.10.17.06.02.48 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 17 Oct 2020 06:02:48 -0700 (PDT) Subject: Re: OpenZFS: using an encrypted dataset without a prompt for its passphrase To: freebsd-current@freebsd.org References: <4fb31ed5-2281-13cf-e45e-28dae27f26b3@FreeBSD.org> From: Graham Perrin Message-ID: <267214f8-a099-b16c-1a13-c082ec95c4b8@gmail.com> Date: Sat, 17 Oct 2020 14:02:47 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4CD38q6psJz46bw X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.76 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.84)[-0.839]; RECEIVED_SPAMHAUS_PBL(0.00)[79.66.147.78:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.94)[-0.936]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.987]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::429:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 13:02:53 -0000 On 17/10/2020 12:35, Ryan Moeller wrote: > > On 10/17/20 5:55 AM, Graham Perrin wrote: >> On 17/10/2020 08:40, Ryan Moeller wrote: >>> This is intentional. The pool can be imported but the filesystem is >>> not mounted until the key is loaded. >> >> Thanks, the file system mounts without me entering a passphrase; is >> this intentional? >> > > It shouldn't be possible. > > # zfs mount storage/crypt > cannot mount 'storage/crypt': encryption key not loaded root@momh167-gjp4-8570p:~ # date ; uname -v ; uptime Sat Oct 17 14:00:10 BST 2020 FreeBSD 13.0-CURRENT #69 r366648: Tue Oct 13 05:49:05 BST 2020 root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG  2:00PM  up 9 mins, 5 users, load averages: 0.29, 0.56, 0.31 root@momh167-gjp4-8570p:~ # zpool export Transcend && ls -hl /Volumes/t500/VirtualBox ; zpool import Transcend && ls -hl /Volumes/t500/VirtualBox ls: /Volumes/t500/VirtualBox: No such file or directory total 18 drwxr-xr-x  2 grahamperrin  grahamperrin     2B Sep 11 19:28 CloudReady drwxr-xr-x  6 grahamperrin  grahamperrin     6B May  8 09:04 FreeBSD drwxr-xr-x  4 grahamperrin  grahamperrin     4B Sep 20 17:03 Linux drwxr-xr-x  4 grahamperrin  grahamperrin     7B Oct 16 17:41 Windows root@momh167-gjp4-8570p:~ # zfs get all Transcend/VirtualBox | grep -e crypt -e key -e mountpoint | sort Transcend/VirtualBox  encryption aes-256-gcm               - Transcend/VirtualBox  encryptionroot Transcend/VirtualBox      - Transcend/VirtualBox  keyformat passphrase                - Transcend/VirtualBox  keylocation prompt                    local Transcend/VirtualBox  keystatus unavailable               - Transcend/VirtualBox  mountpoint /Volumes/t500/VirtualBox  inherited from Transcend root@momh167-gjp4-8570p:~ # zfs --version zfs-0.8.0-1 zfs-kmod-v2020100400-zfs_79f0935fa root@momh167-gjp4-8570p:~ # From owner-freebsd-current@freebsd.org Sat Oct 17 13:08:08 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 47DBE437EA4 for ; Sat, 17 Oct 2020 13:08:08 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CD3Gw1BvZz47j4 for ; Sat, 17 Oct 2020 13:08:08 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Received: from Ryans-MBP.attlocal.net (unknown [IPv6:2600:1700:358a:c660:8100:aa15:a7e4:9e5e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: freqlabs/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id EB33921AF4 for ; Sat, 17 Oct 2020 13:08:07 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Subject: Re: OpenZFS: using an encrypted dataset without a prompt for its passphrase To: freebsd-current@freebsd.org References: <4fb31ed5-2281-13cf-e45e-28dae27f26b3@FreeBSD.org> <267214f8-a099-b16c-1a13-c082ec95c4b8@gmail.com> From: Ryan Moeller Message-ID: <208f93cb-1228-6631-aa9e-afb83c43d4e0@FreeBSD.org> Date: Sat, 17 Oct 2020 09:08:07 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: <267214f8-a099-b16c-1a13-c082ec95c4b8@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 13:08:08 -0000 On 10/17/20 9:02 AM, Graham Perrin wrote: > root@momh167-gjp4-8570p:~ # date ; uname -v ; uptime > Sat Oct 17 14:00:10 BST 2020 > FreeBSD 13.0-CURRENT #69 r366648: Tue Oct 13 05:49:05 BST 2020 > root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG >  2:00PM  up 9 mins, 5 users, load averages: 0.29, 0.56, 0.31 > root@momh167-gjp4-8570p:~ # zpool export Transcend && ls -hl > /Volumes/t500/VirtualBox ; zpool import Transcend && ls -hl > /Volumes/t500/VirtualBox > ls: /Volumes/t500/VirtualBox: No such file or directory > total 18 > drwxr-xr-x  2 grahamperrin  grahamperrin     2B Sep 11 19:28 CloudReady > drwxr-xr-x  6 grahamperrin  grahamperrin     6B May  8 09:04 FreeBSD > drwxr-xr-x  4 grahamperrin  grahamperrin     4B Sep 20 17:03 Linux > drwxr-xr-x  4 grahamperrin  grahamperrin     7B Oct 16 17:41 Windows > root@momh167-gjp4-8570p:~ # zfs get all Transcend/VirtualBox | grep -e > crypt -e key -e mountpoint | sort > Transcend/VirtualBox  encryption aes-256-gcm               - > Transcend/VirtualBox  encryptionroot Transcend/VirtualBox      - > Transcend/VirtualBox  keyformat passphrase                - > Transcend/VirtualBox  keylocation prompt                    local > Transcend/VirtualBox  keystatus unavailable               - > Transcend/VirtualBox  mountpoint /Volumes/t500/VirtualBox inherited > from Transcend > root@momh167-gjp4-8570p:~ # zfs --version > zfs-0.8.0-1 > zfs-kmod-v2020100400-zfs_79f0935fa > root@momh167-gjp4-8570p:~ # This doesn't necessarily mean the encrypted filesystem is mounted though. The contents you are seeing must be in the parent filesystem. Check the output of the mount command, you should find Transcend/VirtualBox is not mounted. From owner-freebsd-current@freebsd.org Sat Oct 17 16:48:49 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 21CC043CAFA for ; Sat, 17 Oct 2020 16:48:49 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CD89W6Np2z4P9N for ; Sat, 17 Oct 2020 16:48:47 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: by mail-wm1-x32d.google.com with SMTP id d81so6393431wmc.1 for ; Sat, 17 Oct 2020 09:48:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=iz1WspmREk2+pxRE6LLgdcBeB1CtCVbadZLNsCDSxME=; b=L0y7Sc+qS/j2yyKdtT67toet5cmZInf+Etx0uRWS5RX1X98wDhfeiIIyspo/Kn7sQv ktHO1D1ukze6Pi7EGPmO20CSCINWV06xxKQRcG+pk9yFklb/LqdS8yPIMHKmWKRWY52l QxTH93M00mCYeX0qXDv/NWeVo3HTZO5BilEATeuKkfI+RjFlSA53a9s7DRZiF5nsdbkU gsLcpBqVQPR+GqjmarjB2/GChujj+OlgdaPxKdRUXEE6asigMCktkJMwOrTBsATxPb8l NFif7iYjOS9T642bAS3m9fB6TQgrVIHOxu5zJLP0jFf0PUWl+cU99kqwv0sgulpXF5bZ PYfw== X-Gm-Message-State: AOAM533sMUf/TZPWz8MYdGwCvbBlxQbCPUbAZYLWwpxhYatdJi9Zbhun OXa6hwnVVvr9BHTobxqrgz/iJbAMG/7g+Q== X-Google-Smtp-Source: ABdhPJyOka4oQNLArNgwPl0jpiUcgyX2V5K8ivCLe9oZlQ6t+cpmCb0v9hqb4pzsE+n+7/YUemaivg== X-Received: by 2002:a1c:7dc5:: with SMTP id y188mr8994791wmc.37.1602953325681; Sat, 17 Oct 2020 09:48:45 -0700 (PDT) Received: from [192.168.1.11] (79-66-147-78.dynamic.dsl.as9105.com. [79.66.147.78]) by smtp.gmail.com with ESMTPSA id g144sm8669304wmg.30.2020.10.17.09.48.44 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 17 Oct 2020 09:48:44 -0700 (PDT) Subject: OpenZFS: encrypted dataset confusion (PEBKAM) To: freebsd-current@freebsd.org References: <4fb31ed5-2281-13cf-e45e-28dae27f26b3@FreeBSD.org> <267214f8-a099-b16c-1a13-c082ec95c4b8@gmail.com> <208f93cb-1228-6631-aa9e-afb83c43d4e0@FreeBSD.org> From: Graham Perrin Message-ID: Date: Sat, 17 Oct 2020 17:48:44 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <208f93cb-1228-6631-aa9e-afb83c43d4e0@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4CD89W6Np2z4P9N X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.89 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RECEIVED_SPAMHAUS_PBL(0.00)[79.66.147.78:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.94)[-0.936]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.988]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_SHORT(0.03)[0.033]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::32d:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 16:48:49 -0000 On 17/10/2020 14:08, Ryan Moeller wrote: > On 10/17/20 9:02 AM, Graham Perrin wrote: >> root@momh167-gjp4-8570p:~ # date ; uname -v ; uptime >> Sat Oct 17 14:00:10 BST 2020 >> FreeBSD 13.0-CURRENT #69 r366648: Tue Oct 13 05:49:05 BST 2020 >> root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG >>  2:00PM  up 9 mins, 5 users, load averages: 0.29, 0.56, 0.31 >> root@momh167-gjp4-8570p:~ # zpool export Transcend && ls -hl >> /Volumes/t500/VirtualBox ; zpool import Transcend && ls -hl >> /Volumes/t500/VirtualBox >> ls: /Volumes/t500/VirtualBox: No such file or directory >> total 18 >> drwxr-xr-x  2 grahamperrin  grahamperrin     2B Sep 11 19:28 CloudReady >> drwxr-xr-x  6 grahamperrin  grahamperrin     6B May  8 09:04 FreeBSD >> drwxr-xr-x  4 grahamperrin  grahamperrin     4B Sep 20 17:03 Linux >> drwxr-xr-x  4 grahamperrin  grahamperrin     7B Oct 16 17:41 Windows >> root@momh167-gjp4-8570p:~ # zfs get all Transcend/VirtualBox | grep >> -e crypt -e key -e mountpoint | sort >> Transcend/VirtualBox  encryption aes-256-gcm               - >> Transcend/VirtualBox  encryptionroot Transcend/VirtualBox      - >> Transcend/VirtualBox  keyformat passphrase                - >> Transcend/VirtualBox  keylocation prompt local >> Transcend/VirtualBox  keystatus unavailable               - >> Transcend/VirtualBox  mountpoint /Volumes/t500/VirtualBox inherited >> from Transcend >> root@momh167-gjp4-8570p:~ # zfs --version >> zfs-0.8.0-1 >> zfs-kmod-v2020100400-zfs_79f0935fa >> root@momh167-gjp4-8570p:~ # > > > This doesn't necessarily mean the encrypted filesystem is mounted > though. The contents you are > seeing must be in the parent filesystem. > > Check the output of the mount command, you should find > Transcend/VirtualBox is not mounted. True! Thank you. I didn't realise that from the outset I had written to the non-encrypted parent. Fixed: ---- root@momh167-gjp4-8570p:~ # mount | grep Transcend Transcend on /Volumes/t500 (zfs, local, nfsv4acls) root@momh167-gjp4-8570p:~ # cd /Volumes/t500/ root@momh167-gjp4-8570p:/Volumes/t500 # mv VirtualBox vbox root@momh167-gjp4-8570p:/Volumes/t500 # zfs create -o encryption=on -o keyformat=passphrase Transcend/VirtualBox cannot create 'Transcend/VirtualBox': dataset already exists root@momh167-gjp4-8570p:/Volumes/t500 # zfs destroy Transcend/VirtualBox root@momh167-gjp4-8570p:/Volumes/t500 # ls -hl vbox total 18 drwxr-xr-x  2 grahamperrin  grahamperrin     2B Sep 11 19:28 CloudReady drwxr-xr-x  6 grahamperrin  grahamperrin     6B May  8 09:04 FreeBSD drwxr-xr-x  4 grahamperrin  grahamperrin     4B Sep 20 17:03 Linux drwxr-xr-x  4 grahamperrin  grahamperrin     7B Oct 16 17:41 Windows root@momh167-gjp4-8570p:/Volumes/t500 # zfs create -o encryption=on -o keyformat=passphrase Transcend/VirtualBox Enter passphrase: Re-enter passphrase: root@momh167-gjp4-8570p:/Volumes/t500 # mount | grep Transcend Transcend on /Volumes/t500 (zfs, local, nfsv4acls) Transcend/VirtualBox on /Volumes/t500/VirtualBox (zfs, local, nfsv4acls) root@momh167-gjp4-8570p:/Volumes/t500 # zpool status -v Transcend   pool: Transcend  state: ONLINE   scan: scrub repaired 0B in 01:11:28 with 0 errors on Sun Oct 11 12:35:27 2020 config:         NAME        STATE     READ WRITE CKSUM         Transcend   ONLINE       0     0     0           da0p1     ONLINE       0     0     0 errors: No known data errors root@momh167-gjp4-8570p:/Volumes/t500 # time mv vbox/* VirtualBox/ 0.630u 1378.236s 3:16:17.32 11.7%       15+167k 0+0io 235pf+0w root@momh167-gjp4-8570p:/Volumes/t500 # From owner-freebsd-current@freebsd.org Sat Oct 17 18:00:20 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EDFD143E591 for ; Sat, 17 Oct 2020 18:00:20 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward101p.mail.yandex.net (forward101p.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CD9m351dFz4SLw for ; Sat, 17 Oct 2020 18:00:19 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from mxback20o.mail.yandex.net (mxback20o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::71]) by forward101p.mail.yandex.net (Yandex) with ESMTP id 1DDC52643B93; Sat, 17 Oct 2020 21:00:15 +0300 (MSK) Received: from myt6-9bdf92ffd111.qloud-c.yandex.net (myt6-9bdf92ffd111.qloud-c.yandex.net [2a02:6b8:c12:468a:0:640:9bdf:92ff]) by mxback20o.mail.yandex.net (mxback/Yandex) with ESMTP id ifY1PYTjgd-0E4apEIg; Sat, 17 Oct 2020 21:00:15 +0300 Received: by myt6-9bdf92ffd111.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id xwtcdObZkG-0DI4AjuI; Sat, 17 Oct 2020 21:00:13 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Subject: Re: kernel panic and fun debugging To: Steve Kargl , freebsd-current@freebsd.org References: <20201015065632.GA72261@troutmask.apl.washington.edu> From: "Andrey V. Elsukov" Autocrypt: addr=bu7cher@yandex.ru; prefer-encrypt=mutual; keydata= mQENBEwBF1kBCADB9sXFhBEUy8qQ4X63Y8eBatYMHGEFWN9ypS5lI3RE6qQW2EYbxNk7qUC5 21YIIS1mMFVBEfvR7J9uc7yaYgFCEb6Sce1RSO4ULN2mRKGHP3/Sl0ijZEjWHV91hY1YTHEF ZW/0GYinDf56sYpDDehaBF5wkWIo1+QK5nmj3vl0DIDCMNd7QEiWpyLVwECgLX2eOAXByT8B bCqVhJGcG6iFP7/B9Ll6uX5gb8thM9LM+ibwErDBVDGiOgvfxqidab7fdkh893IBCXa82H9N CNwnEtcgzh+BSKK5BgvPohFMgRwjti37TSxwLu63QejRGbZWSz3OK3jMOoF63tCgn7FvABEB AAG0JUFuZHJleSBWLiBFbHN1a292IDxidTdjaGVyQHlhbmRleC5ydT6JATgEEwECACIFAkwB F1kCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEAHF6gQQyKF6qmYIAI6ekfm1VA4T vqankI1ISE6ku4jV7UlpIQlEbE7/8n3Zd6teJ+pGOQhN5qk8QE7utdPdbktAzi+x7LIJVzUw 4TywZLXGrkP7VKYkfg6oyCGyzITghefQeJtr2TN4hYCkzPWpylkue8MtmqfZv/6royqwTbN+ +E09FQNvTgRUYJYTeQ1qOsxNRycwvw3dr2rOfuxShbzaHBB1pBIjGrMg8fC5pd65ACH5zuFV A0CoTNGMDrEZSfBkTW604UUHFFXeCoC3dwDZRKOWJ3GmMXns65Ai5YkA63BSHEE1Qle3VBhd cG1w0CB5FBV3pB27UVnf0jEbysrDqW4qN7XMRFSWNAy5AQ0ETAEXWQEIAJ2p6l9LBoqdH/0J PEFDY2t2gTvAuzz+8zs3R03dFuHcNbOwjvWCG0aOmVpAzkRa8egn5JB4sZaFUtKPYJEQ1Iu+ LUBwgvtXf4vWpzC67zs2dDuiW4LamH5p6xkTD61aHR7mCB3bg2TUjrDWn2Jt44cvoYxj3dz4 S49U1rc9ZPgD5axCNv45j72tggWlZvpefThP7xT1OlNTUqye2gAwQravXpZkl5JG4eOqJVIU X316iE3qso0iXRUtO7OseBf0PiVmk+wCahdreHOeOxK5jMhYkPKVn7z1sZiB7W2H2TojbmcK HZC22sz7Z/H36Lhg1+/RCnGzdEcjGc8oFHXHCxUAEQEAAYkBHwQYAQIACQUCTAEXWQIbDAAK CRABxeoEEMihegkYCAC3ivGYNe2taNm/4Nx5GPdzuaAJGKWksV+w9mo7dQvU+NmI2az5w8vw 98OmX7G0OV9snxMW+6cyNqBrVFTu33VVNzz9pnqNCHxGvj5dL5ltP160JV2zw2bUwJBYsgYQ WfyJJIM7l3gv5ZS3DGqaGIm9gOK1ANxfrR5PgPzvI9VxDhlr2juEVMZYAqPLEJe+SSxbwLoz BcFCNdDAyXcaAzXsx/E02YWm1hIWNRxanAe7Vlg7OL+gvLpdtrYCMg28PNqKNyrQ87LQ49O9 50IIZDOtNFeR0FGucjcLPdS9PiEqCoH7/waJxWp6ydJ+g4OYRBYNM0EmMgy1N85JJrV1mi5i Message-ID: <3fdb18c1-5dc3-2e06-8825-db1daa425afb@yandex.ru> Date: Sat, 17 Oct 2020 20:57:31 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20201015065632.GA72261@troutmask.apl.washington.edu> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="ZVP9XMy2CeiqLuDlD6F4gceZ5i1XgJyZj" X-Rspamd-Queue-Id: 4CD9m351dFz4SLw X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.59 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yandex.ru]; R_SPF_ALLOW(-0.20)[+ip6:2a02:6b8:0:1000::/52]; HAS_ATTACHMENT(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[yandex.ru:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; NEURAL_HAM_SHORT(-1.42)[-1.422]; SIGNED_PGP(-2.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[2a02:6b8:0:1472:2741:0:8b7:101:from]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:13238, ipnet:2a02:6b8::/32, country:RU]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.976]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yandex.ru:dkim]; NEURAL_HAM_LONG(-0.99)[-0.994]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 18:00:21 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ZVP9XMy2CeiqLuDlD6F4gceZ5i1XgJyZj Content-Type: multipart/mixed; boundary="sUtwCpO9b5EpfDSJhEeIMkBBwcaDa32Cx" --sUtwCpO9b5EpfDSJhEeIMkBBwcaDa32Cx Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 15.10.2020 09:56, Steve Kargl wrote: > Just had a kernel panic. Best info I give you is >=20 > % uname -a > FreeBSD mobile 13.0-CURRENT FreeBSD 13.0-CURRENT #1 r366176M: Sat Sep 2= 6 10:35:23 PDT 2020 kargl@mobile:/usr/obj/usr/src/i386.i386/sys/MOBIL= E i386 >=20 > % kgdb gdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 > ... > Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug... > /usr/ports/devel/gdb/work-py37/gdb-9.2/gdb/inferior.c:283: internal-err= or: struct inferior *find_inferior_pid(int): Assertion `pid !=3D 0' faile= d. > A problem internal to GDB has been detected, > further debugging may prove unreliable. Hi, do you have /var/crash/core.txt.1 file? It may have some useful info. Also did you try an old version /usr/libexec/kgdb ? --=20 WBR, Andrey V. Elsukov --sUtwCpO9b5EpfDSJhEeIMkBBwcaDa32Cx-- --ZVP9XMy2CeiqLuDlD6F4gceZ5i1XgJyZj Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAl+LMIsACgkQAcXqBBDI oXp71Qf/RQzaycQACMOomWJhqxWpU4nceK6HORCiIAYVoTP8E4yrTJk7mbTom7nV fUyuld+48/xQaqGW09QToCD8FkdxYAyt9ZE7K4SSRQxVDJxVas9BRllslHu4sSr+ ZVrk4KiXd1v2z6/zcqpJgXjCul9LcXXPSdM8Eo43rkT9dDPqloSHsax5JYCEYFss 1ijDAYIkc7tIQfVZ+BmnADXYxqzxfK3frN82V9yak/zdKnSg9MEGoX/T2cb1ynww p8YqbMZZFkCOsN5puA1LEbrSbcElfU7MSDqtNj3sc6kYYQabQd6ViOrrvzj0mvel ZjkhH5lpsecVqRhgP6mYEARvGbJj4A== =e/Aq -----END PGP SIGNATURE----- --ZVP9XMy2CeiqLuDlD6F4gceZ5i1XgJyZj-- From owner-freebsd-current@freebsd.org Sat Oct 17 18:49:45 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 34DB243F0B5 for ; Sat, 17 Oct 2020 18:49:45 +0000 (UTC) (envelope-from bergerkos@yahoo.co.uk) Received: from sonic305-20.consmr.mail.ir2.yahoo.com (sonic305-20.consmr.mail.ir2.yahoo.com [77.238.177.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDBs33XJ7z4VBQ for ; Sat, 17 Oct 2020 18:49:42 +0000 (UTC) (envelope-from bergerkos@yahoo.co.uk) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.co.uk; s=s2048; t=1602960580; bh=NwhTKfKe69bFbPQwLouL/f7UP3f11dwftnq//yM7YFH=; h=Date:From:To:Subject; b=WWPEEe+ZQ7M35St7OkbPojL6Z6xhDBYimg7PANmnSwnM3uqNk+3ZL1trlsWvokMMlVIiR43mU7gclu0y81c5XG1OK5BPgD/outAva+4ETWu+T8pq/k9ja5qgwddsrzrlMqHcgstUyB4QUqEj/8FH3FBsX1vzM3XX/L7bWVNMbssyrIUdbGz33eSCNb3uG/S10NIwicLsjLxcEe6sTdL1RdVJscm5tytcXbXUOisCivZ5+IWVw0UpZIIfykdP/LOERP5sotOd7GmKxAeuYnaaWgAltVX2loCKXBNjmWw/ZqJ8Gyax0ramRMScgF0/BufQHunz7J1wwJPMgNlfl2GcQw== X-YMail-OSG: IwbjaIYVM1nyvb75osPxwems04OI8VSLFsID0VqUthJgT7WYF8Hv6tComPTS_bC B9BfjNtOn7XnkbUdnicHFUCzvQ_YntZJhlsTVQUew_oA6hfje8QD5gZoJjM7cf3KWTMIezwWZwKe WkhtOqo8YOK_FvUU2OOWkcV94uWkXelYD9w.84KtfbGJFjSN5OeKcE1U5o1cVM1EEtUOeVr0InWJ S_AgHbDbrp0VB2c6wvgr7SQMLqqoi09DDu3OHqkQmBp5zKU9Iz7ftOIX9AQiCCCSnmuP4k6SRGXP ET93mvcuUw4VfFV3VSK2.TbpXG6142DHZbst4LOQqVRxcq2kW2XIcEBU.nUepC8UEFCicuPFq09l 2t.9986X0hoKwxgM_8vXiTS.8fp2Yr3XySdxawF_uCdrxZ6hmkCPa0t.16j1JzOJeFKmGXCR7nsG UMX2HiSJ1QsofZdQXrcOW5XSAoUdbMgq1j1U4G49oTYpDY11O0CKc5CHIMr5VaE9ZKuiWkBvVY5_ KNmQkzWuZrHaLQxOsoenzz_bkCJFYVPcIrIDA36fj4n.NdCacASHJfAwYAwGOElF3UW2.AtT__TK plCz_7XS4JmcXMa1vOgQISMHeSWExtxZYZxvpvWWaPmyPkz_bnW4fHuKMC6Iw1WZauLP6paU4Jbx 0L1hjii5YefJVLrTxpGRYh5.mEMkIyeJoLfz6ImCbQqt2EZ1.KQaIIpDJr8Uf4Du4qzFN_MIObaE JlrfDHXCZHdtvhUSAC2Wx.rVd6Wtds86yf2qX1kAWW4Fk2618b7KsEVzF9GayquFS4brird_1N03 TvV67sFgD56A0EZJzxQFqxayNvmD_RRR3MD5OSWOs.EAq3I03PAvf_yw_KvELIogsR_glIr0i3WK KLqtz96eRH76eAkY0RN4CxsF.UclYMMFi5BrUcvw68i9AUo8BOGD1rprpR23i0x7rfBHeo7wg0l8 yOwNvHx2r1olw5BLEEVvp5y.ZzIx4Cbp8BZauxjfnXsJW4ouKcXM_R7oUYFg5BL5aTtyqNnd7ZGe 1islJNMVaSfgbuiLFcLTk2ghi1s.KmXB6xq.hk1Qo_tMgvX7DlV1z6.aP6maP1UJHdxMZU7z0z9K Wz5t6EBzH6WxQQM1llT.FQPF707mwe55XWJFcy4pIWnpH6ZOOxyCUZj_2l.BOkszbHT18e0kyb9a Cusmn.QjzepCcSHtdaIBspIycWCmb4U_9E1m9DD3llwRLaN3XlXCbuLrTyPQsHy4KveHY4neJb9p XoCO11k64Ioqm_Dy8Kzj1c1iFD5MyBSRgwRxvvlILEgDDWNkPxuu2UBhBdDaMojoigfl8dBiDB9A iL_0z26VzIf22Ae0MAzYxCpPw9_83.8JIJ7Y.B9e.GUun0IKPCEKKJjCgNWqfYAl52X3Xgk.zpKX h68MM.KE52uL.isDdS3nn9e98WxCWdcIX3OcSqHB66Plf7yJ2WtxpFpG.HHvyCPL8AEF.3fP1uY6 4QOHyx5KqG0c_ll.g04ScwN.9r6XKSVAt1bHiy7R6nGvANvb_i7g4qv6UMuFqGrONmg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.ir2.yahoo.com with HTTP; Sat, 17 Oct 2020 18:49:40 +0000 Date: Sat, 17 Oct 2020 18:49:35 +0000 (UTC) From: Kostya Berger Reply-To: Kostya Berger To: FreeBSD Current Message-ID: <1896911060.833428.1602960575313@mail.yahoo.com> Subject: bhyve win 10 guest: destroy to restart MIME-Version: 1.0 References: <1896911060.833428.1602960575313.ref@mail.yahoo.com> X-Mailer: WebService/1.1.16868 YahooMailAndroidMobile YMobile/1.0 (com.yahoo.mobile.client.android.mail/6.13.2; Android/9; PKQ1.180904.001; whyred; Xiaomi; Redmi Note 5; 5.7; 2030x1080; ) X-Rspamd-Queue-Id: 4CDBs33XJ7z4VBQ X-Spamd-Bar: - X-Spamd-Result: default: False [-1.81 / 15.00]; HAS_REPLYTO(0.00)[bergerkos@yahoo.co.uk]; RWL_MAILSPIKE_GOOD(0.00)[77.238.177.82:from]; FREEMAIL_FROM(0.00)[yahoo.co.uk]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; URI_COUNT_ODD(1.00)[1]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.co.uk:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.co.uk,reject]; NEURAL_HAM_SHORT(-0.05)[-0.055]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[yahoo.co.uk]; ASN(0.00)[asn:34010, ipnet:77.238.176.0/22, country:GB]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.91)[-0.911]; R_DKIM_ALLOW(-0.20)[yahoo.co.uk:s=s2048]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.84)[-0.842]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FREEMAIL_REPLYTO(0.00)[yahoo.co.uk]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[77.238.177.82:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 18:49:45 -0000 Hello!It's been like that ever since I started using win 10 guest in bhyve:= If I need to restart win 10 guest I have to destroy the VM after guest shu= tdown. Because the bhyve startup script aborts, but the VM remains and won'= t be started again unless destroyed. I'm using bhyve binary with starting command line, nothing else. Is that a = normal behaviour? Or am I supposed to use vm-bhyve? If so, how can I conver= t the existing script to vm-bhyve config without Windows requiring new regi= stration because of new hardware?Hope these aren't too many questions.... =D0=9E=D1=82=D0=BF=D1=80=D0=B0=D0=B2=D0=BB=D0=B5=D0=BD=D0=BE =D0=B8=D0=B7 Y= ahoo =D0=9F=D0=BE=D1=87=D1=82=D1=8B =D0=B4=D0=BB=D1=8F Android From owner-freebsd-current@freebsd.org Sat Oct 17 18:59:02 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A439443F6B7 for ; Sat, 17 Oct 2020 18:59:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDC3n4WBZz4VrD for ; Sat, 17 Oct 2020 18:59:01 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qv1-xf42.google.com with SMTP id de3so2568009qvb.5 for ; Sat, 17 Oct 2020 11:59:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=m7B2TByXqwvSGY8PVg7cAyr7XtFq9EKavxXwdk8tQqo=; b=kYKOLEQ0OPw+kG195hGnsltrYvyTXnItiFnsweF/q0EcUSdiCaAhmHzJxe4OBsB+ih Y9sez+DA6ZLRjpS2hPv1U1E7rauEbwcVF12u79iQ8PuXKdhP/FMFNm3Qg2Fkys0aII3H G6PI0s3cKOfB9Gyc5kIc7fvHaihiVXerGgPZ6OGiqGOEmAaGf87i2jQME/cMF9ONDBmR aNoEAWfBMxNiOA/7chaiegBjgsp7wZQHIgqBgio7Kfnnj3mDV+yKQ3xB8xi8fH4fUl82 ACMfI0XXfN4TL9R3BK6/iopH07tj/HJouZwWcenkhQSf+d4dv03ZsC9kb6rNUN179gHu 4H+A== X-Gm-Message-State: AOAM532fesyIh18hOJuJltUogPJtlULXspsI/G4l7LzEkRtEcKxyjtw4 UUOn/1he5kzc1F41XCkW3QJqokuXLmgfyiplaAVPthBgogs= X-Google-Smtp-Source: ABdhPJzfd9AgA2/ZaDEZL8rOu25RZHJBSg/NV3hyaMEFzCu6VJayE0WprC3N1vtYgJaSYJNq0wIHlJalS/4RdQEQ7f8= X-Received: by 2002:a05:6214:10c4:: with SMTP id r4mr9583190qvs.62.1602961140431; Sat, 17 Oct 2020 11:59:00 -0700 (PDT) MIME-Version: 1.0 References: <20201015065632.GA72261@troutmask.apl.washington.edu> <3fdb18c1-5dc3-2e06-8825-db1daa425afb@yandex.ru> In-Reply-To: <3fdb18c1-5dc3-2e06-8825-db1daa425afb@yandex.ru> From: Warner Losh Date: Sat, 17 Oct 2020 12:58:48 -0600 Message-ID: Subject: Re: kernel panic and fun debugging To: "Andrey V. Elsukov" Cc: Steve Kargl , FreeBSD Current X-Rspamd-Queue-Id: 4CDC3n4WBZz4VrD X-Spamd-Bar: - X-Spamd-Result: default: False [-1.64 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-0.93)[-0.927]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.24)[0.237]; NEURAL_HAM_LONG(-0.95)[-0.951]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f42:from]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[yandex.ru]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-current]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 18:59:02 -0000 On Sat, Oct 17, 2020, 12:00 PM Andrey V. Elsukov wrote: > On 15.10.2020 09:56, Steve Kargl wrote: > > Just had a kernel panic. Best info I give you is > > > > % uname -a > > FreeBSD mobile 13.0-CURRENT FreeBSD 13.0-CURRENT #1 r366176M: Sat Sep 26 > 10:35:23 PDT 2020 kargl@mobile:/usr/obj/usr/src/i386.i386/sys/MOBILE > i386 > > > > % kgdb gdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 > > ... > > Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug... > > /usr/ports/devel/gdb/work-py37/gdb-9.2/gdb/inferior.c:283: > internal-error: struct inferior *find_inferior_pid(int): Assertion `pid != > 0' failed. > > A problem internal to GDB has been detected, > > further debugging may prove unreliable. > > Hi, > > do you have /var/crash/core.txt.1 file? > It may have some useful info. Also did you try an old version > /usr/libexec/kgdb ? > I got this same error, btw, when I pointed kgdb at the wrong kernel for the core file. But you can 'more' the uncompressed vmcore to get at least a traceback if the proper kernel is gone... Warner -- > WBR, Andrey V. Elsukov > > From owner-freebsd-current@freebsd.org Sat Oct 17 19:00:45 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2203943FA90 for ; Sat, 17 Oct 2020 19:00:45 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDC5l3kkDz4W6t for ; Sat, 17 Oct 2020 19:00:43 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.16.1/8.16.1) with ESMTPS id 09HJ0aqT083141 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 17 Oct 2020 12:00:36 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.16.1/8.16.1/Submit) id 09HJ0aCm083140; Sat, 17 Oct 2020 12:00:36 -0700 (PDT) (envelope-from sgk) Date: Sat, 17 Oct 2020 12:00:36 -0700 From: Steve Kargl To: "Andrey V. Elsukov" Cc: freebsd-current@freebsd.org Subject: Re: kernel panic and fun debugging Message-ID: <20201017190036.GA83033@troutmask.apl.washington.edu> References: <20201015065632.GA72261@troutmask.apl.washington.edu> <3fdb18c1-5dc3-2e06-8825-db1daa425afb@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3fdb18c1-5dc3-2e06-8825-db1daa425afb@yandex.ru> X-Rspamd-Queue-Id: 4CDC5l3kkDz4W6t X-Spamd-Bar: - X-Spamd-Result: default: False [-1.23 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.70)[-0.700]; NEURAL_HAM_LONG(-0.88)[-0.881]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.35)[0.351]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[yandex.ru]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:73, ipnet:128.95.0.0/16, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current]; DMARC_POLICY_SOFTFAIL(0.10)[washington.edu : No valid SPF, No valid DKIM, none] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 19:00:45 -0000 On Sat, Oct 17, 2020 at 08:57:31PM +0300, Andrey V. Elsukov wrote: > On 15.10.2020 09:56, Steve Kargl wrote: > > Just had a kernel panic. Best info I give you is > > > > % uname -a > > FreeBSD mobile 13.0-CURRENT FreeBSD 13.0-CURRENT #1 r366176M: Sat Sep 26 10:35:23 PDT 2020 kargl@mobile:/usr/obj/usr/src/i386.i386/sys/MOBILE i386 > > > > % kgdb gdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 > > ... > > Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug... > > /usr/ports/devel/gdb/work-py37/gdb-9.2/gdb/inferior.c:283: internal-error: struct inferior *find_inferior_pid(int): Assertion `pid != 0' failed. > > A problem internal to GDB has been detected, > > further debugging may prove unreliable. > > > do you have /var/crash/core.txt.1 file? > It may have some useful info. Also did you try an old version > /usr/libexec/kgdb ? > The only additional info in that file is mobile dumped core - see /var/crash/vmcore.1 Wed Oct 14 23:47:49 PDT 2020 FreeBSD mobile 13.0-CURRENT FreeBSD 13.0-CURRENT #1 r366176M: Sat Sep 26 10:35:23 PDT 2020 kargl@mobile:/usr/obj/usr/src/i386.i386/sys/MOBILE i386 panic: vm_fault_lookup: fault on nofault entry, addr: 0 It an older laptop, so I'm not ruling out memory showing its age. I'll also note that the laptop will panic once a week or so, when the swapper decides to swap out something drm. -- Steve From owner-freebsd-current@freebsd.org Sat Oct 17 19:09:37 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9EBF843FE83 for ; Sat, 17 Oct 2020 19:09:37 +0000 (UTC) (envelope-from mack@macktronics.com) Received: from mail.macktronics.com (coco.macktronics.com [209.181.253.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDCJ05757z4Wdr for ; Sat, 17 Oct 2020 19:09:36 +0000 (UTC) (envelope-from mack@macktronics.com) Received: from olive.macktronics.com (unknown [209.181.253.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.macktronics.com (Postfix) with ESMTPS id 79D5C18B6B for ; Sat, 17 Oct 2020 14:09:30 -0500 (CDT) Date: Sat, 17 Oct 2020 14:09:29 -0500 (CDT) From: Dan Mack X-X-Sender: mack@localhost.local To: freebsd-current@freebsd.org Subject: bug in motd service / documentation Message-ID: User-Agent: Alpine 2.20 (GSO 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-Rspamd-Queue-Id: 4CDCJ05757z4Wdr X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.48 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.95)[-0.948]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.181.253.64/29]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.96)[-0.959]; NEURAL_SPAM_SHORT(0.23)[0.231]; DMARC_POLICY_ALLOW(-0.50)[macktronics.com,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:209, ipnet:209.181.252.0/23, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 19:09:37 -0000 I've been away for a while and missed the change and complexity added to /etc/motd. I figured it out but I was led astray by not knowing that it was converted into a service that only runs at boot time. If you just follow the instructions in the template file, you will nothing changes. As a suggestion, maybe change this line in the /etc/motd.template file: Edit /etc/motd.template to change this login announcement. to: Edit /etc/motd.template to change this login announcement and reboot or manually restart the motd service in /etc/rc.d/motd as in: /etc/rc.d/motd restart and also add an entry to the man page for motd. This was on: FreeBSD boxolox 13.0-CURRENT FreeBSD 13.0-CURRENT #1 r366790: Sat Oct 17 08:57:56 CDT 2020 Dan From owner-freebsd-current@freebsd.org Sat Oct 17 19:17:58 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5924444023B for ; Sat, 17 Oct 2020 19:17:58 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDCTd4pmFz4Wwp for ; Sat, 17 Oct 2020 19:17:57 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.16.1/8.16.1) with ESMTPS id 09HJHtGg083221 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 17 Oct 2020 12:17:55 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.16.1/8.16.1/Submit) id 09HJHsxZ083220; Sat, 17 Oct 2020 12:17:54 -0700 (PDT) (envelope-from sgk) Date: Sat, 17 Oct 2020 12:17:54 -0700 From: Steve Kargl To: Warner Losh Cc: "Andrey V. Elsukov" , FreeBSD Current Subject: Re: kernel panic and fun debugging Message-ID: <20201017191754.GB83033@troutmask.apl.washington.edu> References: <20201015065632.GA72261@troutmask.apl.washington.edu> <3fdb18c1-5dc3-2e06-8825-db1daa425afb@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4CDCTd4pmFz4Wwp X-Spamd-Bar: - X-Spamd-Result: default: False [-1.85 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[washington.edu : No valid SPF, No valid DKIM,none]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-0.88)[-0.875]; NEURAL_HAM_LONG(-0.91)[-0.910]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.06)[-0.063]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:73, ipnet:128.95.0.0/16, country:US]; MAILMAN_DEST(0.00)[freebsd-current]; FREEMAIL_CC(0.00)[yandex.ru,freebsd.org] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 19:17:58 -0000 On Sat, Oct 17, 2020 at 12:58:48PM -0600, Warner Losh wrote: > On Sat, Oct 17, 2020, 12:00 PM Andrey V. Elsukov wrote: > > > On 15.10.2020 09:56, Steve Kargl wrote: > > > Just had a kernel panic. Best info I give you is > > > > > > % uname -a > > > FreeBSD mobile 13.0-CURRENT FreeBSD 13.0-CURRENT #1 r366176M: Sat Sep 26 > > 10:35:23 PDT 2020 kargl@mobile:/usr/obj/usr/src/i386.i386/sys/MOBILE > > i386 > > > > > > % kgdb gdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 > > > ... > > > Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug... > > > /usr/ports/devel/gdb/work-py37/gdb-9.2/gdb/inferior.c:283: > > internal-error: struct inferior *find_inferior_pid(int): Assertion `pid != > > 0' failed. > > > A problem internal to GDB has been detected, > > > further debugging may prove unreliable. > > > > Hi, > > > > do you have /var/crash/core.txt.1 file? > > It may have some useful info. Also did you try an old version > > /usr/libexec/kgdb ? > > > > I got this same error, btw, when I pointed kgdb at the wrong kernel for the > core file. > > But you can 'more' the uncompressed vmcore to get at least a traceback if > the proper kernel is gone... > > Warner > There are only 2 kernel.debug on the system. % locate kernel.debug /usr/lib/debug/boot/kernel/kernel.debug /usr/lib/debug/boot/kernel.old/kernel.debug Using % kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 % kgdb /usr/lib/debug/boot/kernel.old/kernel.debug vmcore.1 Both have the same result. I didn't realize the panic text was recorded int vmcore.1. If I extracted everything correctly, here's what happen WARNING !drm_modeset_is_locked(&crtc->mutex) failed at /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:621 #0 0x2318d4b9 at linux_dump_stack+0x19 #1 0x23219ec2 at drm_atomic_helper_check_modeset+0x92 #2 0x23091d90 at intel_atomic_check+0x70 #3 0x2321913e at drm_atomic_check_only+0x38e #4 0x23219461 at drm_atomic_commit+0x11 #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 #6 0x2322458e at drm_client_modeset_commit_force+0x5e #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 #8 0x2325b092 at vt_kms_postswitch+0x132 #9 0xa8bec5 at vt_fb_postswitch+0x15 #10 0xa9153d at vt_window_switch+0xfd #11 0xa8f59c at vtterm_cngrab+0x1c #12 0xbec5bf at termcn_cngrab+0xf #13 0xb4b6f6 at cngrab+0x16 #14 0xb9fce2 at vpanic+0xd2 #15 0xb9fc04 at panic+0x14 #16 0xe2c06a at vm_fault_lookup+0x13a #17 0xe2b8e7 at vm_fault+0x77 WARNING !drm_modeset_is_locked(&crtc->mutex) failed at /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:621 #0 0x2318d4b9 at linux_dump_stack+0x19 #1 0x23219ec2 at drm_atomic_helper_check_modeset+0x92 #2 0x23091d90 at intel_atomic_check+0x70 #3 0x2321913e at drm_atomic_check_only+0x38e #4 0x23219461 at drm_atomic_commit+0x11 #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 #6 0x2322458e at drm_client_modeset_commit_force+0x5e #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 #8 0x2325b092 at vt_kms_postswitch+0x132 #9 0xa8bec5 at vt_fb_postswitch+0x15 #10 0xa9153d at vt_window_switch+0xfd #11 0xa8f59c at vtterm_cngrab+0x1c #12 0xbec5bf at termcn_cngrab+0xf #13 0xb4b6f6 at cngrab+0x16 #14 0xb9fce2 at vpanic+0xd2 #15 0xb9fc04 at panic+0x14 #16 0xe2c06a at vm_fault_lookup+0x13a #17 0xe2b8e7 at vm_fault+0x77 WARNING !drm_modeset_is_locked(&dev->mode_config.connection_mutex) failed at /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:666 #0 0x2318d4b9 at linux_dump_stack+0x19 #1 0x2321a005 at drm_atomic_helper_check_modeset+0x1d5 #2 0x23091d90 at intel_atomic_check+0x70 #3 0x2321913e at drm_atomic_check_only+0x38e #4 0x23219461 at drm_atomic_commit+0x11 #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 #6 0x2322458e at drm_client_modeset_commit_force+0x5e #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 #8 0x2325b092 at vt_kms_postswitch+0x132 #9 0xa8bec5 at vt_fb_postswitch+0x15 #10 0xa9153d at vt_window_switch+0xfd #11 0xa8f59c at vtterm_cngrab+0x1c #12 0xbec5bf at termcn_cngrab+0xf #13 0xb4b6f6 at cngrab+0x16 #14 0xb9fce2 at vpanic+0xd2 #15 0xb9fc04 at panic+0x14 #16 0xe2c06a at vm_fault_lookup+0x13a #17 0xe2b8e7 at vm_fault+0x77 WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:871 #0 0x2318d4b9 at linux_dump_stack+0x19 #1 0x2321adbd at drm_atomic_helper_check_planes+0x8d #2 0x23092da7 at intel_atomic_check+0x1087 #3 0x2321913e at drm_atomic_check_only+0x38e #4 0x23219461 at drm_atomic_commit+0x11 #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 #6 0x2322458e at drm_client_modeset_commit_force+0x5e #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 #8 0x2325b092 at vt_kms_postswitch+0x132 #9 0xa8bec5 at vt_fb_postswitch+0x15 #10 0xa9153d at vt_window_switch+0xfd #11 0xa8f59c at vtterm_cngrab+0x1c #12 0xbec5bf at termcn_cngrab+0xf #13 0xb4b6f6 at cngrab+0x16 #14 0xb9fce2 at vpanic+0xd2 #15 0xb9fc04 at panic+0x14 #16 0xe2c06a at vm_fault_lookup+0x13a #17 0xe2b8e7 at vm_fault+0x77 WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:871 #0 0x2318d4b9 at linux_dump_stack+0x19 #1 0x2321adbd at drm_atomic_helper_check_planes+0x8d #2 0x23092da7 at intel_atomic_check+0x1087 #3 0x2321913e at drm_atomic_check_only+0x38e #4 0x23219461 at drm_atomic_commit+0x11 #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 #6 0x2322458e at drm_client_modeset_commit_force+0x5e #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 #8 0x2325b092 at vt_kms_postswitch+0x132 #9 0xa8bec5 at vt_fb_postswitch+0x15 #10 0xa9153d at vt_window_switch+0xfd #11 0xa8f59c at vtterm_cngrab+0x1c #12 0xbec5bf at termcn_cngrab+0xf #13 0xb4b6f6 at cngrab+0x16 #14 0xb9fce2 at vpanic+0xd2 #15 0xb9fc04 at panic+0x14 #16 0xe2c06a at vm_fault_lookup+0x13a #17 0xe2b8e7 at vm_fault+0x77 WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:871 #0 0x2318d4b9 at linux_dump_stack+0x19 #1 0x2321adbd at drm_atomic_helper_check_planes+0x8d #2 0x23092da7 at intel_atomic_check+0x1087 #3 0x2321913e at drm_atomic_check_only+0x38e #4 0x23219461 at drm_atomic_commit+0x11 #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 #6 0x2322458e at drm_client_modeset_commit_force+0x5e #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 #8 0x2325b092 at vt_kms_postswitch+0x132 #9 0xa8bec5 at vt_fb_postswitch+0x15 #10 0xa9153d at vt_window_switch+0xfd #11 0xa8f59c at vtterm_cngrab+0x1c #12 0xbec5bf at termcn_cngrab+0xf #13 0xb4b6f6 at cngrab+0x16 #14 0xb9fce2 at vpanic+0xd2 #15 0xb9fc04 at panic+0x14 #16 0xe2c06a at vm_fault_lookup+0x13a #17 0xe2b8e7 at vm_fault+0x77 WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:871 #0 0x2318d4b9 at linux_dump_stack+0x19 #1 0x2321adbd at drm_atomic_helper_check_planes+0x8d #2 0x23092da7 at intel_atomic_check+0x1087 #3 0x2321913e at drm_atomic_check_only+0x38e #4 0x23219461 at drm_atomic_commit+0x11 #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 #6 0x2322458e at drm_client_modeset_commit_force+0x5e #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 #8 0x2325b092 at vt_kms_postswitch+0x132 #9 0xa8bec5 at vt_fb_postswitch+0x15 #10 0xa9153d at vt_window_switch+0xfd #11 0xa8f59c at vtterm_cngrab+0x1c #12 0xbec5bf at termcn_cngrab+0xf #13 0xb4b6f6 at cngrab+0x16 #14 0xb9fce2 at vpanic+0xd2 #15 0xb9fc04 at panic+0x14 #16 0xe2c06a at vm_fault_lookup+0x13a #17 0xe2b8e7 at vm_fault+0x77 <4>WARN_ON(!mutex_is_locked(&dev->struct_mutex)) <4>WARN_ON(!mutex_is_locked(&fbc->lock))WARN_ON(!mutex_is_locked(&fbc->lock)) panic: vm_fault_lookup: fault on nofault entry, addr: 0 cpuid = 1 time = 1602744240 KDB: stack backtrace: db_trace_self_wrapper(1,22b9e8b4,9c8f69,fdabf8,12742000,...) at db_trace_self_wrapper+0x28/frame 0x22b9e898 kdb_backtrace(22c28700,1,121ad98,22b9e960,22b9e9a0,...) at kdb_backtrace+0x2b/frame 0x22b9e8f0 vpanic(f2d22a,22b9e92c,22b9e92c,22b9e950,e2c06a,...) at vpanic+0x130/frame 0x22b9e90c panic(f2d22a,f1fec9,0,22b9e974,87a2328,...) at panic+0x14/frame 0x22b9e920 vm_fault_lookup(ffffffff,0,0,0,701,...) at vm_fault_lookup+0x13a/frame 0x22b9e950 vm_fault(121ad98,0,1,0,0) at vm_fault+0x77/frame 0x22b9e9d0 vm_fault_trap(121ad98,0,1,0,0,0) at vm_fault_trap+0x44/frame 0x22b9e9f8 trap_pfault(0,0,0) at trap_pfault+0x119/frame 0x22b9ea40 trap(22b9eb08,8,28,28,40,...) at trap+0x2fb/frame 0x22b9eafc calltrap() at 0xffc0319f/frame 0x22b9eafc --- trap 0xc, eip = 0x231a72c4, esp = 0x22b9eb48, ebp = 0x22b9eb48 --- __linux_clflushopt(0,90000,0,3bea6000,2,...) at __linux_clflushopt+0x24/frame 0x22b9eb48 drm_clflush_sg(34cca3c0,34992dc0,22b9ebb0,22b9eb8c,ba82b8,...) at drm_clflush_sg+0x129/frame 0x22b9eb70 shmem_put_pages(34dab8c0,34cca3c0,0,0,22fe6038,...) at shmem_put_pages+0x3f/frame 0x22b9ebd0 __i915_gem_object_put_pages(34dab8c0,0,0,22c28700,22fe6038,...) at __i915_gem_object_put_pages+0x9d/frame 0x22b9ebf4 i915_gem_flush_free_objects(22fe6000,22b9ec5c,231981bd,22fe87bc,885aec0,...) at i915_gem_flush_free_objects+0x11e/frame 0x22b9ec20 __i915_gem_free_work(22fe87bc,885aec0,0,885aed8,0,...) at __i915_gem_free_work+0x11/frame 0x22b9ec2c linux_work_fn(22fe87bc,1) at linux_work_fn+0xed/frame 0x22b9ec5c taskqueue_run_locked(0,22c28700,22b9ecd4,b67a57,885aec0,...) at taskqueue_run_locked+0x1c4/frame 0x22b9eca8 taskqueue_thread_loop(885aec0,22b9ece8) at taskqueue_thread_loop+0x9e/frame 0x22b9ecb8 fork_exit(beb5d0,885aec0,22b9ece8,0,0,...) at fork_exit+0x67/frame 0x22b9ecd4 fork_trampoline() at 0xffc0340e/frame 0x22b9ecd4 --- kthread start KDB: enter: panic -- Steve From owner-freebsd-current@freebsd.org Sat Oct 17 19:23:32 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5872440449 for ; Sat, 17 Oct 2020 19:23:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDCc414GFz4XVg for ; Sat, 17 Oct 2020 19:23:31 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x841.google.com with SMTP id z33so3697494qth.8 for ; Sat, 17 Oct 2020 12:23:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=QXc7eLr6Lll8uw3VxZiX1zL6AxvcuMe1dbNOGQoJzRA=; b=ZJ68fJ8M3V8eK5VGEUpZH7V9YGNPCBHV7INuZkUfQdikcF0vxCv5lunVVuYSPWaq99 7PQpURnXuQ3Smk3+kAB/hYXqKPP33NxYxrm1++p6q+ZP870/R5uMSd/+3ZpwOGLZ5RNd pQoYXVDAV0JV1skAUjB9d8+OcR8gqY4cCwX/rQe2s63qJgOWh/eIBMZYJHbARngNWBnz H08VZaR2Jc5pFxlyi+1jMKF145U0uPvOmB661/YB8UgKtBR8eMYVxf8en/AoituSKmVF 2/KMsUKhXIq/PpViDWY3mx+LYiVlQUSaesO8y9M2gL56LDv33ewCOO8gTLhpHAFa3UBj AdDg== X-Gm-Message-State: AOAM531EshxUtr6DCzz+hONmrfDkFrpW4xQSuxweAMUrXmTO10H7j3HH SBpr3+rdRb5dLrgTplqWEbkxDsB7HHakyHw8PrjAz5cz+EzLAA== X-Google-Smtp-Source: ABdhPJyHeauY9qYw7N/U3+Si4UybPIjCGKYP5Dy+eJqwQWDdGToxqrAous7Eo5qieKiV8QCnz3eUT8VkKU82T43c0bI= X-Received: by 2002:a05:622a:10b:: with SMTP id u11mr8809209qtw.235.1602962610916; Sat, 17 Oct 2020 12:23:30 -0700 (PDT) MIME-Version: 1.0 References: <20201015065632.GA72261@troutmask.apl.washington.edu> <3fdb18c1-5dc3-2e06-8825-db1daa425afb@yandex.ru> <20201017191754.GB83033@troutmask.apl.washington.edu> In-Reply-To: <20201017191754.GB83033@troutmask.apl.washington.edu> From: Warner Losh Date: Sat, 17 Oct 2020 13:23:20 -0600 Message-ID: Subject: Re: kernel panic and fun debugging To: Steve Kargl Cc: "Andrey V. Elsukov" , FreeBSD Current X-Rspamd-Queue-Id: 4CDCc414GFz4XVg X-Spamd-Bar: - X-Spamd-Result: default: False [-1.97 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-0.94)[-0.937]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_CC(0.00)[yandex.ru,freebsd.org]; NEURAL_HAM_LONG(-0.94)[-0.941]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-0.09)[-0.091]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::841:from]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-current] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 19:23:33 -0000 On Sat, Oct 17, 2020 at 1:17 PM Steve Kargl < sgk@troutmask.apl.washington.edu> wrote: > On Sat, Oct 17, 2020 at 12:58:48PM -0600, Warner Losh wrote: > > On Sat, Oct 17, 2020, 12:00 PM Andrey V. Elsukov > wrote: > > > > > On 15.10.2020 09:56, Steve Kargl wrote: > > > > Just had a kernel panic. Best info I give you is > > > > > > > > % uname -a > > > > FreeBSD mobile 13.0-CURRENT FreeBSD 13.0-CURRENT #1 r366176M: Sat > Sep 26 > > > 10:35:23 PDT 2020 kargl@mobile > :/usr/obj/usr/src/i386.i386/sys/MOBILE > > > i386 > > > > > > > > % kgdb gdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 > > > > ... > > > > Reading symbols from /usr/lib/debug/boot/kernel/kernel.debug... > > > > /usr/ports/devel/gdb/work-py37/gdb-9.2/gdb/inferior.c:283: > > > internal-error: struct inferior *find_inferior_pid(int): Assertion > `pid != > > > 0' failed. > > > > A problem internal to GDB has been detected, > > > > further debugging may prove unreliable. > > > > > > Hi, > > > > > > do you have /var/crash/core.txt.1 file? > > > It may have some useful info. Also did you try an old version > > > /usr/libexec/kgdb ? > > > > > > > I got this same error, btw, when I pointed kgdb at the wrong kernel for > the > > core file. > > > > But you can 'more' the uncompressed vmcore to get at least a traceback if > > the proper kernel is gone... > > > > Warner > > > > There are only 2 kernel.debug on the system. > > % locate kernel.debug > /usr/lib/debug/boot/kernel/kernel.debug > /usr/lib/debug/boot/kernel.old/kernel.debug > > Using > > % kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 > % kgdb /usr/lib/debug/boot/kernel.old/kernel.debug vmcore.1 > > Both have the same result. In a pinch, you can use /boot/kernel/kernel if you suspect a difference... > I didn't realize the panic text > was recorded int vmcore.1. If I extracted everything correctly, > here's what happen > > WARNING !drm_modeset_is_locked(&crtc->mutex) failed at > /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:621 > #0 0x2318d4b9 at linux_dump_stack+0x19 > #1 0x23219ec2 at drm_atomic_helper_check_modeset+0x92 > #2 0x23091d90 at intel_atomic_check+0x70 > #3 0x2321913e at drm_atomic_check_only+0x38e > #4 0x23219461 at drm_atomic_commit+0x11 > #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 > #6 0x2322458e at drm_client_modeset_commit_force+0x5e > #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 > #8 0x2325b092 at vt_kms_postswitch+0x132 > #9 0xa8bec5 at vt_fb_postswitch+0x15 > #10 0xa9153d at vt_window_switch+0xfd > #11 0xa8f59c at vtterm_cngrab+0x1c > #12 0xbec5bf at termcn_cngrab+0xf > #13 0xb4b6f6 at cngrab+0x16 > #14 0xb9fce2 at vpanic+0xd2 > #15 0xb9fc04 at panic+0x14 > #16 0xe2c06a at vm_fault_lookup+0x13a > #17 0xe2b8e7 at vm_fault+0x77 > WARNING !drm_modeset_is_locked(&crtc->mutex) failed at > /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:621 > #0 0x2318d4b9 at linux_dump_stack+0x19 > #1 0x23219ec2 at drm_atomic_helper_check_modeset+0x92 > #2 0x23091d90 at intel_atomic_check+0x70 > #3 0x2321913e at drm_atomic_check_only+0x38e > #4 0x23219461 at drm_atomic_commit+0x11 > #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 > #6 0x2322458e at drm_client_modeset_commit_force+0x5e > #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 > #8 0x2325b092 at vt_kms_postswitch+0x132 > #9 0xa8bec5 at vt_fb_postswitch+0x15 > #10 0xa9153d at vt_window_switch+0xfd > #11 0xa8f59c at vtterm_cngrab+0x1c > #12 0xbec5bf at termcn_cngrab+0xf > #13 0xb4b6f6 at cngrab+0x16 > #14 0xb9fce2 at vpanic+0xd2 > #15 0xb9fc04 at panic+0x14 > #16 0xe2c06a at vm_fault_lookup+0x13a > #17 0xe2b8e7 at vm_fault+0x77 > WARNING !drm_modeset_is_locked(&dev->mode_config.connection_mutex) failed > at > /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:666 > #0 0x2318d4b9 at linux_dump_stack+0x19 > #1 0x2321a005 at drm_atomic_helper_check_modeset+0x1d5 > #2 0x23091d90 at intel_atomic_check+0x70 > #3 0x2321913e at drm_atomic_check_only+0x38e > #4 0x23219461 at drm_atomic_commit+0x11 > #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 > #6 0x2322458e at drm_client_modeset_commit_force+0x5e > #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 > #8 0x2325b092 at vt_kms_postswitch+0x132 > #9 0xa8bec5 at vt_fb_postswitch+0x15 > #10 0xa9153d at vt_window_switch+0xfd > #11 0xa8f59c at vtterm_cngrab+0x1c > #12 0xbec5bf at termcn_cngrab+0xf > #13 0xb4b6f6 at cngrab+0x16 > #14 0xb9fce2 at vpanic+0xd2 > #15 0xb9fc04 at panic+0x14 > #16 0xe2c06a at vm_fault_lookup+0x13a > #17 0xe2b8e7 at vm_fault+0x77 > WARNING !drm_modeset_is_locked(&plane->mutex) failed at > /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:871 > #0 0x2318d4b9 at linux_dump_stack+0x19 > #1 0x2321adbd at drm_atomic_helper_check_planes+0x8d > #2 0x23092da7 at intel_atomic_check+0x1087 > #3 0x2321913e at drm_atomic_check_only+0x38e > #4 0x23219461 at drm_atomic_commit+0x11 > #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 > #6 0x2322458e at drm_client_modeset_commit_force+0x5e > #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 > #8 0x2325b092 at vt_kms_postswitch+0x132 > #9 0xa8bec5 at vt_fb_postswitch+0x15 > #10 0xa9153d at vt_window_switch+0xfd > #11 0xa8f59c at vtterm_cngrab+0x1c > #12 0xbec5bf at termcn_cngrab+0xf > #13 0xb4b6f6 at cngrab+0x16 > #14 0xb9fce2 at vpanic+0xd2 > #15 0xb9fc04 at panic+0x14 > #16 0xe2c06a at vm_fault_lookup+0x13a > #17 0xe2b8e7 at vm_fault+0x77 > WARNING !drm_modeset_is_locked(&plane->mutex) failed at > /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:871 > #0 0x2318d4b9 at linux_dump_stack+0x19 > #1 0x2321adbd at drm_atomic_helper_check_planes+0x8d > #2 0x23092da7 at intel_atomic_check+0x1087 > #3 0x2321913e at drm_atomic_check_only+0x38e > #4 0x23219461 at drm_atomic_commit+0x11 > #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 > #6 0x2322458e at drm_client_modeset_commit_force+0x5e > #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 > #8 0x2325b092 at vt_kms_postswitch+0x132 > #9 0xa8bec5 at vt_fb_postswitch+0x15 > #10 0xa9153d at vt_window_switch+0xfd > #11 0xa8f59c at vtterm_cngrab+0x1c > #12 0xbec5bf at termcn_cngrab+0xf > #13 0xb4b6f6 at cngrab+0x16 > #14 0xb9fce2 at vpanic+0xd2 > #15 0xb9fc04 at panic+0x14 > #16 0xe2c06a at vm_fault_lookup+0x13a > #17 0xe2b8e7 at vm_fault+0x77 > WARNING !drm_modeset_is_locked(&plane->mutex) failed at > /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:871 > #0 0x2318d4b9 at linux_dump_stack+0x19 > #1 0x2321adbd at drm_atomic_helper_check_planes+0x8d > #2 0x23092da7 at intel_atomic_check+0x1087 > #3 0x2321913e at drm_atomic_check_only+0x38e > #4 0x23219461 at drm_atomic_commit+0x11 > #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 > #6 0x2322458e at drm_client_modeset_commit_force+0x5e > #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 > #8 0x2325b092 at vt_kms_postswitch+0x132 > #9 0xa8bec5 at vt_fb_postswitch+0x15 > #10 0xa9153d at vt_window_switch+0xfd > #11 0xa8f59c at vtterm_cngrab+0x1c > #12 0xbec5bf at termcn_cngrab+0xf > #13 0xb4b6f6 at cngrab+0x16 > #14 0xb9fce2 at vpanic+0xd2 > #15 0xb9fc04 at panic+0x14 > #16 0xe2c06a at vm_fault_lookup+0x13a > #17 0xe2b8e7 at vm_fault+0x77 > WARNING !drm_modeset_is_locked(&plane->mutex) failed at > /usr/ports/graphics/drm-current-kmod/work/drm-kmod-drm_v5.4.62_1/drivers/gpu/drm/drm_atomic_helper.c:871 > #0 0x2318d4b9 at linux_dump_stack+0x19 > #1 0x2321adbd at drm_atomic_helper_check_planes+0x8d > #2 0x23092da7 at intel_atomic_check+0x1087 > #3 0x2321913e at drm_atomic_check_only+0x38e > #4 0x23219461 at drm_atomic_commit+0x11 > #5 0x23224793 at drm_client_modeset_commit_atomic+0xb3 > #6 0x2322458e at drm_client_modeset_commit_force+0x5e > #7 0x23260851 at drm_fb_helper_restore_fbdev_mode_unlocked+0x71 > #8 0x2325b092 at vt_kms_postswitch+0x132 > #9 0xa8bec5 at vt_fb_postswitch+0x15 > #10 0xa9153d at vt_window_switch+0xfd > #11 0xa8f59c at vtterm_cngrab+0x1c > #12 0xbec5bf at termcn_cngrab+0xf > #13 0xb4b6f6 at cngrab+0x16 > #14 0xb9fce2 at vpanic+0xd2 > #15 0xb9fc04 at panic+0x14 > #16 0xe2c06a at vm_fault_lookup+0x13a > #17 0xe2b8e7 at vm_fault+0x77 > <4>WARN_ON(!mutex_is_locked(&dev->struct_mutex)) > > > <4>WARN_ON(!mutex_is_locked(&fbc->lock))WARN_ON(!mutex_is_locked(&fbc->lock)) > panic: vm_fault_lookup: fault on nofault entry, addr: 0 > cpuid = 1 > time = 1602744240 > KDB: stack backtrace: > db_trace_self_wrapper(1,22b9e8b4,9c8f69,fdabf8,12742000,...) at > db_trace_self_wrapper+0x28/frame 0x22b9e898 > kdb_backtrace(22c28700,1,121ad98,22b9e960,22b9e9a0,...) at > kdb_backtrace+0x2b/frame 0x22b9e8f0 > vpanic(f2d22a,22b9e92c,22b9e92c,22b9e950,e2c06a,...) at vpanic+0x130/frame > 0x22b9e90c > panic(f2d22a,f1fec9,0,22b9e974,87a2328,...) at panic+0x14/frame 0x22b9e920 > vm_fault_lookup(ffffffff,0,0,0,701,...) at vm_fault_lookup+0x13a/frame > 0x22b9e950 > vm_fault(121ad98,0,1,0,0) at vm_fault+0x77/frame 0x22b9e9d0 > vm_fault_trap(121ad98,0,1,0,0,0) at vm_fault_trap+0x44/frame 0x22b9e9f8 > trap_pfault(0,0,0) at trap_pfault+0x119/frame 0x22b9ea40 > trap(22b9eb08,8,28,28,40,...) at trap+0x2fb/frame 0x22b9eafc > calltrap() at 0xffc0319f/frame 0x22b9eafc > --- trap 0xc, eip = 0x231a72c4, esp = 0x22b9eb48, ebp = 0x22b9eb48 --- > __linux_clflushopt(0,90000,0,3bea6000,2,...) at > __linux_clflushopt+0x24/frame 0x22b9eb48 > drm_clflush_sg(34cca3c0,34992dc0,22b9ebb0,22b9eb8c,ba82b8,...) at > drm_clflush_sg+0x129/frame 0x22b9eb70 > shmem_put_pages(34dab8c0,34cca3c0,0,0,22fe6038,...) at > shmem_put_pages+0x3f/frame 0x22b9ebd0 > __i915_gem_object_put_pages(34dab8c0,0,0,22c28700,22fe6038,...) at > __i915_gem_object_put_pages+0x9d/frame 0x22b9ebf4 > i915_gem_flush_free_objects(22fe6000,22b9ec5c,231981bd,22fe87bc,885aec0,...) > at i915_gem_flush_free_objects+0x11e/frame 0x22b9ec20 > __i915_gem_free_work(22fe87bc,885aec0,0,885aed8,0,...) at > __i915_gem_free_work+0x11/frame 0x22b9ec2c > linux_work_fn(22fe87bc,1) at linux_work_fn+0xed/frame 0x22b9ec5c > taskqueue_run_locked(0,22c28700,22b9ecd4,b67a57,885aec0,...) at > taskqueue_run_locked+0x1c4/frame 0x22b9eca8 > taskqueue_thread_loop(885aec0,22b9ece8) at > taskqueue_thread_loop+0x9e/frame 0x22b9ecb8 > fork_exit(beb5d0,885aec0,22b9ece8,0,0,...) at fork_exit+0x67/frame > 0x22b9ecd4 > fork_trampoline() at 0xffc0340e/frame 0x22b9ecd4 > --- kthread start > KDB: enter: panic > I'm guessing that there's a locking issue with the 32-bit drm emulation code. This leads to unsynchronized access which leads to corrupt and this error. But that's a wild guess. At a guess, I'd say this is likely a bug in the 32-bit linux emulation, but that's just a crazy WAG. I'd expect it would take a fair amount to track this down... And it would be useful to know what the first of these warnings that eventually lead to this was. But my familiarity with the drm code and how it plumbs into the VM is quite low. Warner From owner-freebsd-current@freebsd.org Sat Oct 17 20:45:56 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BB2F3441D2E for ; Sat, 17 Oct 2020 20:45:56 +0000 (UTC) (envelope-from mack@macktronics.com) Received: from mail.macktronics.com (coco.macktronics.com [209.181.253.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDFR749gPz4cJW for ; Sat, 17 Oct 2020 20:45:55 +0000 (UTC) (envelope-from mack@macktronics.com) Received: from olive.macktronics.com (unknown [209.181.253.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.macktronics.com (Postfix) with ESMTPS id 4042918B81 for ; Sat, 17 Oct 2020 15:45:54 -0500 (CDT) Date: Sat, 17 Oct 2020 15:45:54 -0500 (CDT) From: Dan Mack X-X-Sender: mack@localhost.local To: freebsd-current@freebsd.org Subject: Re: bug in motd service / documentation In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (GSO 67 2015-01-07) MIME-Version: 1.0 Content-Type: multipart/mixed; BOUNDARY="3735943886-1594794192-1602967554=:725" X-Rspamd-Queue-Id: 4CDFR749gPz4cJW X-Spamd-Bar: / X-Spamd-Result: default: False [-0.93 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.93)[-0.927]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.181.253.64/29:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; MIME_BASE64_TEXT_BOGUS(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.96)[-0.958]; HAS_ATTACHMENT(0.00)[]; NEURAL_HAM_SHORT(-0.35)[-0.347]; MIME_BASE64_TEXT(0.10)[]; CTYPE_MIXED_BOGUS(1.00)[]; DMARC_POLICY_ALLOW(-0.50)[macktronics.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:209, ipnet:209.181.252.0/23, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Oct 2020 20:45:56 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3735943886-1594794192-1602967554=:725 Content-Type: text/plain; charset=US-ASCII; format=flowed On Sat, 17 Oct 2020, Dan Mack wrote: > I've been away for a while and missed the change and complexity added to > /etc/motd. I figured it out but I was led astray by not knowing that it was > converted into a service that only runs at boot time. If you just follow the > instructions in the template file, you will nothing changes. > > As a suggestion, maybe change this line in the /etc/motd.template file: > > Edit /etc/motd.template to change this login announcement. > > to: > > Edit /etc/motd.template to change this login announcement and > reboot or manually restart the motd service in /etc/rc.d/motd as in: > > /etc/rc.d/motd restart > > and also add an entry to the man page for motd. > > This was on: FreeBSD boxolox 13.0-CURRENT FreeBSD 13.0-CURRENT #1 > r366790: Sat Oct 17 08:57:56 CDT 2020 Instead of just complaining, I've attached a diff of a proposed fix. Diff against base/head is here fwiw: https://github.com/danmack/freebsd/tree/motd-doc-fix Dan --3735943886-1594794192-1602967554=:725 Content-Type: text/plain; charset=US-ASCII; name=motd-fix-txt Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: motd doc patch Content-Disposition: attachment; filename=motd-fix-txt ZGlmZiAtLWdpdCBhL3NoYXJlL21hbi9tYW41L21vdGQuNSBiL3NoYXJlL21h bi9tYW41L21vdGQuNQ0KaW5kZXggOWZlYjA0NzliOTIuLjQ1N2M2OWRhNmI2 IDEwMDY0NA0KLS0tIGEvc2hhcmUvbWFuL21hbjUvbW90ZC41DQorKysgYi9z aGFyZS9tYW4vbWFuNS9tb3RkLjUNCkBAIC0yMiw2ICsyMiwxNCBAQCBwcmVw ZW5kZWQgdG8NCiBhbmQgdGhlIGNvbnRlbnRzIGFyZSB3cml0dGVuIHRvDQog LlBhIC92YXIvcnVuL21vdGQgLg0KIC5QcA0KKy5QYSAvdmFyL3J1bi9tb3Rk IA0KK2NhbiBiZSB1cGRhdGVkIHdpdGhvdXQgYSBzeXN0ZW0gcmVib290IGJ5 IG1hbnVhbGx5IHJlc3RhcnRpbmcgdGhlDQorbW90ZCBzZXJ2aWNlIGFmdGVy IHVwZGF0aW5nDQorLlBhIC9ldGMvbW90ZC50ZW1wbGF0ZToNCisuQmQgLWxp dGVyYWwgLW9mZnNldCAtaWRlbnQNCitzZXJ2aWNlIG1vdGQgcmVzdGFydA0K Ky5FZA0KKy5QcA0KIEluZGl2aWR1YWwgdXNlcnMgbWF5IHN1cHByZXNzIHRo ZSBkaXNwbGF5IG9mIHRoaXMgZmlsZSBieQ0KIGNyZWF0aW5nIGEgZmlsZSBu YW1lZA0KIC5EcSBQYSAuaHVzaGxvZ2luDQpkaWZmIC0tZ2l0IGEvdXNyLmJp bi9sb2dpbi9tb3RkLnRlbXBsYXRlIGIvdXNyLmJpbi9sb2dpbi9tb3RkLnRl bXBsYXRlDQppbmRleCA4MGQ3OTA5NTk4MC4uOTk3Mzg1MGM2ODMgMTAwNjQ0 DQotLS0gYS91c3IuYmluL2xvZ2luL21vdGQudGVtcGxhdGUNCisrKyBiL3Vz ci5iaW4vbG9naW4vbW90ZC50ZW1wbGF0ZQ0KQEAgLTE3LDQgKzE3LDUgQEAg UGxlYXNlIGluY2x1ZGUgdGhhdCBvdXRwdXQgYW5kIGFueSBlcnJvciBtZXNz YWdlcyB3aGVuIHBvc3RpbmcgcXVlc3Rpb25zLg0KIEludHJvZHVjdGlvbiB0 byBtYW51YWwgcGFnZXM6ICBtYW4gbWFuDQogRnJlZUJTRCBkaXJlY3Rvcnkg bGF5b3V0OiAgICAgIG1hbiBoaWVyDQogDQotRWRpdCAvZXRjL21vdGQudGVt cGxhdGUgdG8gY2hhbmdlIHRoaXMgbG9naW4gYW5ub3VuY2VtZW50Lg0KK1Rv IGNoYW5nZSB0aGlzIGxvZ2luIGFubm91bmNlbWVudCwgZWRpdCAvZXRjL21v dGQudGVtcGxhdGUgYW5kIGVpdGhlcg0KK3JlYm9vdCBvciByZXN0YXJ0IHRo ZSBtb3RkIHNlcnZpY2UgaW4gL2V0Yy9yYy5kL21vdGQuDQo= --3735943886-1594794192-1602967554=:725--