From owner-freebsd-current@freebsd.org Mon Oct 5 21:39:14 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6BF45430446 for ; Mon, 5 Oct 2020 21:39:14 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C4vB927GGz4WD6; Mon, 5 Oct 2020 21:39:13 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-qk1-x72e.google.com with SMTP id f142so14096028qke.13; Mon, 05 Oct 2020 14:39:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=otpf2ehpGNPvgmRPrAek48PpLParVch96oBBA/yM1P8=; b=b0DXWNyynxNrFpKW3oBLgnjmI+cLxxmG9PyS1iq5Uqh9z96Xtu2h4BTVIkz/+ALr/q Z/dmWZLyobRneX1iZieWpGem3FXdUQpWTDHTvE9poCe/QnJmRTTj15+Wdmz8WjnBeEwz S9LGzQ/DHKgE1l4KR1rFBGiCIgnayVXya/8Fum18RaGpQOJ6GM/xshF/mGNfnwJOCsRc vKjYfvjDuTdGIW7P7YKFuCEz92j+Vj5ZmeG7+YMZLW+zXJnsEzDYDWpl2ZdeuukAJ/U9 SlRN8ekXbCBszui0oT1ENI6oXYjWcJPBpPwoUw7HWBijjsVPRerUusxlyvTCmwAJtxXl o7BQ== X-Gm-Message-State: AOAM530l/K9vA6AGnihZvefuT0yq/J6cqLbo4liwbdE7QFd00Rt6nTgS 3tFC/YYGjAVca419BEjfPS+4VkL0vBNb/Q== X-Google-Smtp-Source: ABdhPJwoiIZYsetGrW0RTkhSwYcGKmrgGGCcTY+h4y58MwfXQcqFpU2n4/r7IbQUrh92RLE2uywNnw== X-Received: by 2002:a37:a00b:: with SMTP id j11mr2130024qke.109.1601933952077; Mon, 05 Oct 2020 14:39:12 -0700 (PDT) Received: from spectre.mavhome.dp.ua (104-55-12-234.lightspeed.knvltn.sbcglobal.net. [104.55.12.234]) by smtp.gmail.com with ESMTPSA id n3sm717546qta.10.2020.10.05.14.39.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 05 Oct 2020 14:39:11 -0700 (PDT) Sender: Alexander Motin Subject: Re: GPF on boot with devmatch To: Warner Losh Cc: Xin LI , FreeBSD Current , Warner Losh References: <02fa309e-9467-f741-8092-974bfc145c9a@FreeBSD.org> From: Alexander Motin Autocrypt: addr=mav@FreeBSD.org; prefer-encrypt=mutual; keydata= mQENBFOzxAwBCADkPrax0pI2W/ig0CK9nRJJwsHitAGEZ2HZiFEuti+6/4UVxj81yr4ak/4g 9bKUyC7rMEAp/ZHNhd+MFCPAAcHPvtovnfykqE/vuosCS3wlSLloix2iKVLks0CwbLHGAyne 46lTQW74Xl/33c3W1Z6d8jD9gVFT/xaVzZ0U9xdzOmsYAZaAj4ki0tuxO9F7L+ct9grRe7iP g8t9hai7BL4ee3VRwk2JXnKb7UvBiVITKYWKz1jRvZIrjPokgEcCLOSlv7x/1kjuFnj3xWZU 7HSFFT8J93epBbrSSCsYsppIk2fZH41kaaFXsMQfTPH8wkeM6qwrvOh4HiQM08R+9tThABEB AAG0IUFsZXhhbmRlciBNb3RpbiA8bWF2QEZyZWVCU0Qub3JnPokBVwQTAQoAQQIbAwULCQgH AwUVCgkICwUWAwIBAAIeAQIXgAIZARYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMKuBQkN McyiAAoJEIMYw5VbqyJ/tuUIAOG3ONOSNYqjK4eTZ1TVh9jdUBAhWk5nhDFnODN49Wj0AbYm 7aIqy8O1hnCDSZG5LttjSAo3UfXJZDKQM0BLb0gpRMBnAYqO6tdolLNqAbPGJBnGoPjsh24y 6KcbDaNnis+lD4GwPXwQM+92wZGhCUFElPV9NciZGVS65TNIgk7X+yEjjhD1MSWKKijZ1r9Z zIt4OzUTxxNOvzdlABZS88nNRdJkatOQJPmFdd1mpP6UzTNCiLUo1pIqOEtJgvVVDYq5WHY6 tciWWYdmZG/tIBexJmv2mV2OLVjXR6ZeKmntVH14H72/wRHJuYHQC+r5SVRcWWayrThsY6jZ Yr4+raS5AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6Z AXgDtmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8Flv mI/c40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt 3ytU8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZ R1EdEIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm5 9R8AEQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczM AAoJEIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLq A6xe6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHu uC5vgPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15Gc sS9YcQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9 TevwGsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCg lz65AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6ZAXgD tmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8FlvmI/c 40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt3ytU 8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZR1Ed EIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm59R8A EQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczMAAoJ EIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLqA6xe 6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHuuC5v gPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15GcsS9Y cQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9Tevw Gsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCglz4= Message-ID: <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> Date: Mon, 5 Oct 2020 17:39:10 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4C4vB927GGz4WD6 X-Spamd-Bar: ++++ X-Spamd-Result: default: False [4.74 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(0.00)[+ip6:2607:f8b0:4000::/36:c]; URIBL_RED(3.50)[ixsystems.com:url]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; HAS_ANON_DOMAIN(0.10)[]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; ARC_NA(0.00)[]; R_DKIM_ALLOW(0.00)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.40)[0.404]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_SPAM_MEDIUM(0.35)[0.354]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.08)[0.078]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72e:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2020 21:39:14 -0000 On 05.10.2020 17:20, Warner Losh wrote: > On Mon, Oct 5, 2020 at 12:36 PM Alexander Motin > wrote: > > I can add that we've received report about identical panic on FreeBSD > releng/12.2 of r365436, AKA TrueNAS 12.0-RC1: > https://jira.ixsystems.com/browse/NAS-107578 .  So it looks a) pretty > rate (one report from thousands of early adopters and none in our lab), > and b) it is in stable/12 too, not only head. > > Thanks! I'll see if I can recreate here....  But we're accessing the > sysctl tree from devmatch to get some information, which should always > be OK (the fact that it isn't suggests either a bug in some driver > leaving bad pointers, or some race or both)...  It would be nice to know > which nodes they were, or to have a kernel panic I can look at... All we have now in this case is a screenshot you may see in the ticket. Also previously the same user on some earlier version of stable/12 reported other very weird panics on process lock being dropped where it can't be in some other sysctls inside kern.proc, so if we guess those are related, I suspect there may be some kind of memory corruption happening, but have no clue where. Unfortunately we have only textdumps for those. So if Xin is able to reproduce it locally, it may be our best chance to debug it, at least this specific issue. -- Alexander Motin