From owner-freebsd-current@freebsd.org Mon Oct 12 15:25:57 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1A6D2439FF9 for ; Mon, 12 Oct 2020 15:25:57 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C92ZD0BFMz4LJ4; Mon, 12 Oct 2020 15:25:55 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-qt1-x843.google.com with SMTP id t9so13014637qtp.9; Mon, 12 Oct 2020 08:25:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:references:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=2WwGaLZer7NKZBGu5BOwaOejYMI97GTaJqbRPaCxj28=; b=lu4pQL8lt3jtN2IOYZ7uBbEhggi1RRtIl4b7PpM0OWjIuaASamXlVf/23DbBkH82fp LDCfDf2bARNXbEUqigpIgCD7atAaryRDEA5jkMP+dUso8B+Gc8A/TLYhRZog9j79XESr AxItrP7QGgPifXFoD9llAOb47WmnCxpdn6KOrJhunUsSvCqIe6uzM8vOvhlb5/P3yDY0 59Y7t5pVETQmQD5KtTNCzdLd4ZajPDdCH+TCrl/DJhzQk8qdL4AVUIvto0inAkeR7jPy CwHJr1im3g2h3aVDLKJ6b2jS47CLxBfOCMUSlNsrZlEbmpCPXySHPrjozMMWF8TAKsDd eh7Q== X-Gm-Message-State: AOAM532BxnoUVqTyZ8SgdsowNvVvq3NcacYaCRsSbyYi0va3srquilKQ ZPirnEhxatDTrhO1Ci6+xwfN5AZht+CvJw== X-Google-Smtp-Source: ABdhPJyYa6bC03SAPtG6O9vXEdiT7/4YvKhpf8r9gwshiq3mSKDeXdGOkF/dT6yA+g+Yy0VLNA2DNg== X-Received: by 2002:ac8:a87:: with SMTP id d7mr10812928qti.336.1602516354983; Mon, 12 Oct 2020 08:25:54 -0700 (PDT) Received: from spectre.mavhome.dp.ua (104-55-12-234.lightspeed.knvltn.sbcglobal.net. [104.55.12.234]) by smtp.gmail.com with ESMTPSA id f3sm12684325qtp.57.2020.10.12.08.25.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Oct 2020 08:25:53 -0700 (PDT) Sender: Alexander Motin Subject: Re: GPF on boot with devmatch From: Alexander Motin To: Warner Losh Cc: Xin LI , FreeBSD Current , Warner Losh References: <02fa309e-9467-f741-8092-974bfc145c9a@FreeBSD.org> <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> Autocrypt: addr=mav@FreeBSD.org; prefer-encrypt=mutual; keydata= mQENBFOzxAwBCADkPrax0pI2W/ig0CK9nRJJwsHitAGEZ2HZiFEuti+6/4UVxj81yr4ak/4g 9bKUyC7rMEAp/ZHNhd+MFCPAAcHPvtovnfykqE/vuosCS3wlSLloix2iKVLks0CwbLHGAyne 46lTQW74Xl/33c3W1Z6d8jD9gVFT/xaVzZ0U9xdzOmsYAZaAj4ki0tuxO9F7L+ct9grRe7iP g8t9hai7BL4ee3VRwk2JXnKb7UvBiVITKYWKz1jRvZIrjPokgEcCLOSlv7x/1kjuFnj3xWZU 7HSFFT8J93epBbrSSCsYsppIk2fZH41kaaFXsMQfTPH8wkeM6qwrvOh4HiQM08R+9tThABEB AAG0IUFsZXhhbmRlciBNb3RpbiA8bWF2QEZyZWVCU0Qub3JnPokBVwQTAQoAQQIbAwULCQgH AwUVCgkICwUWAwIBAAIeAQIXgAIZARYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMKuBQkN McyiAAoJEIMYw5VbqyJ/tuUIAOG3ONOSNYqjK4eTZ1TVh9jdUBAhWk5nhDFnODN49Wj0AbYm 7aIqy8O1hnCDSZG5LttjSAo3UfXJZDKQM0BLb0gpRMBnAYqO6tdolLNqAbPGJBnGoPjsh24y 6KcbDaNnis+lD4GwPXwQM+92wZGhCUFElPV9NciZGVS65TNIgk7X+yEjjhD1MSWKKijZ1r9Z zIt4OzUTxxNOvzdlABZS88nNRdJkatOQJPmFdd1mpP6UzTNCiLUo1pIqOEtJgvVVDYq5WHY6 tciWWYdmZG/tIBexJmv2mV2OLVjXR6ZeKmntVH14H72/wRHJuYHQC+r5SVRcWWayrThsY6jZ Yr4+raS5AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6Z AXgDtmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8Flv mI/c40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt 3ytU8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZ R1EdEIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm5 9R8AEQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczM AAoJEIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLq A6xe6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHu uC5vgPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15Gc sS9YcQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9 TevwGsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCg lz65AQ0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6ZAXgD tmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8FlvmI/c 40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt3ytU 8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZR1Ed EIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm59R8A EQEAAYkBPAQYAQoAJgIbDBYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMLYBQkNMczMAAoJ EIMYw5VbqyJ/TqgH/RQHClkvecE0262lwKoP/m0Mh4I5TLRgoJJn8S7G1BnqohYJkiLqA6xe 6urGD7OqdNAl12UbrjWbdJV+zvea3vJoM4MZuYiYrGaXWxzFXqWJcPwMU9sAh8MRghHuuC5v gPb45Tnftw9/+n0i8GfVhQhOqepUGdQg4NPcXviSkoAvig6pp9Lcxisn0groUQKt15GcsS9Y cQWg3j9Hnipc6Mu416HX98Fb113NHJqc2geTHLkRyuBFOoyIqB6N9GKjzOAIzxxsVdl9Tevw Gsrp4M4/RFzWbSgsbOnbE7454lmuVZGfReEjnUm8RHp9Q2UWKXlp3exlZjvOp/uVEpCglz4= Message-ID: <385234b0-0ca4-cec6-6a2e-6dec0a84b0f0@FreeBSD.org> Date: Mon, 12 Oct 2020 11:25:52 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.0 MIME-Version: 1.0 In-Reply-To: <5e4f0439-08fa-7715-7672-05793d05cc6d@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4C92ZD0BFMz4LJ4 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.68 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_HAM_LONG(-0.99)[-0.988]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.47)[-0.473]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::843:from]; NEURAL_HAM_MEDIUM(-1.02)[-1.019]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 15:25:57 -0000 On 05.10.2020 17:39, Alexander Motin wrote: > On 05.10.2020 17:20, Warner Losh wrote: >> On Mon, Oct 5, 2020 at 12:36 PM Alexander Motin > > wrote: >> >> I can add that we've received report about identical panic on FreeBSD >> releng/12.2 of r365436, AKA TrueNAS 12.0-RC1: >> https://jira.ixsystems.com/browse/NAS-107578 .  So it looks a) pretty >> rate (one report from thousands of early adopters and none in our lab), >> and b) it is in stable/12 too, not only head. >> >> Thanks! I'll see if I can recreate here....  But we're accessing the >> sysctl tree from devmatch to get some information, which should always >> be OK (the fact that it isn't suggests either a bug in some driver >> leaving bad pointers, or some race or both)...  It would be nice to know >> which nodes they were, or to have a kernel panic I can look at... > > All we have now in this case is a screenshot you may see in the ticket. > Also previously the same user on some earlier version of stable/12 > reported other very weird panics on process lock being dropped where it > can't be in some other sysctls inside kern.proc, so if we guess those > are related, I suspect there may be some kind of memory corruption > happening, but have no clue where. Unfortunately we have only textdumps > for those. So if Xin is able to reproduce it locally, it may be our > best chance to debug it, at least this specific issue. The TrueNAS user has confirmed that preloading the modules as Xin did workarounds the problem for him also. -- Alexander Motin