From nobody Mon Jun 30 14:03:28 2025 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bW7DK4Pb8z60VlX for ; Mon, 30 Jun 2025 14:03:29 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bW7DK3mrwz421q; Mon, 30 Jun 2025 14:03:29 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1751292209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SFAxdu3UCPvZjQeQunJsyGYhmTMxw9r295GqoeP3tME=; b=S9PNKtXSkx+ox1DF6DhPK99mEZMolbciK5aGL+8oUqJKKv1F88LyMy/mzjzrDBB5wHN3Fm oRJ5CzDg/1KNrvJiMBtI/E+U5kUbKFeApOdlMKoHdcPKSQ6ms5Mji2SPhJm5J+eRl3cI6/ Fun1yQyR0FTLi0IWPsWnVx8BrLAS7V17uJJX1loUSDY5D3jIbIrQvf43qNVOUqtBI8++2H pwXT8Hbb6dD+jXtDY3U3B7OLfOGUiL3IVo1no6TFYNygaJzwf0+OV0nYvqUaxoPUNP3i97 5rhNVw+ehHmt5FD+LheD9aZoEObFsKNpP2ppCkpUJmoup/bF6aIepZgLGqz8lw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1751292209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SFAxdu3UCPvZjQeQunJsyGYhmTMxw9r295GqoeP3tME=; b=lwZYCeBsT/eVDY6Oc/iDTTbW7x30clzYYRCfLcyLilBEWN1Dfptj6B2DolX5rmFts7AQ94 09iMLFE3Y3ebjKZMqgEAPBLccZ2velXB81+EGopyAEG+JWmOrwDGSB/2yf2LUMMRQlOuZJ p/MRTKZKwtneOaRx1p3CGLbwe0P+aQVjE43ibm7Pm54STwYNDQfw9YU6/ers2++Ar3yWU4 93gh6cAiBKLneoNqDsatlfqRZu9x6xcacMalXe8WepdnOeIVkOkvAQGf80D3D2da8SQf10 Zu1gbWoiFjh+bT3MtkYQaciSI92ltZ3qox4ImtB88PRmLUkLYcXgCCykjIPiRA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1751292209; a=rsa-sha256; cv=none; b=DOYGBPb1kAZVQ8ARPzDkbiI/j56p59aG6PB8hPY7gr1M8qXIHyWUTjuYnfhzX/HYOYylyK FhwopM/9mG1LKrwrxyfLTpOp6gsMn/t0/ss1YGYgj9gQUQGiBDXrwREmaFJY2L3CyxBUKX OPbAh01MiC9kXkHlrlvj2kDO+CJraV8TQPUkidpxnMWExkKCfYycHo/buBu3JUW+BChEDE hbvHESkEaXzLP5EcfYzZqF02a3ksRBXZQML1VUn4SAIkz7Xayw+31Xt+aMH+wlecvqOgHx 0tUOOa4WfHp7YK9sILIk8gpFXgb6NhHQ3DcnY5szx5eg8XhNxDqPqvbaG7t3Dw== Received: from [IPV6:2601:5c0:4200:b830:41cc:e0d8:c39:f14d] (unknown [IPv6:2601:5c0:4200:b830:41cc:e0d8:c39:f14d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4bW7DK1pqjzRsR; Mon, 30 Jun 2025 14:03:29 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <274774b0-9137-4fa7-a0a5-a0ce8976dc27@FreeBSD.org> Date: Mon, 30 Jun 2025 10:03:28 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: regression: memory issues on main/arm64 over sched/runq changes Content-Language: en-US To: Zhenlei Huang , "Bjoern A. Zeeb" Cc: FreeBSD Current , Olivier Certner References: <43005447-2rq0-6nn2-pnr5-4939s112npr4@yvfgf.mnoonqbm.arg> <0A01B9F5-C49C-41D8-BAB7-4378DEDBF647@FreeBSD.org> <28o26o81-so5r-qq79-6q6n-0q6746o7oo79@yvfgf.mnoonqbm.arg> <6A003013-415A-4594-AB04-AF5A9B2D660D@FreeBSD.org> <23n1773o-10o2-5p5o-25s4-r623rnn44649@yvfgf.mnoonqbm.arg> <907D042E-AE8A-4818-A807-AD45F36354FD@FreeBSD.org> From: John Baldwin In-Reply-To: <907D042E-AE8A-4818-A807-AD45F36354FD@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 6/28/25 11:35, Zhenlei Huang wrote: > I boot from disk. > > Updates on this locking issue, > > I think I finally figured out why. More stack trace from my video: > > ``` > shared lock of (sx) ifnet_sx @/usr/home/zlei/freebsd-src/sys/net/if.c:1467 > while exclusively locked from /usr/home/zlei/freebsd-src/sys/net/if.c:1416 > panic: excl->share > ... > witness_checkorder() at ... > _sx_slock_int() at _sx_slock_int+0x64/frame .... > if_addgroup() at ... > if_attach_internal() at ... > ether_ifattach() at ... > iflib_device_register() at ... > iflib_device_attach() at ... > device_attach() at ... > ... > root_bus_configure() at ... > configure() at ... > mi_startup() at ... > ``` > > The ifnet_sx has flag bit SX_RECURSE then it can be recursively locked. > > iflib_device_register() acquired ifnet_sx exclusively and then calls ethernet_ifattach() which will then calls if_addgroup(). It is prohibited to re-acquire the same lock shared so the witness blames. > > I think the witness should show the first file location of the exclusively lock, i.e. sys/net/iflib.c rather than the sys/net/if.c:1416 . So that it is more straight forward to figure out how that happens. CC John to see if that can be improved. Hmm, I think we have stopped at the first lle we found walking back up the lle list (find_instance() always works this way). You could add a 'find_last_instance' and use it in a few places perhaps. I guess both the share->excl and excl->share are places where you would maybe use it. Alternatively, you could have a 'find_next_instance' and maybe output all of them before the panic? -- John Baldwin