From owner-freebsd-current@freebsd.org Mon Sep 16 06:12:28 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 277CBF4693; Mon, 16 Sep 2019 06:12:28 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vtr.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46WwrW0DwXz4BQj; Mon, 16 Sep 2019 06:12:26 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp239-208.static.internode.on.net [59.167.239.208]) by vtr.rulingia.com (8.15.2/8.15.2) with ESMTPS id x8G6CBe3093769 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Sep 2019 16:12:17 +1000 (AEST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id x8G6C57l031607 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 16 Sep 2019 16:12:05 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id x8G6C54q031606; Mon, 16 Sep 2019 16:12:05 +1000 (AEST) (envelope-from peter) Date: Mon, 16 Sep 2019 16:12:05 +1000 From: Peter Jeremy To: freebsd-current@FreeBSD.org, freebsd-arm@FreeBSD.org Subject: "Sleeping with non-sleepable lock" in NFS on recent -current Message-ID: <20190916061205.GE97181@server.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="6sX45UoQRIJXqkqR" Content-Disposition: inline X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 46WwrW0DwXz4BQj X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of peter@rulingia.com designates 2001:19f0:5801:ebe:5400:1ff:fe53:30fd as permitted sender) smtp.mailfrom=peter@rulingia.com X-Spamd-Result: default: False [-5.59 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[rulingia.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.19)[ipnet: 2001:19f0:5800::/38(-4.94), asn: 20473(-0.93), country: US(-0.05)]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:20473, ipnet:2001:19f0:5800::/38, country:US]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Sep 2019 06:12:28 -0000 --6sX45UoQRIJXqkqR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I'm consistently seeing panics in the NFS code on recent -current on aarm64. The panics are one of the following two: Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NEWNFSnode lock (NEWNFSnode lock) r =3D 0 (0xfffffd00= 78b346f0) locked @ /usr/src/sys/fs/nfsclient/nfs_clport.c:432 Sleeping thread (tid 100077, pid 35) owns a non-sleepable lock Both panics have nearly identical backtraces (see below). I'm running diskless on a Rock64 with both filesystem and swap over NFS. The panics can be fairly reliably triggered by any of: * "make -j4 buildworld" * linking the kernel (as part of buildkernel) * "make installworld" Has anyone else seen this? The first panic (sleeping on vmopar) has a backtrace: sched_switch() at mi_switch+0x19c pc =3D 0xffff0000002ab368 lr =3D 0xffff00000028a9f4 sp =3D 0xffff000061192660 fp =3D 0xffff000061192680 mi_switch() at sleepq_switch+0x100 pc =3D 0xffff00000028a9f4 lr =3D 0xffff0000002d56dc sp =3D 0xffff000061192690 fp =3D 0xffff0000611926d0 sleepq_switch() at sleepq_wait+0x48 pc =3D 0xffff0000002d56dc lr =3D 0xffff0000002d5594 sp =3D 0xffff0000611926e0 fp =3D 0xffff000061192700 sleepq_wait() at _sleep+0x2c4 [***] pc =3D 0xffff0000002d5594 lr =3D 0xffff000000289eec sp =3D 0xffff000061192710 fp =3D 0xffff0000611927b0 _sleep() at vm_object_page_remove+0x178 [***] pc =3D 0xffff000000289eec lr =3D 0xffff00000052211c sp =3D 0xffff0000611927c0 fp =3D 0xffff000061192820 vm_object_page_remove() at vnode_pager_setsize+0xc0 pc =3D 0xffff00000052211c lr =3D 0xffff000000539a70 sp =3D 0xffff000061192830 fp =3D 0xffff000061192870 vnode_pager_setsize() at nfscl_loadattrcache+0x2e8 pc =3D 0xffff000000539a70 lr =3D 0xffff0000001ed4b4 sp =3D 0xffff000061192880 fp =3D 0xffff0000611928e0 nfscl_loadattrcache() at ncl_writerpc+0x104 pc =3D 0xffff0000001ed4b4 lr =3D 0xffff0000001e2158 sp =3D 0xffff0000611928f0 fp =3D 0xffff000061192a40 ncl_writerpc() at ncl_doio+0x36c pc =3D 0xffff0000001e2158 lr =3D 0xffff0000001f0370 sp =3D 0xffff000061192a50 fp =3D 0xffff000061192ae0 ncl_doio() at nfssvc_iod+0x228 pc =3D 0xffff0000001f0370 lr =3D 0xffff0000001f1d88 sp =3D 0xffff000061192af0 fp =3D 0xffff000061192b50 nfssvc_iod() at fork_exit+0x7c pc =3D 0xffff0000001f1d88 lr =3D 0xffff00000023ff5c sp =3D 0xffff000061192b60 fp =3D 0xffff000061192b90 fork_exit() at fork_trampoline+0x10 pc =3D 0xffff00000023ff5c lr =3D 0xffff000000562c34 sp =3D 0xffff000061192ba0 fp =3D 0x0000000000000000 For the second panic, the [***] change to: sleepq_wait() at vm_page_sleep_if_busy+0x80 vm_page_sleep_if_busy() at vm_object_page_remove+0xfc --=20 Peter Jeremy --6sX45UoQRIJXqkqR Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAl1/J61fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzR4Og/9HqQct7Cnm/MNzuBeWMo7KoA1kvVjD1o863CciBLSf1eUr61MPKwr2ODZ nkHuJwhMGr0wr8LqYmwjQHFlE73kBGlKsFLODQLw/6tXRS2ehkOLIII2StrBujaa Lv5zywwM9+SSYTW9lVf17qlumZjAurPBQ2gC/bJzGx+S9Vp6GWSDH6ibTvG9GMop 7EEQ2yN6fwZ37gCfvh7y8VoTgJa+Drbnrt7YTF5QNiSwk2l0gF9mPDYWp/rjR4nU DjJKiWaH9ZJg9CG5IDY13Ff/Rur6mHYcvSgzDpq5KmFVf9vEgRoS7s3r/JcdcnS8 +vukqy1XJFOzZAP2cPLstwd5QdRPWmOVT/jmVOUQplMtWQLtzEBFks88VmnVFS1n 1Em8qHfbukV6+RBR/dz4u/DiuVlsf7xtZ94BWh84z9rUu6c1KgNRG1b8B3VHiAs9 uiKUSRoCdzBY55GP/ZUinVdCWkbY/60iWIuMFV77IW81kLxmUketjjpst1TWelpN WppJlJSHW4UeTtM0Adbc64BhnhU4TvA01/H9EcmqcS4eSPQk49FSFjJL+RbKqsna XoSoO5Jw+KH9H6MG/sVEi69mJSAj1LsuHvuXrhJdmUOHXJ5gMyz2RWs5ixIGv13J oVdnBHnKkQBWwUGVYP5jLQldy9AtmntgJhJZh7I8lS0h+YvnNo8= =w1bx -----END PGP SIGNATURE----- --6sX45UoQRIJXqkqR--