Date: Wed, 1 Sep 2021 15:10:53 +0200 From: Alexander Lochmann <alexander.lochmann@tu-dortmund.de> To: Konstantin Belousov <kostikbel@gmail.com> Cc: freebsd-fs <freebsd-fs@freebsd.org>, Horst Schirmeier <horst@schirmeier.de> Subject: Re: Various unprotected accesses to buf and vnode Message-ID: <c4f7e26f-aa5a-aacb-51f7-5edbe5ebad47@tu-dortmund.de> In-Reply-To: <YS5AWi7k2ie2MB4u@kib.kiev.ua> References: <55f3661e-2173-793e-4834-bbcd79d3d99e@tu-dortmund.de> <YSkxgXyXZfNvrXA/@kib.kiev.ua> <380bdcc8-bede-2a64-8e5e-031552231d82@tu-dortmund.de> <YSqhe3WI8dVvUq7g@kib.kiev.ua> <46649402-d28a-6f81-f0a8-39180b681f4c@tu-dortmund.de> <YSq42Cb48SMv%2BsIO@kib.kiev.ua> <e50f4583-5150-a162-e188-7207e5e7eb61@tu-dortmund.de> <YS5AWi7k2ie2MB4u@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --04TxWz7gO8SglOxkkiEWFPFYUunGc16dc Content-Type: multipart/mixed; boundary="WfgMYjAh9e85p3j6EnDfzFPMssO4F8VSz"; protected-headers="v1" From: Alexander Lochmann <alexander.lochmann@tu-dortmund.de> To: Konstantin Belousov <kostikbel@gmail.com> Cc: freebsd-fs <freebsd-fs@freebsd.org>, Horst Schirmeier <horst@schirmeier.de> Message-ID: <c4f7e26f-aa5a-aacb-51f7-5edbe5ebad47@tu-dortmund.de> Subject: Re: Various unprotected accesses to buf and vnode References: <55f3661e-2173-793e-4834-bbcd79d3d99e@tu-dortmund.de> <YSkxgXyXZfNvrXA/@kib.kiev.ua> <380bdcc8-bede-2a64-8e5e-031552231d82@tu-dortmund.de> <YSqhe3WI8dVvUq7g@kib.kiev.ua> <46649402-d28a-6f81-f0a8-39180b681f4c@tu-dortmund.de> <YSq42Cb48SMv+sIO@kib.kiev.ua> <e50f4583-5150-a162-e188-7207e5e7eb61@tu-dortmund.de> <YS5AWi7k2ie2MB4u@kib.kiev.ua> In-Reply-To: <YS5AWi7k2ie2MB4u@kib.kiev.ua> --WfgMYjAh9e85p3j6EnDfzFPMssO4F8VSz Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 31.08.21 16:44, Konstantin Belousov wrote: >> So in all of those call sequences the buffer lock is not acquired. >> However, I'd not rule out that our tooling could be broken as well. > Buffer is locked inside UFS_BALLOC(), which returns it to the ffs_write= () > use. I took a deep dive into our data, and had a closer look at two samples. In both cases the b_lock is not acquired. Since the debug information seems to be damaged, I had to use 'objdump -S' to translate the pc of the unguarded memory access to a source code position. It seems to be vp->v_lasta =3D bp->b_blkno; in https://thasos.cs.tu-dortmund.de/freebsd-lockdoc/lockdoc-v13.0-0.6/source= /sys/kern/vfs_cluster.c#L802. It was release in binsfree() and bq_insert(): https://thasos.cs.tu-dortmund.de/freebsd-lockdoc/latest/source/sys/kern/v= fs_bio.c#L1537 https://thasos.cs.tu-dortmund.de/freebsd-lockdoc/latest/source/sys/kern/v= fs_bio.c#L1977 Right before the entry that records the unlock, there was a memory access recorded including the stracketrace. I assume that memory access belongs to the unlock operation, and translated the stacktrace. For binsfree(): /opt/kernel/freebsd/src/sys/sys/lockdoc.h:104 /opt/kernel/freebsd/src/sys/kern/kern_lock.c:247 (inlined by) /opt/kernel/freebsd/src/sys/kern/kern_lock.c:1408 /opt/kernel/freebsd/src/sys/sys/lockmgr.h:107 (inlined by) /opt/kernel/freebsd/src/sys/kern/vfs_bio.c:1537 /opt/kernel/freebsd/src/sys/kern/vfs_bio.c:2437 /opt/kernel/freebsd/src/sys/kern/vfs_cluster.c:775 /opt/kernel/freebsd/src/sys/ufs/ffs/ffs_vnops.c:894 /opt/kernel/freebsd/obj/lockdoc-kernproc/vnode_if.c:1108 /opt/kernel/freebsd/obj/lockdoc-kernproc/./vnode_if.h:569 (inlined by) /opt/kernel/freebsd/src/sys/kern/vfs_vnops.c:1093 /opt/kernel/freebsd/src/sys/kern/vfs_vnops.c:1158 /opt/kernel/freebsd/src/sys/kern/vfs_vnops.c:1276 /opt/kernel/freebsd/src/sys/kern/vfs_vnops.c:1398 /opt/kernel/freebsd/src/sys/sys/file.h:327 (inlined by) /opt/kernel/freebsd/src/sys/kern/sys_generic.c:564 /opt/kernel/freebsd/src/sys/kern/sys_generic.c:491 /opt/kernel/freebsd/src/sys/i386/i386/../../kern/subr_syscall.c:189 For bq_insert(): /opt/kernel/freebsd/src/sys/sys/lockdoc.h:104 /opt/kernel/freebsd/src/sys/kern/kern_lock.c:247 (inlined by) /opt/kernel/freebsd/src/sys/kern/kern_lock.c:1408 /opt/kernel/freebsd/src/sys/sys/lockmgr.h:107 (inlined by) /opt/kernel/freebsd/src/sys/kern/vfs_bio.c:1977 /opt/kernel/freebsd/src/sys/kern/vfs_bio.c:1552 /opt/kernel/freebsd/src/sys/kern/vfs_bio.c:2437 /opt/kernel/freebsd/src/sys/kern/vfs_cluster.c:775 /opt/kernel/freebsd/src/sys/ufs/ffs/ffs_vnops.c:894 /opt/kernel/freebsd/obj/lockdoc-kernproc/vnode_if.c:1108 /opt/kernel/freebsd/obj/lockdoc-kernproc/./vnode_if.h:569 (inlined by) /opt/kernel/freebsd/src/sys/kern/vfs_vnops.c:1093 /opt/kernel/freebsd/src/sys/kern/vfs_vnops.c:1158 /opt/kernel/freebsd/src/sys/kern/vfs_vnops.c:1276 /opt/kernel/freebsd/src/sys/kern/vfs_vnops.c:1398 /opt/kernel/freebsd/src/sys/sys/file.h:327 (inlined by) /opt/kernel/freebsd/src/sys/kern/sys_generic.c:564 /opt/kernel/freebsd/src/sys/kern/sys_generic.c:491 /opt/kernel/freebsd/src/sys/i386/i386/../../kern/subr_syscall.c:189 > Read e.g. sys/ufs/ufs/inode.h gerald comment above struct inode definit= ion. > It provides more detailed exposure. Aaah. Thx. This is about the struct inode. So I assume it also applies for a vnode belonging to an inode. Am I right?> Vnode lock is a lock obtained with vn_lock(). It is up to filesystem > to implement VOP_LOCK() which locks the vnode. >=20 > Default VOP_LOCK() locks vp->v_vnlock, which again by default points > to &vp->v_lock. >=20 > There are several special cases. For instance, for FFS and snapshot vn= odes, > v_vnlock points to the snapdata->sn_lock (snapdata is unique per FFS mo= unt). > For nullfs non-reclaimed vnodes, v_vnlock points to the lower vnode loc= k. >=20 Thx! Is this written down somewhere? --=20 Technische Universit=C3=A4t Dortmund Alexander Lochmann PGP key: 0xBC3EF6FD Otto-Hahn-Str. 16 phone: +49.231.7556141 D-44227 Dortmund fax: +49.231.7556116 http://ess.cs.tu-dortmund.de/Staff/al --WfgMYjAh9e85p3j6EnDfzFPMssO4F8VSz-- --04TxWz7gO8SglOxkkiEWFPFYUunGc16dc Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEElhZsUHzVP0dbkjCRWT7tBbw+9v0FAmEve90FAwAAAAAACgkQWT7tBbw+9v0+ Kw//eUQMK0XnS1gE8LdLDDImOqyN+/Avb+UnWNtia+npz8t3+kVNex25pTb6/yFJyHCbZTLelZYs NAA+uGnuYyAK1D67v5R8KSrKjof39DRbXErD1OfkTnyTm9lrJyqx7mCult+9XfDivAPBEOfN+qBT gACsguJ5jDmaqS4Wb6W8xk2ga/1LbrfNdYmavWHK7Pc2oQdaUapN927t5gREqlasfHN5Cj7Rp2i9 pmHgJmrHjf+X14uyiUGYsoMBBHeRgVf4Ow523PTHv3hO0ZFxbuam9AZxoM+48dowWRoVvAsbIGX0 ebKdAMZ84aPATOmdypK5D7GJajuMwM718Bg4oBYjuH/jrriBS4fRIn6hq9pDFsr4J5dQaErAi1CH YTxQ+PlRFok6NUMOQs0bXFNK2cTD5UR2u/Nrt/DVmZn2rWiFviT+BD8WV6FV7eYvzqii5ReKiT+7 0Wa4y1TWfYEM1qDWiA7B5NOlNNp/Bxn8NpTL9r2KrQrR/uoXADSN8lTLP0JcSGsN0GjRgVJ0AN6P D3MNsgqyQ3MNyb3QCbE6npZjR0SquGoqvITZvSewpzpycb7Z/0U3UBTNxiOFSgdyVmK0zEMN6CAx ZVqqx4mZ3wA5lw84AkN2TeQyFoFvzWCGSxI2nC9Ld594EU3O5hJxtvOIaa2NYOY1KJ1wKxrYT7zm b9Y= =z0tu -----END PGP SIGNATURE----- --04TxWz7gO8SglOxkkiEWFPFYUunGc16dc--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c4f7e26f-aa5a-aacb-51f7-5edbe5ebad47>