From owner-freebsd-fs@freebsd.org Sat Mar 13 03:30:41 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 582345B7BAF for ; Sat, 13 Mar 2021 03:30:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dy7Vm2ZV3z3pZs for ; Sat, 13 Mar 2021 03:30:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 12D3UQ1v001367 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 13 Mar 2021 05:30:29 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 12D3UQ1v001367 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 12D3UPTm001361; Sat, 13 Mar 2021 05:30:25 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 13 Mar 2021 05:30:25 +0200 From: Konstantin Belousov To: Alexander Lochmann Cc: freebsd-fs@freebsd.org Subject: Re: [RFC] Understanding the locking of struct buf Message-ID: References: <49130618-349a-bfc7-6d26-0c3763904dc5@tu-dortmund.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <49130618-349a-bfc7-6d26-0c3763904dc5@tu-dortmund.de> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4Dy7Vm2ZV3z3pZs X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-1.97 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; NEURAL_HAM_MEDIUM(-0.16)[-0.159]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-0.90)[-0.898]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; R_SPF_SOFTFAIL(0.00)[~all]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.92)[-0.917]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-fs]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2021 03:30:41 -0000 On Fri, Mar 12, 2021 at 07:57:54PM +0100, Alexander Lochmann wrote: > Hi folks! > > According to the definition [1], any access to a struct buf is protected > by a lock: > 1) Are there exceptions to those locking rules for reading and/or > writing a member? > E.g. any read is permitted without a lock being held. > Can b_bcount, for example, be read without a lock? Sure you can read it without lock. Question is, what do you intent to do with this information. Buffers used to track io requests for vnodes, that is, buffers appearing on bufobj dirty/clean queues, are type-stable, so you are guaranteed that the memory is not reused for something else. On the other hand, only buffer lock guarantees that the buffer identity, i.e. (vnode, range) for data described by the buffer, is valid. > > 2) In vfs_bio.c, for example, the BUF_KERNPROC macro is used to move > ownership of a buf.b_lock to the kernel. > In _lockmgr_disown() [2] WITNESS_UNLOCK() is called to emulate an > unlock. So, shouldn't be there a WITNESS_LOCK() for that lock? > Otherwise, the Witness system might complain about the upcoming unlock. Are you reporting a bug or just asking about LK_KERNPROC. Lockmgr (kern_lock.c)is careful enough to handle LK_KERNPROC owner specially. In particular, it does not report unlock of such lock to witness. > > 3) Is the function brelse()/bqrelse() considered to be a destruction > function? As I said above, buffers are type-stable, they are not destructed. brelse/bqrelse relinguish ownership of the buffer by the calling thread. After that, buffer typically goes to corresponding queue of unlocked clean or dirty buffer, from where it could be picked up or reclaimed to represent a different buffer. > Hence, no lock (buf.b_lock) is needed within this function to access a > struct buf. The buffer lock is required for call to brelse (and bqrelse). As noted, buffer lock represents an ownership of the buffer. It is typically obtained when the thread calls bread(9), which returns valid and locked buffer. Then caller do whatever he needed, then it is either brelse() if the buffer is clean, or some variant of bwrite() if buffer become dirty. Buffer lock is taken somewhere in bread() and released in brelse/bwrite, from the PoV of the consumer of the buffer cache. Actual interactions are usually more complex, due to a lot of features grown in the buffer cache for 40+ years, but the main principle remained the same, thread that owns the buffer owns its lock.