From owner-svn-src-all@freebsd.org Wed Nov 18 04:45:12 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 624E42E18F7; Wed, 18 Nov 2020 04:45:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CbVbp0f4yz4fMn; Wed, 18 Nov 2020 04:45:09 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 0AI4iwYf037400 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 18 Nov 2020 06:45:01 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 0AI4iwYf037400 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 0AI4iwVf037399; Wed, 18 Nov 2020 06:44:58 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 18 Nov 2020 06:44:58 +0200 From: Konstantin Belousov To: Mateusz Guzik Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r367713 - head/sys/kern Message-ID: References: <202011160309.0AG39JmP067464@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4CbVbp0f4yz4fMn X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2020 04:45:12 -0000 On Tue, Nov 17, 2020 at 03:36:31PM +0100, Mateusz Guzik wrote: > On 11/17/20, Konstantin Belousov wrote: > > On Tue, Nov 17, 2020 at 04:15:12AM +0100, Mateusz Guzik wrote: > >> On 11/17/20, Konstantin Belousov wrote: > >> > On Mon, Nov 16, 2020 at 03:09:19AM +0000, Mateusz Guzik wrote: > >> >> Author: mjg > >> >> Date: Mon Nov 16 03:09:18 2020 > >> >> New Revision: 367713 > >> >> URL: https://svnweb.freebsd.org/changeset/base/367713 > >> >> > >> >> Log: > >> >> select: replace reference counting with memory barriers in selfd > >> >> > >> >> Refcounting was added to combat a race between selfdfree and > >> >> doselwakup, > >> >> but it adds avoidable overhead. > >> >> > >> >> selfdfree detects it can free the object by ->sf_si == NULL, thus we > >> >> can > >> >> ensure that the condition only holds after all accesses are > >> >> completed. > >> >> > >> >> Modified: > >> >> head/sys/kern/sys_generic.c > >> >> > >> >> Modified: head/sys/kern/sys_generic.c > >> >> ============================================================================== > >> >> --- head/sys/kern/sys_generic.c Sun Nov 15 22:49:28 2020 (r367712) > >> >> +++ head/sys/kern/sys_generic.c Mon Nov 16 03:09:18 2020 (r367713) > >> >> @@ -156,7 +156,6 @@ struct selfd { > >> >> struct mtx *sf_mtx; /* Pointer to selinfo mtx. */ > >> >> struct seltd *sf_td; /* (k) owning seltd. */ > >> >> void *sf_cookie; /* (k) fd or pollfd. */ > >> >> - u_int sf_refs; > >> >> }; > >> >> > >> >> MALLOC_DEFINE(M_SELFD, "selfd", "selfd"); > >> >> @@ -1704,16 +1703,17 @@ static void > >> >> selfdfree(struct seltd *stp, struct selfd *sfp) > >> >> { > >> >> STAILQ_REMOVE(&stp->st_selq, sfp, selfd, sf_link); > >> >> - if (sfp->sf_si != NULL) { > >> >> + /* > >> >> + * Paired with doselwakeup. > >> >> + */ > >> >> + if (atomic_load_acq_ptr((uintptr_t *)&sfp->sf_si) != (uintptr_t)NULL) > >> >> { > >> > This could be != 0. > >> > > >> >> mtx_lock(sfp->sf_mtx); > >> >> if (sfp->sf_si != NULL) { > >> >> TAILQ_REMOVE(&sfp->sf_si->si_tdlist, sfp, sf_threads); > >> >> - refcount_release(&sfp->sf_refs); > >> >> } > >> >> mtx_unlock(sfp->sf_mtx); > >> >> } > >> >> - if (refcount_release(&sfp->sf_refs)) > >> >> - free(sfp, M_SELFD); > >> >> + free(sfp, M_SELFD); > >> > What guarantees that doselwakeup() finished with sfp ? > >> > > >> > >> Release semantics provided by atomic_store_rel_ptr -- it means the > >> NULL store is the last access, neither CPU nor the compiler are going > >> to reorder preceding loads/stores past it. > > It only guarantees that if we observed NULL as the result of load. > > > > If that did not happen selfdfree takes the lock. If the entry is still > there, it removes it and doselwakeup will not see it after it takes > the lock. If the entry is not there, doselwaekup is done with it > because it released the lock. I still do not understand it. selfdfree() indeed takes sf_mtx and rechecks sf_si. But what prevents doselwakeup() to store NULL into sf_si at any moment. e.g. after free() is done ? selfdfree() takes seltd mutex, not selinfo.