From owner-freebsd-net@freebsd.org Fri Jul 17 18:56:16 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A436636BF44 for ; Fri, 17 Jul 2020 18:56:16 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B7gM358S4z4NvD for ; Fri, 17 Jul 2020 18:56:15 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 06HIuAW5071081 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 17 Jul 2020 11:56:10 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 06HIu97t071080; Fri, 17 Jul 2020 11:56:09 -0700 (PDT) (envelope-from jmg) Date: Fri, 17 Jul 2020 11:56:09 -0700 From: John-Mark Gurney To: Marko Zec Cc: freebsd-net@FreeBSD.org Subject: Re: IF_DRV_PREPEND unlocked? Message-ID: <20200717185609.GX4213@funkthat.com> Mail-Followup-To: Marko Zec , freebsd-net@FreeBSD.org References: <20200715232624.GR4213@funkthat.com> <20200716072622.5fa35ba2@x23> <20200716074917.04445daa@x23> <20200716185629.GT4213@funkthat.com> <20200717120311.59377e0d@x23> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200717120311.59377e0d@x23> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Fri, 17 Jul 2020 11:56:10 -0700 (PDT) X-Rspamd-Queue-Id: 4B7gM358S4z4NvD X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [2.80 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.21)[0.214]; NEURAL_HAM_LONG(-0.03)[-0.026]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.41)[0.407]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jul 2020 18:56:16 -0000 Marko Zec wrote this message on Fri, Jul 17, 2020 at 12:03 +0200: > On Thu, 16 Jul 2020 11:56:29 -0700 > John-Mark Gurney wrote: > > > Marko Zec wrote this message on Thu, Jul 16, 2020 at 07:49 +0200: > > > On Thu, 16 Jul 2020 07:26:22 +0200 > > > Marko Zec wrote: > > > > > > > On Wed, 15 Jul 2020 16:26:25 -0700 > > > > John-Mark Gurney wrote: > > > > > > > > > I happen to be looking at the implementation of > > > > > IFQ_DRV_PREPEND, and unlike IFQ_DRV_DEQUEUE, it doesn't obtain > > > > > the lock when manipulating the ifq. I took a brief look at a > > > > > few drivers, and it looks like some of them expect that > > > > > _PREPEND lock the Q like _DEQUEUE does. > > > > > > > > > > This is likely not an issue often, since it's an error path that > > > > > likely rarely happens, but we should fix it. > > > > > > > > > > Should we just add the IFQ_LOCK/_UNLOCK to the macro? > > > > > > > > > > Comments or thoughts? > > > > > > > > I also have a hard time understanding the semantics of IFQ_DRV_* > > > > macros: per altq(9) they appear to be variants of their IFQ_ > > > > counterparts which are intended to be protected by some other > > > > mechanism rather than IFQ_LOCK, but in some instances this isn't > > > > the case, as they do grab the IFQ_LOCK. > > > > > > Hmm it's not that unclear after looking better: ifq_drv_* parts of > > > struct ifaltq should be protected by some external mechanisms when > > > using IFQ_DRV_ macros, and ifq_ parts are still protected by > > > ifq_mtx. So it doesn't look like IFQ_LOCK/_UNLOCK should be added to > > > IFQ_DRV_PREPEND()... > > > > Ahh, I was miss reading the macros on what was protected. Yes, it > > does look like the ifq_drv_* fields are supposed to be protected by > > another lock... > > We were both too fast to conclude that all IFQ_DRV_* are OK, but now > I'm not sure any more: > > #define IFQ_DRV_IS_EMPTY(ifq) \ > (((ifq)->ifq_drv_len == 0) && ((ifq)->ifq_len == 0)) > > So, if per altq(9) the contract is that with IFQ_DRV_* the ifq_drv_* > fields should be protected by some caller-provided mechanism, while the > other ifq_* fields will be implictly protected by ifq_mtx, how can > accessing ifw_len without holding ifq_mtx in the above example be safe? Reading is safe when you aren't modifying it, and only using it to inform if you should recheck w/ a lock... This way a driver can do: if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) { mtx_lock(sc->sc_mtx); for (;;) { IFQ_DRV_DEQUEUE(&ifp->if_snd, m); if (m == NULL) break; sendpkt(m); } mtx_unlock(sc->sc_mtx); } which saves an expensive lock/unlock op when there are no packets in the queue... > > I just now found that altq(9) documents this interface... I had > > assumed it as undocumented after man IFQ_DRV_DEQUEUE didn't return > > results.. > > > > Looks like I REALLY do need to complete my script to make sure that > > ALL Nm's and related pages have MLINKs. ALTQ.9 is missing 16 links > > to the functions it documents... > > > > Just as a reminder to people who write docs, please, make sure you > > link ALL of the function in your docs... Otherwise your docs won't > > be as easy to find.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."