From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 13 14:35:35 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CA061065687 for ; Tue, 13 Dec 2011 14:35:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3369E8FC0A for ; Tue, 13 Dec 2011 14:35:35 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id DE92D46B0D; Tue, 13 Dec 2011 09:35:34 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 764AFB95B; Tue, 13 Dec 2011 09:35:34 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Tue, 13 Dec 2011 09:35:33 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112130935.33975.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 13 Dec 2011 09:35:34 -0500 (EST) Cc: Monthadar Al Jaberi Subject: Re: loop inside uma_zfree critical section X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 14:35:35 -0000 On Tuesday, December 13, 2011 7:46:34 am Monthadar Al Jaberi wrote: > Hi, > > I am not sure why I am having this problem, but looking > at the code I dont understand uma_core.c really good. > So I hope someone can shed a light on this: > > I am running on an arm board and and running a kernel module > that behaves like a wlan interface. so I tx and rx packets. > > For now tx is only only sending beacon like frames. > This is done through using ieee80211_beacon_alloc(). > > Then in a callout task to generate periodic beacons: > > m_dup(avp->beacon, M_DONTWAIT); > mtx_lock(...); > STAILQ_INSERT_TAIL(...); > taskqueue_enqueue(...); > mtx_unlock(...); > callout_schedule(...); > > On the RX side, the interrupt handler will read out buffer > then place it on a queue to be handled by wlan-glue code. > For now wlan-glue code just frees the mbuf it instead of > calling net80211 ieee80211_input() functions: > > m_copyback(...); > /* Allocate new mbuf for next RX. */ > MGETHDR(..., M_DONTWAIT, MT_DATA); > bzero((mtod(sc->Rx_m, void *)), MHLEN); > sc->Rx_m->m_len = 0; /* NB: m_gethdr does not set */ > sc->Rx_m->m_data += 20; /* make headroom */ > > Then I use a lockmgr inside my kernel module that should > make sure that we either are on TX or RX path. Uh, you can't use a lockmgr lock in interrupt handlers or in if_transmit/if_start routines. You should most likely just be using a plain mutex instead. Also, new code shouldn't use lockmgr in general. If you need a sleepable lock, use sx instead. It has a more straightforward API. > The problem seems to be at [2], somehow after swapping > buckets in uma_zfree m_dup returns a pointer to > an mbuf that is still being used by us, [1] and [3] > have same address. > Then we call m_freem twice on same mbuf, [4] and [5]. > And a loop occurs inside uma_free. > I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? > Problem seems to occur while swapping buckets. Hmm, the uma uses its own locking, so it should be safe, yes. However, you are correct about [1] and [3]. The thing is, after [1] the mbuf shouldn't be in any buckets at all (it only gets put back into the bucket during a free). Are you sure the mbuf wasn't double free'd previously? -- John Baldwin