From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 13 15:50:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B96C11065672 for ; Tue, 13 Dec 2011 15:50:01 +0000 (UTC) (envelope-from monthadar@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 729FE8FC14 for ; Tue, 13 Dec 2011 15:50:01 +0000 (UTC) Received: by iakl21 with SMTP id l21so7394071iak.13 for ; Tue, 13 Dec 2011 07:50:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=slrp51mCFyNxfkAg9IVF5SIAAkEa4Q/LO4fzww/FSp0=; b=ka+DGgHjyV87+iTZCdYaZ4KlzZYk2UkaH0TBvfj8ctip+rJGMY8IkTLjlnZY1MEC3J s8DTwfTaw0FvEEtuFD640B3fOm/y54qLWEfiSj8crLnZXkXEslaDfUzPubC8rf6m5s1b yyqMdvSwPWKrM0JRJIUG7av3gIzYsxWbRcOAc= MIME-Version: 1.0 Received: by 10.42.72.135 with SMTP id o7mr17074122icj.45.1323791400786; Tue, 13 Dec 2011 07:50:00 -0800 (PST) Received: by 10.50.51.233 with HTTP; Tue, 13 Dec 2011 07:50:00 -0800 (PST) In-Reply-To: <201112130935.33975.jhb@freebsd.org> References: <201112130935.33975.jhb@freebsd.org> Date: Tue, 13 Dec 2011 16:50:00 +0100 Message-ID: From: Monthadar Al Jaberi To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: loop inside uma_zfree critical section X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 15:50:01 -0000 On Tue, Dec 13, 2011 at 3:35 PM, John Baldwin wrote: > On Tuesday, December 13, 2011 7:46:34 am Monthadar Al Jaberi wrote: >> Hi, >> >> I am not sure why I am having this problem, but looking >> at the code I dont understand uma_core.c really good. >> So I hope someone can shed a light on this: >> >> I am running on an arm board and and running a kernel module >> that behaves like a wlan interface. so I tx and rx packets. >> >> For now tx is only only sending beacon like frames. >> This is done through using ieee80211_beacon_alloc(). >> >> Then in a callout task to generate periodic beacons: >> >> =A0 =A0 m_dup(avp->beacon, M_DONTWAIT); >> =A0 =A0 mtx_lock(...); >> =A0 =A0 STAILQ_INSERT_TAIL(...); >> =A0 =A0 taskqueue_enqueue(...); >> =A0 =A0 mtx_unlock(...); >> =A0 =A0 callout_schedule(...); >> >> On the RX side, the interrupt handler will read out buffer >> then place it on a queue to be handled by wlan-glue code. >> For now wlan-glue code just frees the mbuf it instead of >> calling net80211 ieee80211_input() functions: >> >> =A0 =A0 m_copyback(...); >> =A0 =A0 /* Allocate new mbuf for next RX. */ >> =A0 =A0 MGETHDR(..., M_DONTWAIT, MT_DATA); >> =A0 =A0 bzero((mtod(sc->Rx_m, void *)), MHLEN); >> =A0 =A0 sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ >> =A0 =A0 sc->Rx_m->m_data +=3D 20; /* make headroom */ >> >> Then I use a lockmgr inside my kernel module that should >> make sure that we either are on TX or RX path. > > Uh, you can't use a lockmgr lock in interrupt handlers or in > if_transmit/if_start routines. =A0You should most likely just be using a = plain > mutex instead. =A0Also, new code shouldn't use lockmgr in general. =A0If = you > need a sleepable lock, use sx instead. =A0It has a more straightforward A= PI. Ok, I will change the interrupt handler to do something like this: disaple_interrupt(); taskqueue_enqueue(...); /* on new rx task queue */ Then on the new rx proc: sx_slock(...); m_copyback(...); enable_interrupt(); /* Allocate new mbuf for next RX. */ MGETHDR(..., M_DONTWAIT, MT_DATA); bzero((mtod(sc->Rx_m, void *)), MHLEN); sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ sc->Rx_m->m_data +=3D 20; /* make headroom */ sx_sunlock(...); I lock TX/RX paths to make sure my code is threading safe. Also because while programming my deivce (SPI communicatioin) there will be a tsleep() waiting for the DMA interrupt and thus we could be prempted by e.g. a beacon_callout etc... > >> The problem seems to be at [2], somehow after swapping >> buckets in uma_zfree m_dup returns a pointer to >> an mbuf that is still being used by us, [1] and [3] >> have same address. >> Then we call m_freem twice on same mbuf, [4] and [5]. >> And a loop occurs inside uma_free. >> I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? >> Problem seems to occur while swapping buckets. > > Hmm, the uma uses its own locking, so it should be safe, yes. =A0However,= you > are correct about [1] and [3]. =A0The thing is, after [1] the mbuf should= n't > be in any buckets at all (it only gets put back into the bucket during a > free). =A0Are you sure the mbuf wasn't double free'd previously? >From my log I can only see the mbuf being used once before by a beacon_callout and then it was freed by m_freem(). So I cant see that it was freed twice before that. How can I go by debbuging this? > > -- > John Baldwin --=20 Monthadar Al Jaberi