From owner-freebsd-wireless@FreeBSD.ORG Fri Feb 15 22:51:48 2013 Return-Path: Delivered-To: freebsd-wireless@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BE34A269 for ; Fri, 15 Feb 2013 22:51:48 +0000 (UTC) (envelope-from moonlightakkiy@yahoo.ca) Received: from nm9-vm7.bullet.mail.gq1.yahoo.com (nm9-vm7.bullet.mail.gq1.yahoo.com [98.136.218.246]) by mx1.freebsd.org (Postfix) with SMTP id 85C8FB46 for ; Fri, 15 Feb 2013 22:51:48 +0000 (UTC) Received: from [98.137.12.61] by nm9.bullet.mail.gq1.yahoo.com with NNFMP; 15 Feb 2013 22:51:47 -0000 Received: from [208.71.42.210] by tm6.bullet.mail.gq1.yahoo.com with NNFMP; 15 Feb 2013 22:51:47 -0000 Received: from [127.0.0.1] by smtp221.mail.gq1.yahoo.com with NNFMP; 15 Feb 2013 22:51:47 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.ca; s=s1024; t=1360968707; bh=LFdKPlZmdooaDlt/ReokrxqwyEJcoCK9ELhu8IwUyzI=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Received:Received:MIME-Version:X-Received:Received:Date:Message-ID:Subject:From:To:Cc:Content-Type; b=hQs7ePF6rrC/ONXwCGQPWOLfStmGrDi6P5spX0qIaCho52UVfEnqunNvJ9/bx3F1cil2er262oDkMTGTlcxnDIrOBk2pF9BODcANV7IMZ8WNb0vjfqwerJkth7/Hz8qvfkwUdb/3mvS9oVq9sMsjgU8WVD7+F/oggSOuQ8ZREzM= X-Yahoo-Newman-Id: 558393.24418.bm@smtp221.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: ILH_FBgVM1nP.gAK1Dy1krPFQBOyA2LjGLuqu3ZEWwncKv4 1dff3OcJnur0uxl0wjkrc7xCw.oEH3Op_YP_ajX1t2_.G17DnuiSFQ9.AJ2W F1whXhHAaYbzXrE9oyjOj.L3w7_xiLm9xokNfdOqGUY2y.0vpA3.a.bwkfqN 9D0IYP.aTXaNyGZ9zrk2ekWN3nNcfhQfmT.N84esLue8qn7WmNcyFUZ0NxjA LeN5ZV18Ool9xM6km2emkgIuWOG.FX1jXqcS4H21eAE8Gjkjs1G7MAOyAHKs SYEBjXbMG3_gUHA6FcmSImFhTnBUb2TSbflWTFrrZ2DRf0_PUdcCVVKlljom ZgiXjnPwhRUYu1AHykbS8xPqbOAcDteyjY7AshcVUouI6LpldjOzqiXRGlgm qQ4U6HnS4qTbWxzphux7145X8zTPQZuVnx30mClQCFkEMKHDh0PH0xjAZQID dgMpoamt.q8q7Bh7i9ik- X-Yahoo-SMTP: Xr6qjFWswBAEmd20sAvB4Q3keqXvXsIH9TjJ Received: from mail-lb0-f174.google.com (moonlightakkiy@209.85.217.174 with plain) by smtp221.mail.gq1.yahoo.com with SMTP; 15 Feb 2013 14:51:47 -0800 PST Received: by mail-lb0-f174.google.com with SMTP id l12so2992885lbo.33 for ; Fri, 15 Feb 2013 14:51:45 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.152.130.131 with SMTP id oe3mr3698478lab.1.1360968705476; Fri, 15 Feb 2013 14:51:45 -0800 (PST) Received: by 10.112.1.37 with HTTP; Fri, 15 Feb 2013 14:51:45 -0800 (PST) Date: Fri, 15 Feb 2013 15:51:45 -0700 Message-ID: Subject: Re: [RFC] serialising net80211 TX From: PseudoCylon To: Adrian Chadd , Monthadar Al Jaberi Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-wireless@freebsd.org X-BeenThere: freebsd-wireless@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussions of 802.11 stack, tools device driver development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2013 22:51:48 -0000 > ------------------------------ > > Message: 2 > Date: Wed, 13 Feb 2013 21:14:53 -0800 > From: Adrian Chadd > To: freebsd-wireless@freebsd.org > Subject: [RFC] serialising net80211 TX > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi, > > I'd like to work on the net80211 TX serialisation now. I'll worry > about driver serialisation and if_transmit methods later. > > The 30 second version - it happens in parallel, which means preemption > and multi-core devices can and will hit lots of subtle and > hard-to-debug races in the TX path. > > We actually need an end-to-end serialisation method - not only for the > 802.11 state (sequence number, correct aggregation handling, etc) but > to line up 802.11 sequence number allocation with the encryption IV/PN > values. Otherwise you end up with lots of crazy subtle out of order > packets occuring. The other is the seqno/CCMP IV race between the raw > transmit path and the normal transmit path. There are other nagging > issues that I'm trying to resolve - but, one thing at a time. > > So there are three current contenders: > > * wrap everything in net80211 TX in a per-vap TX lock; grab it at the > beginning of ieee80211_output() and ieee80211_start(), and don't > release it until the frame is queued to something (a power save queue, > an age queue, the driver.) That guarantees that the driver is called > in lock-step with each frame being processed. Long held locks could be worse. While another thread holding a lock, when one thread try to grab a lock, it will spin a bit then be suspended. This could be more expensive than context switching by the scheduler. If a thread hold a lock longer, there will be more chance this to happen. > * do deferred transmit- ie, the net80211 entry points simply queue > mbufs to a queue, and a taskqueue runs over the ifnet queue and runs > those frames in-order. There's no need for a lock here as there's only > one sending context (either per-VAP or per-IC). I tried taskqueue on run(4), changed it from 1) to 2). Queue with shared thaskqueue, i.e taskqueue_thread, didn't work good, but with own taskqueue works good (60mbp->70+mbp). The catch is I only tested on core2 duo and dual core + ht atom. I saw you patched ath(4) to use taskqueue. How is it working? There are threads other than Tx thread are running on a system, so context switching will happen one way or other. We want to do it smart way, i.e. rather than changing thread on every tx, switch on multiple tx. Currently, tx threads are kept alive until packets are passed to h/w. Instead, if we kill them right after queuing a frame to if_snd and let one thread (would be new ieee80211_tx thread) handle all the packets, there will be fewer threads, so should be less context switching. Anyhow, I don't think we need to queue/dequeue twice, one on vap->iv_ifp->if_snd and one on ic->ic_ifp->if_snd. AK