From owner-freebsd-net@FreeBSD.ORG Wed Aug 7 20:13:44 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 26247B48 for ; Wed, 7 Aug 2013 20:13:44 +0000 (UTC) (envelope-from scott4long@yahoo.com) Received: from nm14-vm6.bullet.mail.ne1.yahoo.com (nm14-vm6.bullet.mail.ne1.yahoo.com [98.138.91.107]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A8C6E2645 for ; Wed, 7 Aug 2013 20:13:43 +0000 (UTC) Received: from [98.138.90.57] by nm14.bullet.mail.ne1.yahoo.com with NNFMP; 07 Aug 2013 20:08:31 -0000 Received: from [98.138.84.39] by tm10.bullet.mail.ne1.yahoo.com with NNFMP; 07 Aug 2013 20:08:31 -0000 Received: from [127.0.0.1] by smtp107.mail.ne1.yahoo.com with NNFMP; 07 Aug 2013 20:08:31 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1375906111; bh=IR9oBaPkDlnUFSzZL0CnnqJa0XoQ4zgP8uUQ5F9ZiBg=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=rkLuNmvekaud9/jOv/oD7hdqJW5kDYGpYnx13x1Saz9nvLhVgEdXj6duM9xyOXlHvvAeT+CjAaLyL8HPHdv5I4iJpvGSN+BNOt7+etr9xKL9A9GY99eiQEd+xCQRtPeW3yHoQ8bNFy51GPzHbq6zLCbC3iwfUUlXeqceiyrFU0Y= X-Yahoo-Newman-Id: 236115.32192.bm@smtp107.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: fgmBe8AVM1mafGhh1bx84CL3Aa9JBL.MZZz_uPWDjFXWH6o IfBCuts0hZxyW1oGA.q3ZOw45AceUTITvPasRuNWo3jj0wlJRAFjLYy9INL4 m48TAIovVQMzyshAwgxSzjYHwyC3zrmkoqpix5pOHku0KoPntKBLZjrLolm2 hvlRDIe6c99hZ9CcFeZbIhrgDzmuYGcCLlsheJFmeScjIj2bf1waOM9umfbl bmSpszN.9lUXcNS51ou6OEJxdpbZCvfCvvWE63KsSGj1AHB7cKHllYJKpCnP D0qCpyDgV41BNga2LL5MzhLY3QoRy8EBvporNEnen.w4lVr5wNu..j1EAz_A V.nWqyV34dCsyPC_.iLnQQ8tRKaPt1F_o682Knv6qS4Zdy8lYmmPhRh.rr0g lqR6EwdtDDGbrhB_L3Zo8hCmg1_GO9wqJLs9i4.W6j0CJXxQS8g0hWaDDvjj usAzPsnjlrz6mXwgm.zXnOKbBVEF.O4dA1zWXycp6SyzzE1wax6HZg2HtPbm Gp82bNdYNlt_v2Bu0VgijNXxNRuuOQ.Lk8xZvWuC8V3a1lkYiZUz4iOfhKyZ 1uQ-- X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- X-Rocket-Received: from phobos.corp.netflix.com (scott4long@69.53.237.66 with ) by smtp107.mail.ne1.yahoo.com with SMTP; 07 Aug 2013 13:08:31 -0700 PDT Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: [net] protecting interfaces from races between control and data ? From: Scott Long In-Reply-To: <5202A74E.4060602@freebsd.org> Date: Wed, 7 Aug 2013 14:08:29 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20130805215319.GA43271@onelab2.iet.unipi.it> <201308070326.r773QCLD035541@mail.karels.net> <5202A74E.4060602@freebsd.org> To: Andre Oppermann X-Mailer: Apple Mail (2.1508) Cc: Adrian Chadd , mike@karels.net, Bryan Venteicher , Navdeep Parhar , net@freebsd.org, Giuseppe Lettieri , Luigi Rizzo X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Aug 2013 20:13:44 -0000 On Aug 7, 2013, at 2:00 PM, Andre Oppermann wrote: > On 07.08.2013 09:18, Luigi Rizzo wrote: >> On Wed, Aug 7, 2013 at 5:26 AM, Mike Karels > wrote: >> Jumping to (near) the end of the thread, I like most of Andre's = proposal. >> Running with minimal locks at this layer is an admirable goal, and = I agree >> with most of what was said. I have a few observations on the = general changes, >> or related issues: >>=20 >> There was mention of taskqueues. I think that with MSI-X, = taskqueues >> should not be needed or used. More specifically, having separate = ithreads >> and taskqueues, with ithreads deferring to taskqueues after some = limit, makes >> sense only for MSI and legacy interrupts. With MSI-X, an = interrupt thread >> should be able to process packets indefinitely with sufficient CPU = resources, >> and there is no reason to context switch to a different thread = periodically. >> A periodic "yield" might be reasonable, but if it is necessary, = small packet >> performance will suffer. However, most of this is internal to the = driver. >>=20 >>=20 >> i am not completely clear on what is the difference between ithreads = and taskqueues. >=20 > The difference between ithreads and taskqueues is actually very small = and mostly in > name and how they're set up and kicked off when work is to be done. = Both are real > kernel threads. Most of the confusion, also in Mikes response, seems = to come from > the name ithreads for interrupt-threads. However an ithread is not = running in > interrupt context, only the interrupt handler (called filter) does. = Scheduling a > taskqueue from an ithread is a bit round-about but commonly done. The = bus_setup_intr(9) > man page isn't helping to clear up the confusion. >=20 I think it might still be the case that threads are not allowed to = sleep. That means no sx locks and no malloc(M_WAITOK), along with the obvious = tsleep/msleep. Taskqueues have no such restriction. An even rore relevant difference is that taskqueues have a much stronger management API. Ithreads can only be scheduled by generating a hardware = interrupt, can only be drained by calling bus_teardown_intr(), and cannot be = paused. Taskqueues give you direct access to this control. Scott