From owner-freebsd-current@FreeBSD.ORG Mon Aug 5 08:57:13 2013 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A6E00CC5 for ; Mon, 5 Aug 2013 08:57:13 +0000 (UTC) (envelope-from scott4long@yahoo.com) Received: from nm43-vm10.bullet.mail.bf1.yahoo.com (nm43-vm10.bullet.mail.bf1.yahoo.com [216.109.114.171]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 10F572FD1 for ; Mon, 5 Aug 2013 08:57:12 +0000 (UTC) Received: from [98.139.215.140] by nm43.bullet.mail.bf1.yahoo.com with NNFMP; 05 Aug 2013 08:57:06 -0000 Received: from [68.142.230.76] by tm11.bullet.mail.bf1.yahoo.com with NNFMP; 05 Aug 2013 08:57:06 -0000 Received: from [127.0.0.1] by smtp233.mail.bf1.yahoo.com with NNFMP; 05 Aug 2013 08:57:06 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1375693026; bh=g6MvKxMXufr38DVZ9QRpodvH8ZmW/f1OMpHnkowr2k4=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=gxNqdyBz7JXtFiVhtRTQtwlMJedchz/fzQ2nKd9QiylodjiJJYMW7NGGsOWcr9SwcXyrsB9DiQOytwrSWXWz0qaqqTI86tRLzvglbyOeylnS/AkrbZhMEbiiOLhqEWOwFBCR2YARG3LnpLNs119+VhpjTdxD5fGEFedvww+nOAo= X-Yahoo-Newman-Id: 186511.61154.bm@smtp233.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 2diJR4IVM1kXMFVXVWX7nonHpAFk90cHXkMTV.HVJZFXG1w llGVwNsKZkA_dNp9tT8IaIS4Rwgzk5TeqL.s6JXjitWlgoDlxhKHmuAD.6ng 5NzmGIuyP.atDetYsoBtg3tzW6H8xbYO2OQ8oafqcNI40hNuJ8FTj_QWF2J8 JAnZxH86Xg1sFVvl..Qr8ZrzUVjVtXYkC6TspXHtQJBS.PHoOe2p0F50oW7X _9eM8lnu5qexXTUjeHGReqSrkHy9Zkubjugbf.MJEg.wF7Ltg93Jk0EDQ8sE jbVu7P0MlNDBvCvk9rQ.HdSQc9LMFnRz7h6QeiYlNGEmlU7nSm9b4V0bFKyM JlOf.WeEE9S2bOWZ211_E6_K2z2tRMs7UKPtynZFeVtjkAAzK1cxXiA75Pe1 2gXdGUdrTJC52aJ3uzbYiUvlpieEFBZ8axKt.ABbjGKEawVd9Sv0bm9mP6_x J55hyCbfw_6mcYbqclk3YE2z5AWjh5dUqho3yZpXf8g8nRCLtT8XcJpAKXve gQhLOhlDq3IGkvf9BadwPLaWfOw6xDBO45ToDifPVoJVlyL33VEE_hRwHYg- - X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- X-Rocket-Received: from [192.168.254.206] (scott4long@168.103.85.57 with ) by smtp233.mail.bf1.yahoo.com with SMTP; 05 Aug 2013 08:57:06 +0000 UTC Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: [net] protecting interfaces from races between control and data ? From: Scott Long In-Reply-To: <20130805082307.GA35162@onelab2.iet.unipi.it> Date: Mon, 5 Aug 2013 02:57:04 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <179713DC-926B-4E8A-9BE4-EC4337B8736E@yahoo.com> References: <20130805082307.GA35162@onelab2.iet.unipi.it> To: Luigi Rizzo X-Mailer: Apple Mail (2.1508) X-Mailman-Approved-At: Mon, 05 Aug 2013 12:01:24 +0000 Cc: current@freebsd.org, net@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Aug 2013 08:57:13 -0000 On Aug 5, 2013, at 2:23 AM, Luigi Rizzo wrote: > i am slightly unclear of what mechanisms we use to prevent races > between interface being reconfigured (up/down/multicast setting, etc, > all causing reinitialization of the rx and tx rings) and >=20 > i) packets from the host stack being sent out; > ii) interrupts from the network card being processed. >=20 > I think in the old times IFF_DRV_RUNNING was used for this purpose, > but now it is not enough. > Acquiring the "core lock" in the NIC does not seem enough, either, > because newer drivers, especially multiqueue ones, have per-queue > rx and tx locks. >=20 > Does anyone know if there is a generic mechanism, or each driver > reimplements its own way ? >=20 I'll speak to the RX side of the question. Several years ago I modified = the if_em driver to use a fast interrupt handler that would signal actual = processing in a taskqueue thread. The result was a system with no more latency = than the classic ithread model, but with the ability to allow RX processing = to be halted, drained, and restarted via an explicit API. This in turn was = extended to cause the RX processing to be safely paused during the control events that you described above. The system worked well on many fronts, but unfortunately I was unable to actively maintain it, and it was slowly garbage collected over time. I = think that it could have been extended without much effort to cover = TX-complete processing. TX dispatch is a different matter, but I don't think that = it would be hard to have the if_transmit/if_start path respond to control = synchronization events. Scott