From owner-freebsd-net@FreeBSD.ORG  Mon Aug  5 15:46:07 2013
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 9FFCFAAA;
 Mon,  5 Aug 2013 15:46:07 +0000 (UTC)
 (envelope-from adrian.chadd@gmail.com)
Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com
 [IPv6:2a00:1450:400c:c05::233])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 0ED8F240B;
 Mon,  5 Aug 2013 15:46:06 +0000 (UTC)
Received: by mail-wi0-f179.google.com with SMTP id hr7so1688988wib.12
 for <multiple recipients>; Mon, 05 Aug 2013 08:46:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:sender:in-reply-to:references:date:message-id:subject
 :from:to:cc:content-type;
 bh=wcEuz7SwM5lnp1ELEuS15xUD9fl379Cc+SC2aQRyM78=;
 b=tcOyfMPS041odEFpBwOXKVh3RL9uDtZGC5KYflLdbx/bEezHlUCNgeKvTJzAi6h/yy
 h1z44vSNFYQWO8N0LdNqaehfXX53/vS9Hf1Iy6c0PjQ7jgqLkv4I3g/f7VNbB1URoux+
 EYL9yvIUL4baqHjIsHNW1JMToKPjz3wSZ6ybdL9DKKvYZczxCgzxmnS/7YYr+qirCXM+
 wmxgPaEwr6VZMfBEOA5JcQXIygAi9rO7F+fdlsNiorXDQcykENFZ95xsZLZmM9NBNUFF
 Gez1mdGngJ8Mo83Jlxgbpy1TwAekc+enhL1c5seJVSsAMclieiARV09noAa1rGB4Drds
 p4CQ==
MIME-Version: 1.0
X-Received: by 10.180.160.165 with SMTP id xl5mr7139867wib.46.1375717565346;
 Mon, 05 Aug 2013 08:46:05 -0700 (PDT)
Sender: adrian.chadd@gmail.com
Received: by 10.217.116.136 with HTTP; Mon, 5 Aug 2013 08:46:05 -0700 (PDT)
In-Reply-To: <2034715395.855.1375714772487.JavaMail.root@daemoninthecloset.org>
References: <20130805082307.GA35162@onelab2.iet.unipi.it>
 <2034715395.855.1375714772487.JavaMail.root@daemoninthecloset.org>
Date: Mon, 5 Aug 2013 08:46:05 -0700
X-Google-Sender-Auth: LktrRb4KALl0kOzD1nz2ankEW3E
Message-ID: <CAJ-VmokT6YKPR7CXsoCavEmWv3W8urZu4eBVgKWaj9iMaVJFZg@mail.gmail.com>
Subject: Re: [net] protecting interfaces from races between control and data ?
From: Adrian Chadd <adrian@freebsd.org>
To: Bryan Venteicher <bryanv@daemoninthecloset.org>
Content-Type: text/plain; charset=ISO-8859-1
Cc: Luigi Rizzo <rizzo@iet.unipi.it>, current@freebsd.org, net@freebsd.org
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 05 Aug 2013 15:46:07 -0000

On 5 August 2013 07:59, Bryan Venteicher <bryanv@daemoninthecloset.org> wrote:

> What I've done in my drivers is:
>   * Lock the core mutex
>   * Clear IFF_DRV_RUNNING
>   * Lock/unlock each queue's lock

.. and I think that's the only sane way of doing it.

I'm going to (soon) propose something similar for cxgbe/ixgbe as we
use these NICs at work, then feed this experiment back into the
network stack so we can have a unified way of doing this.

You may also want to synchronize against the driver TX/RX/core locks
and state when doing things like, say, halting DMA in preparation for
multicast reprogramming on some hardware; or even doing a chip reset.

I had to hand-roll this for ath(4) to make it completely correct - any
kind of overlapping reset, reset during TX, reset during RX etc would
cause all kinds of instability and random-crap-scribbled-everywhere
issues. So yes, this is a larger scale issue that needs to be solved.


-adrian