From owner-freebsd-net@freebsd.org Sun Apr 2 08:04:45 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B5B1D29580 for ; Sun, 2 Apr 2017 08:04:45 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A380A3F for ; Sun, 2 Apr 2017 08:04:44 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by mail-oi0-x22a.google.com with SMTP id o67so98626267oib.1 for ; Sun, 02 Apr 2017 01:04:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=t6LUTL7LEbRFXtyNnTBxx8DbMSpOorC8MP0bEcnNyaM=; b=cTVYG0V0fT2vw1MiBxonmiUO8IKwqLJjmKUE0AHdj/0T4teh15PuCuoCD8C5ECxMuu Wn7afLOSjvGPXg13BPznu8HPhBVwWtiSblVpV0nFA20YHNdj7Ur/f33IeKAdFpQJnT59 f6o8OErxJV/6Snh/v8VqVUMuWv+anq9l6Paxw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=t6LUTL7LEbRFXtyNnTBxx8DbMSpOorC8MP0bEcnNyaM=; b=JOoR+SgnPXH5DgK8fykbpgewGC9q1BlCQ6HO4946GpQNFqye+pN5YxhAu2rNN22OOo te+91TvCyv1jAkhSRSzkUYMXtFDj0oaaDb4POqJLZ9lUHtCjggBCT+UwGjQyUeiaYg7Z inGhjpCBgkCtPzckqcrEyIdBHGUuqHVdpocNFf5LtBgQPoTmu4+3gKIP9xquPcA/7KkJ AUZFMTfriCE2hd2AnP3Br1BvBe60OQlhr5GqvNEPT4KPO/jx9fxuRGeeRb5nvoOhAyRo miNbzO8B+HY7xo45g0Yrne0XB0wL7gYdLq0pqx3/417r1nCTcOf0CUsprMwNKQG/iw60 75cw== X-Gm-Message-State: AFeK/H0jS6nJs5jPp2L1kLPxdGybAA/69dq9FjTVWk0VIjn8RwP42DlcvRC0cgilxTkZ4sUxhYuBuwcV03kZJg== X-Received: by 10.202.169.139 with SMTP id s133mr5745019oie.184.1491120284244; Sun, 02 Apr 2017 01:04:44 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.3.239 with HTTP; Sun, 2 Apr 2017 01:04:43 -0700 (PDT) In-Reply-To: References: From: Kevin Bowling Date: Sun, 2 Apr 2017 01:04:43 -0700 Message-ID: Subject: Re: interface down, console output: igb3: TX(7) desc avail = 41, pidx = 308 To: Ben Woods Cc: freebsd-net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Apr 2017 08:04:45 -0000 Sean Bruno committed a couple fixes to the watchdog code this week that should at least allow for a usable TSO although the frequency of the watchdog events is still cause for concern. It seems some timeouts are part of Intel's expectations during normal operations for several chipsets. If you could share which exact NIC chipset you have I will check the datasheets and see if we're missing anything. Regards, On Sat, Apr 1, 2017 at 6:41 PM, Ben Woods wrote: > On 27 March 2017 at 15:35, Kevin Bowling wrote: > >> Try turning TSO off.. i.e. ifconfig igb3 -tso or sysctl net.inet.tcp.tso=0 >> >> The transition to iflib has exposed much jankiness in the Intel "shared >> code" of the e1000 drivers. In particular, the locking contracts may not >> align with FreeBSD locking primitives. I have boxes running the legacy >> driver that are clearly reliant on the watchdog reset for steady state >> which is unacceptable. We are actively looking into this at LLNW, but >> additional reports and help are appreciated. >> > > > Hi Kevin, > > Thanks for the reply. Sorry it took so long for me to get back to you, I > first had to wait for the problem to be repeated. > > Indeed, running "ifconfig igb3 -tso" did fix the issue. It didn't seem to > the first time, but I may have been too quick to judge. After re-enabling > TSO and disabling it a second time, the problem stopped, and the interface > immediately came up. > > Please let me know if there is anything I can do to try and help diagnose > this. In the mean time, I have added net.inet.tcp.tso=0 to my > /etc/sysctl.conf, so I don't think this will re-occur unless I remove it. > > Regards, > Ben > > -- > From: Benjamin Woods > woodsb02@gmail.com >