From owner-freebsd-net@FreeBSD.ORG Sun Sep 7 02:06:13 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DAABE23F for ; Sun, 7 Sep 2014 02:06:13 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB30D1FBB for ; Sun, 7 Sep 2014 02:06:13 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s8726DQa009777 for ; Sun, 7 Sep 2014 02:06:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 193246] Bug in IPv6 multicast join(), uncovered by Jenkins Date: Sun, 07 Sep 2014 02:06:14 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: rodrigc@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Sep 2014 02:06:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193246 --- Comment #11 from Craig Rodrigues --- -------- Forwarded Message -------- Subject: Re: [Bug 193246] Bug in IPv6 multicast join(), uncovered by Jenkins Date: Fri, 05 Sep 2014 03:05:14 +0100 From: Bruce Simpson To: bugzilla-noreply@freebsd.org On 04/09/2014 19:16, bugzilla-noreply@freebsd.org wrote: > I think what this means is that in_mcast6.c is very much tied to doing MLDv6 > (or whatever the v6 equivalent of IGMP is), but for these groups, you need to > be somehow calling into in_mcast.c to do IGMP instead. However, I'm not sure > at what level in_mcast6.c needs to call into in_mcast.c myself. I do think > Bruce is your best bet for someone to ask. > Unfortunately I'm fully booked. But yes, that is a good summary. If Craig (or someone else) is willing to volunteer to support v4-mapped addresses: one approach would be to extend in6_mship{} to include them. Pushing state down to IGMP will need to be added as a special case. To keep it simple, assume that only the legacy any-source multicast (ASM) model will be supported, i.e. listeners will not specify source filters. Looking at the JDK source, it appears they used to handle the v4/v6 swizzle themselves because of limitations in Linux 2.4/2.6. In other words, we do not support RFC 3493 Sec 3.7 for multicast groups at the moment. A more appropriate errno value to return is EPROTONOOPT. [Interestingly, Sec 5.2 discusses IPv6 multicast options, but does not contain any reference to IPv4 at all.] There now follows a late-night writeup of the rationale behind this code -- and this is as concise as I can make it, I'm afraid. in[6]_mcast.c is split into a top half (i.e. per-socket state) and a bottom half (i.e. stack-wide state, and the IGMPv1/2/3 and MLDv1/2 wire protocols). IPv6 mcast options are processed separately from IPv4. Both implement a transaction scheme to protect global membership state (in the bottom half) from failures at the granularity of a single socket (or PCB) in the top half. Why all the complexity? Well, this is to support source-specific multicast (SSM, aka "inclusive mode"). To cut a long story short: as the size of an internetwork increases, it gets more difficult for routers to track the state of multicast listeners, unless they are aware of where the traffic originates from. The book "Interdomain Multicast Routing" by Brian M. Edwards discusses this in lurid detail. So, SSM was introduced to support inter-domain multicast. In this model, joining a multicast group is no longer a simple matter of registering for a channel -- you must also specify the sources you are interested in. However, the majority of multicast applications are built on the older model: i.e. they do not cross more than one IP network hop, and do not specify sources ("any-source multicast", aka ASM). The network stack must be able to cope with both of these uses. It does so by representing the legacy ASM scheme as "exclusive mode". The RFC 3678 APIs also have the advantage that the application can block unwanted senders, even if ASM is in use. [The main API it specifies, setsourcefilter(), does not explicitly mandate v4-mapped support.] So, in the bottom half of mcast, each group has a shared RB-tree of listener state. This is created from the set union of the filter state on each subscribed socket. If there are no filters for a particular group, i.e. all of the sockets/PCBs in the system are in "exclusive" mode and have no filters, then of course the RB-tree for that group will be empty. Otherwise, if there is a mix of "exclusive" and "inclusive" mode listeners, the tree will need to be recomputed. The shared tree is then used to fill out the IGMP/MLD messages sent to on-link routers to request group subscription. It is also used to filter input traffic from within the relevant transports (e.g. UDP, raw sockets). Previously, this filtering required that the network layer take a socket-layer lock. In closing: this isn't just a simple matter of adding a few defines. The v6 code will need to check that a v4-mapped group was passed, and make sure to perform the transaction pushdown to IGMP. -- You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sun Sep 7 12:49:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 065DDED7 for ; Sun, 7 Sep 2014 12:49:33 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BDBF51C03 for ; Sun, 7 Sep 2014 12:49:32 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id l6so14618840qcy.14 for ; Sun, 07 Sep 2014 05:49:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=TLohPU0JBy0D8vaoPX6eUo2fsdA2DvyGUTeYqgNdDdE=; b=fU1PEONHwPAiwpVuKGZaAWdGoRFLXk75Sj+B2FH3FbV2XNjIw4BlS8Huz0TF0U5o8D 2d1RS0uGzTunwp4sJsTQlLa47vm2X/RwrKjQHfTBTrA/ioU7akgulpRTMlNTiFEDxNbG uyvl54VimCNJGg5ndpoWyEh3xklhqTwqyQt7JqCibuEBZ8fR6AqxDyOUP2pi5c10K0B/ GowlBhzUNqq8zN/eqxUb5XYpjeP+a8yYhACqYHlUB7cDakh19HgD15J2DgzzYI8RziLQ 987gxC5bedR7x7wbuYoHJTrp5UlY4ukd15mpvG3lYwNnLTNDsl6mfC4FbbUU3Tb1cHS/ ws7g== MIME-Version: 1.0 X-Received: by 10.224.60.129 with SMTP id p1mr799397qah.99.1410094171901; Sun, 07 Sep 2014 05:49:31 -0700 (PDT) Received: by 10.229.234.194 with HTTP; Sun, 7 Sep 2014 05:49:31 -0700 (PDT) Date: Sun, 7 Sep 2014 17:19:31 +0430 Message-ID: Subject: netmap pkt-gen From: Mahnaz Talebi To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Sep 2014 12:49:33 -0000 Hi Dear Luigi, I use your netmap pkt-gen for testing my applications and thanks you for your great nice work. I want to modify (if I can!) your pkt-gen such that I can insert timestamps in udp packets in order to find delay of receiving packets (that forward from my DUT). =D9=90Do you have any suggestion to me to do this work? Thanks in advance. From owner-freebsd-net@FreeBSD.ORG Sun Sep 7 17:03:14 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4E31B0A for ; Sun, 7 Sep 2014 17:03:14 +0000 (UTC) Received: from oj.bangj.com (amt0.gin.ntt.net [129.250.11.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE3CB1E7D for ; Sun, 7 Sep 2014 17:03:14 +0000 (UTC) Received: from [172.16.25.105] (69-77-129-4.static.skybest.com [69.77.129.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by oj.bangj.com (Postfix) with ESMTPSA id 1F37B13095; Sun, 7 Sep 2014 12:54:02 -0400 (EDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1974.6\)) Subject: Re: RFC 7217 From: Tom Pusateri In-Reply-To: <21515.36548.561554.872920@hergotha.csail.mit.edu> Date: Sun, 7 Sep 2014 12:53:58 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <2354C099-C801-43D2-8EE9-0DA0AAAD8181@bangj.com> References: <21515.36548.561554.872920@hergotha.csail.mit.edu> To: Garrett Wollman X-Mailer: Apple Mail (2.1974.6) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Sep 2014 17:03:14 -0000 I read the draft and wasn't sure what problem they were trying to solve. = They talk about moving from network to network which seems aimed at = laptops or mobile devices. But randomizing the interface id but keeping = it the same on a particular network allows consistency on each network = but you could still be tracked on that network (like using the MAC = address) even though you couldn't be tracked across networks (unlike = using the MAC address). Why not just use temporary addresses (RFC 4941)? On the server side, this draft could be implemented in rtadvd. Not sure = who maintains that. Is that what you're looking for? To see if someone is working on changes = to rtadvd? Thanks, Tom > On Sep 6, 2014, at 6:46 PM, Garrett Wollman = wrote: >=20 > So is anyone working on an RFC 7217 ("Stable and Opaque IIDs with > SLAAC") implementation for FreeBSD yet? >=20 > -GAWollman >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Sep 7 17:23:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 510AA98C for ; Sun, 7 Sep 2014 17:23:55 +0000 (UTC) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E970C13D0 for ; Sun, 7 Sep 2014 17:23:54 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s87HNoCA009712; Sun, 7 Sep 2014 13:23:50 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s87HNoUG009711; Sun, 7 Sep 2014 13:23:50 -0400 (EDT) (envelope-from wollman) Date: Sun, 7 Sep 2014 13:23:50 -0400 (EDT) Message-Id: <201409071723.s87HNoUG009711@hergotha.csail.mit.edu> From: wollman@bimajority.org To: pusateri@bangj.com Subject: Re: RFC 7217 References: <21515.36548.561554.872920@hergotha.csail.mit.edu> <2354C099-C801-43D2-8EE9-0DA0AAAD8181@bangj.com> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (hergotha.csail.mit.edu [127.0.0.1]); Sun, 07 Sep 2014 13:23:50 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED, HEADER_FROM_DIFFERENT_DOMAINS autolearn=disabled version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Sep 2014 17:23:55 -0000 In article <2354C099-C801-43D2-8EE9-0DA0AAAD8181@bangj.com>, pusateri@bangj.com write: >I read the draft and wasn't sure what problem they were trying to solve. The problem of having zillions of IPv6 addresses in use on a network with far fewer than a zillion actual end stations, which no network in existence is engineered to support. And also the requirement of network operators to be able to trace problem IP addresses back to end stations, which doesn't work if the end stations are constantly generating new IP addresses and abandoning the old ones. This has caused many network operators to disable SLAAC and use DHCPv6 instead, and I'm about to have to go down this path as well. >Why not just use temporary addresses (RFC 4941)? Because they're evil. >On the server side, this draft could be implemented in rtadvd. Not sure >who maintains that. There is no client/server here. The network infrastructure doesn't *care* how the client generates its IID. What it does care is that the client generate one, unique, long-term-stable, IID, rather than using nine different ones that change all the time. rtadvd has nothing to do with this. (Note: FreeBSD doesn't currently ship with RFC 4941 enabled. Most other operating systems do. RFC 7217 needs to be implemented, as a replacement for RFC 4941, before it is even considered to turn this switch on in FreeBSD.. RFC 7217 *actually solves the problem* that RFC 4941 was intended to solve, without requiring network operators to size their switch hardware for ten times the number of addresses as they have end stations.) -GAWollman From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 01:32:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B096BECD for ; Mon, 8 Sep 2014 01:32:59 +0000 (UTC) Received: from gpo4.cc.swin.edu.au (gpo4.cc.swin.edu.au [136.186.1.33]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C61B174A for ; Mon, 8 Sep 2014 01:32:58 +0000 (UTC) Received: from [136.186.229.154] (nwilliams-laptop.caia.swin.edu.au [136.186.229.154]) by gpo4.cc.swin.edu.au (8.14.3/8.14.3) with ESMTP id s881Wthg015316 for ; Mon, 8 Sep 2014 11:32:55 +1000 Message-ID: <540D0741.6030403@swin.edu.au> Date: Mon, 08 Sep 2014 11:32:49 +1000 From: Nigel Williams User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Multipath TCP for FreeBSD v0.4 References: <513CB9AF.3090409@swin.edu.au> <53BF8945.3000802@swin.edu.au> In-Reply-To: <53BF8945.3000802@swin.edu.au> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 01:32:59 -0000 Hi, We recently released a new tech report "Design Overview of Multipath TCP version 0.4 for FreeBSD-11" [1]. The report provides some details on various aspects of the implementation (session management, data-level retransmission etc), as of the most recent v0.4 patch [2]. cheers, nigel [1] http://caia.swin.edu.au/reports/140822A/CAIA-TR-140822A.pdf [2] http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html On 11/07/14 16:50, Nigel Williams wrote: > Hello all, > > A new v0.4 patch is available at [1]. This release is mostly bug-fixes > and improvements to core functionality (establishing/closing > connections, retransmissions etc), and also brings the implementation up > to a more recent version of FreeBSD-HEAD. > > The full list of changes and caveats can be found in [2] and [3], but > briefly: > - Patched against r265307 of FreeBSD-HEAD. This is prior to some recent > TCP reassembly memory management changes and the patch will be brought > up to a newer revision soon (currently working on integrating those > changes). > - Added data-level retransmits and subflows can now stall and recover > (or timeout) during a connection. > - The path management and packet scheduler are still fairly rudimentary, > and I haven't yet implemented coupled CC. > - The patch is still under heavy development so consider this release > code to be of alpha quality. > > This release ties up work that was partially supported by a gift from > The Cisco University Research Program Fund. Future releases will be > supported by a grant from the FreeBSD Foundation. > > P.S. I will be working on the patch full-time again so updates should be > a little more frequent from this point onwards. > > cheers, > nigel > > [1] http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html > [2] http://caia.swin.edu.au/urp/newtcp/mptcp/tools/mptcp-changelog-v0.4.txt > [3] http://caia.swin.edu.au/urp/newtcp/mptcp/tools/mptcp-readme-v0.4.txt > > > > On 11/03/13 03:49, Lawrence Stewart wrote: >> Hi all, >> >> The CAIA MPTCP team is pleased to announce the initial release of our >> multipath TCP implementation for FreeBSD 10-CURRENT which is available >> from [1]. This release contains wire-related protocol code and a lot of >> core stack infrastructure. It is capable of running regular TCP flows >> and single or multi-subflow MPTCP flows (with some caveats as documented >> in the readme [2]). >> >> We consider this code to be of alpha quality and plan to release >> frequent updates going forward as we continue to flesh out additional >> features and fix the rough edges. >> >> That being said, we welcome everyone to start playing with the code and >> provide feedback, bug reports, fixes, praise and/or abuse ;) >> >> The "Multipath TCP for FreeBSD" project team consists of: >> >> Nigel Williams: lead R&D engineer >> Lawrence Stewart: supporting R&D engineer >> Grenville Armitage: principal investigator & overall project lead >> >> Many thanks go to the Cisco University Research Program Fund at >> Community Foundation Silicon Valley for their support of this work. >> >> Have fun with it! >> >> Cheers, >> Lawrence, Nigel & Grenville >> >> http://caia.swin.edu.au >> >> >> >> [1] http://caia.swin.edu.au/urp/newtcp/mptcp/tools.html >> >> [2] http://caia.swin.edu.au/urp/newtcp/mptcp/tools/mptcp-readme-v0.1.txt >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 05:53:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA1DC6F9; Mon, 8 Sep 2014 05:53:32 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A4C071904; Mon, 8 Sep 2014 05:53:31 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 32B0E1FE027; Mon, 8 Sep 2014 07:53:23 +0200 (CEST) Message-ID: <540D444C.6090904@selasky.org> Date: Mon, 08 Sep 2014 07:53:16 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Rick Macklem Subject: Re: [RFC] Patch to improve TSO limitation formula in general References: <1762951742.33012989.1409954952800.JavaMail.root@uoguelph.ca> In-Reply-To: <1762951742.33012989.1409954952800.JavaMail.root@uoguelph.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Eric Joyner , FreeBSD Current , Scott Long , Jack F Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 05:53:33 -0000 On 09/06/14 00:09, Rick Macklem wrote: > Hans Petter Selesky wrote: >> On 09/05/14 23:19, Eric Joyner wrote: >>> There are some concerns if we use this with devices that ixl >>> supports: >>> >>> - The maximum fragment size is 16KB-1, which isn't a power of 2. >>> >> >> Hi Eric, >> >> Multiplying by powers of two are more fast, than non-powers of two. >> So >> in this case you would have to use 8KB as a maximum. >> > Well, I'm no architecture expert, but I really doubt the CPU delay of a > non-power of 2 multiply/divide is significant related to doing smaller > TSO segments. Long ago (as in 1970s) I did work on machines where shifts > for power of 2 multiply/divide was preferable, but these days I doubt it > is going to matter?? > >>> - You can't get the maximum TSO size for ixl devices by multiplying >>> the >>> maximum number of fragments by the maximum size. >>> Instead the number of fragments is AFAIK unlimited, but a segment >>> can only >>> span 8 mbufs (including the [up to 3] mbufs containing the header), >>> and the >>> maximum TSO size is 256KB. Hi, Maybe that can be a separate parameter? I see that your patch assumes that a segment can be any-length. That is not always the case. Remember there are JUMBO mbufs too. With my patch, the maximum segment size is a separate parameter. The total number of TSO bytes is then not so useful. --HPS From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 08:00:09 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E19EF99 for ; Mon, 8 Sep 2014 08:00:09 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B5BEB14A8 for ; Mon, 8 Sep 2014 08:00:09 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s88809gM066738 for ; Mon, 8 Sep 2014 08:00:09 GMT (envelope-from bugzilla-noreply@freebsd.org) Message-Id: <201409080800.s88809gM066738@kenobi.freebsd.org> From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 08 Sep 2014 08:00:09 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 08:00:10 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (1 bugs) Bug 183659: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183659 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-net@FreeBSD.org Status: Needs MFC Resolution: Summary: [tcp] TCP stack lock contention with short-lived connections From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 12:05:47 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20C9016A; Mon, 8 Sep 2014 12:05:47 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id B7DFD1297; Mon, 8 Sep 2014 12:05:46 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArwEAMCaDVSDaFve/2dsb2JhbABZg2BXBIJ4xm4KhnlTAYEneIQDAQEBAwEBAQEgKx0DCwUWEgYCAg0EFQIpAQkYDgYIBwQBHASIGQgNpWqVGAEXgSyNSgYBAQEaNAcKgm+BUwWVcIN/hGKTTYFnHoF4IS8HgQAIFyKBBwEBAQ X-IronPort-AV: E=Sophos;i="5.04,486,1406606400"; d="scan'208";a="152309133" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 08 Sep 2014 08:05:40 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id BC45CB4035; Mon, 8 Sep 2014 08:05:39 -0400 (EDT) Date: Mon, 8 Sep 2014 08:05:39 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Message-ID: <1882852102.33387181.1410177939640.JavaMail.root@uoguelph.ca> In-Reply-To: <540D444C.6090904@selasky.org> Subject: Re: [RFC] Patch to improve TSO limitation formula in general MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org, Eric Joyner , FreeBSD Current , Scott Long , Jack F Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 12:05:47 -0000 Hans Petter Selasky wrote: > On 09/06/14 00:09, Rick Macklem wrote: > > Hans Petter Selesky wrote: > >> On 09/05/14 23:19, Eric Joyner wrote: > >>> There are some concerns if we use this with devices that ixl > >>> supports: > >>> > >>> - The maximum fragment size is 16KB-1, which isn't a power of 2. > >>> > >> > >> Hi Eric, > >> > >> Multiplying by powers of two are more fast, than non-powers of > >> two. > >> So > >> in this case you would have to use 8KB as a maximum. > >> > > Well, I'm no architecture expert, but I really doubt the CPU delay > > of a > > non-power of 2 multiply/divide is significant related to doing > > smaller > > TSO segments. Long ago (as in 1970s) I did work on machines where > > shifts > > for power of 2 multiply/divide was preferable, but these days I > > doubt it > > is going to matter?? > > > >>> - You can't get the maximum TSO size for ixl devices by > >>> multiplying > >>> the > >>> maximum number of fragments by the maximum size. > >>> Instead the number of fragments is AFAIK unlimited, but a segment > >>> can only > >>> span 8 mbufs (including the [up to 3] mbufs containing the > >>> header), > >>> and the > >>> maximum TSO size is 256KB. > > Hi, > > Maybe that can be a separate parameter? > > I see that your patch assumes that a segment can be any-length. That > is > not always the case. Remember there are JUMBO mbufs too. > I thought JUMBO mbufs were only going to be used on the receive side, however I suppose if a packet is received into a JUMBO mbuf and then forwarded on another interface... > With my patch, the maximum segment size is a separate parameter. The > total number of TSO bytes is then not so useful. > Well, if you are referring to if_hw_tsomax, I'm not sure it was the best plan to begin with. It was implemented for xen and I'm not sure that any other driver uses it as of now. However... I'm not a network/hardware guy, but it seems some devices do have the IP_MAXPACKET limit (use the ip_len field in the ip header to know how large the TSO segment is). There is also at least one device (82598 chip for "ix" driver) that can handle more that IP_MAXPACKET, so it seems appropriate to have a value that the driver can set. Since the maximum size of the gather list for transmit does seem to vary a lot between devices, with several handling less than 35, it does seem appropriate to allow drivers to specify that. If devices can't handle a single gather entry over a certain size, I think that does need to be specified along with the max size of the gather list, since the driver will need to use multiple gather entries for a single mbuf and tcp_output() should take that into account. In summary, yep, I basically agree. rick ps: Who will look at your patch soon. > --HPS > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 15:54:37 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C72F29A for ; Mon, 8 Sep 2014 15:54:37 +0000 (UTC) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5966F1084 for ; Mon, 8 Sep 2014 15:54:37 +0000 (UTC) Received: by mail-ie0-f177.google.com with SMTP id at20so5490502iec.22 for ; Mon, 08 Sep 2014 08:54:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Jmruc4JJ2dPg/s9MgQCiPU2MhYSiSv87M1uWgVviwxE=; b=uPi/mAaASKgiLgYFGpmKykRQsxQgevOMZ6RjwgKxPPDJB2VdnZoqthu1MRbcJozOWM 67ZZeMT64v0Knb9ia6FU34SvM5aMtFF1sOuW7aCGQGaxJM3w5TAFUTrNWrm3/xIZHSFo xqSIt74O6DBGn4SbJh31wUuCQerCupbZfa/62RzC6p1lPGcbTH6XibhIXVpuro/BvqII ePVndOEyoz+UG8X7phspxSL2Pw5tgOQFIUPWbKo+2y9Txpu1zi6whnp+FXUlyuY3yKVs kXIKQ6klZJAVu5wgRqQiPhUQTNbTYgHW0mHeSywVWJgfcF5SIO4SYMRKqEd4xlIAZ4UK zBFw== MIME-Version: 1.0 X-Received: by 10.50.88.98 with SMTP id bf2mr11107009igb.11.1410191676522; Mon, 08 Sep 2014 08:54:36 -0700 (PDT) Received: by 10.107.129.65 with HTTP; Mon, 8 Sep 2014 08:54:36 -0700 (PDT) In-Reply-To: References: Date: Mon, 8 Sep 2014 09:54:36 -0600 Message-ID: Subject: Re: help porting netmap to new driver From: David To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 15:54:37 -0000 Hi Sorry for the late response, I'll be using an ARM processor for a 1G link. My first issue came when doing what you suggested, and try the "emulated netmap mode". I need to cross compile the code from my x86, which I think I did correctly by setting up my arm toolchain and passing the kernel source when doing the "make". Now when I run the pkt-gen example I get : [ 57.602575] Unable to handle kernel paging request at virtual address f143ca04 [ 57.609740] pgd =3D 894c4000 [ 57.612420] [f143ca04] *pgd=3D00000000 [ 57.615973] Internal error: Oops: 5 [#1] PREEMPT SMP ARM Where I'm stuck since a while ago trying to figure it out. regards 2014-08-27 16:23 GMT-06:00 Luigi Rizzo : > > > > On Wed, Aug 27, 2014 at 3:18 PM, David wrote: > >> Hi, >> >> I'm needing to use netmap on a custom driver, I don't understand the >> content of the functions I need to implement and that are detailed on >> "PORTING" file. >> >> > =E2=80=8Bsometimes (often, actually) the hw has bottlenecks that make nat= ive > netmap mode almost useless. > =E2=80=8BOne thing you could try is the "emulated netmap mode" which > is used by default=E2=80=8B if the driver has no native support. > This is in the code on code.google.com/p/netmap/ , > the branch "next" (which is basically the code now in FreeBSD) > has some fixes for that specific feature > > cheers > luigi > > > can someone give a hand to understand it better? >> >> regards >> >> -- >> David D=C3=ADaz Barquero >> >> Ingenier=C3=ADa en Computadores >> Tecnol=C3=B3gico de Costa Rica >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2211611 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > --=20 David D=C3=ADaz Barquero Ingenier=C3=ADa en Computadores Tecnol=C3=B3gico de Costa Rica From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 16:12:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 57996BAD for ; Mon, 8 Sep 2014 16:12:36 +0000 (UTC) Received: from sdf.lonestar.org (mx.sdf.org [192.94.73.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx.sdf.org", Issuer "SDF.ORG" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A8201346 for ; Mon, 8 Sep 2014 16:12:35 +0000 (UTC) Received: from otaku.freeshell.org (IDENT:case@otaku.freeshell.org [192.94.73.9]) by sdf.lonestar.org (8.14.8/8.14.5) with ESMTP id s88GCV1u018223 (using TLSv1/SSLv3 with cipher DHE-RSA-AES256-SHA (256 bits) verified NO) for ; Mon, 8 Sep 2014 16:12:31 GMT Date: Mon, 8 Sep 2014 16:12:31 +0000 (UTC) From: John Case X-X-Sender: case@faeroes.freeshell.org To: freebsd-net@freebsd.org Subject: RE: How can sshuttle be used properly with FreeBSD (and with DNS) ? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 16:12:36 -0000 Hi Ryan, Thanks for responding. Just for the record, I removed my natd and ipdivert lines, so that sshuttles divert rules were the only rules on the system ... I made my system work without my own natd/divert by putting some static route definitions into rc.conf. Anyway, it still worked fine for tcp over the ssh tunnel, but it didn't help the UDP tunneling, which supports your conclusion. What is the solution here ? Or more importantly, what is even the problem? sshuttle documentation (the readme) makes some vague references to FreeBSD not handling forwarding of UDP properly, which is why the diverts for it go into place at all ... Do we solve this problem by fixing sshuttle (perhaps putting in more complex ipfw rules for it to inject) ? Or do we solve this problem by fixing FreeBSD, and making forwarding "work" with UDP properly ? It doesn't work at all now, but I'd like to at least get a sense as to what the real problem to solve here is ... Thanks. From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 18:39:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E70DE2C3; Mon, 8 Sep 2014 18:39:04 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACDBB1846; Mon, 8 Sep 2014 18:39:04 +0000 (UTC) Received: by mail-pd0-f174.google.com with SMTP id v10so7234062pde.5 for ; Mon, 08 Sep 2014 11:39:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=vQM7Ce0IQjlutRqPsTzpxWD7jX5qcKeNMGeqjeONxBQ=; b=CULfiVemlcfdDoSAinadVl2sGQF4+lTifSMzrlGvD/5yWDzbRcQ/5x2xW1JCNPlq4K NPWeXHD3yCg+0VOq93al4eexdMTRU3YF/YllYKEy+7up6h6iNKxcqpXaV2OFJwBqDnRM 7zjrl4+6iYsXImp12nEU0Ng1OAL67pVtAaX/pmMDUizm9CPZhuaucHiJ54oSoS7noEIA ZzX9f/BKJnxlzP9mHyAJruIUXg76pYvnHX9BRdEd7U24PNBSghNofJyN60ZaloR7GIaY EDR3XBXo3drmOrk3vmgAP8kfznmdwc+L9V5s/MYDjaRAtY/GjMFdxbcmM+9sla7fsLdH 5e7A== X-Received: by 10.68.196.226 with SMTP id ip2mr22582448pbc.120.1410201543838; Mon, 08 Sep 2014 11:39:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.70.55.162 with HTTP; Mon, 8 Sep 2014 11:38:23 -0700 (PDT) In-Reply-To: <540A8200.5010404@bsdinfo.com.br> References: <5408F23C.2030309@bsdinfo.com.br> <54091607.9010100@bsdinfo.com.br> <5409CA44.8070203@bsdinfo.com.br> <540A1A3E.2040306@bsdinfo.com.br> <540A8200.5010404@bsdinfo.com.br> From: Eric Joyner Date: Mon, 8 Sep 2014 11:38:23 -0700 Message-ID: Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! To: Marcelo Gondim Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Jack F Vogel , FreeBSD Net , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 18:39:05 -0000 Getting local / remote faults is strange; are these stats from after you rebooted? --- - Eric Joyner On Fri, Sep 5, 2014 at 8:39 PM, Marcelo Gondim wrote: > On 05/09/2014 17:17, Marcelo Gondim wrote: > >> On 05/09/2014 16:49, Adrian Chadd wrote: >> >>> Hi, >>> >>> But is the airflow in the unit sufficient? >>> >>> I had this problem at a previous job - the box was running fine, the >>> room was very cold, but the internal fans in the server were set to >>> "be very quiet". It wasn't enough to keep the ixgbe NICs happy. I had >>> to change the fan settings to "just always run full speed". >>> >>> The fan temperature feedback loop was based on sensors on the CPU, >>> _not_ on the peripherals. >>> >> Hi Adrian, >> >> Ummm. I'll check it and improve internal cooling. :) >> She is not happy and I'm also not. rsrsrsr >> >> Cheers, >> > > Besides the problem of heating of the network interface, I am putting some > information here. Could you tell me if there is something strange or is it > normal? > > dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - > 2.5.15 > dev.ix.0.%driver: ix > dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0 > dev.ix.0.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 > subdevice=0x7b11 class=0x020000 > dev.ix.0.%parent: pci131 > dev.ix.0.fc: 3 > dev.ix.0.enable_aim: 1 > dev.ix.0.advertise_speed: 0 > dev.ix.0.dropped: 0 > dev.ix.0.mbuf_defrag_failed: 0 > dev.ix.0.watchdog_events: 0 > dev.ix.0.link_irq: 121769 > dev.ix.0.queue0.interrupt_rate: 5319 > dev.ix.0.queue0.irqs: 7900830877 > dev.ix.0.queue0.txd_head: 1037 > dev.ix.0.queue0.txd_tail: 1037 > dev.ix.0.queue0.tso_tx: 142 > dev.ix.0.queue0.no_tx_dma_setup: 0 > dev.ix.0.queue0.no_desc_avail: 0 > dev.ix.0.queue0.tx_packets: 9725701450 > dev.ix.0.queue0.rxd_head: 1175 > dev.ix.0.queue0.rxd_tail: 1174 > dev.ix.0.queue0.rx_packets: 13069276955 > dev.ix.0.queue0.rx_bytes: 3391061018 > dev.ix.0.queue0.rx_copies: 8574407 > dev.ix.0.queue0.lro_queued: 0 > dev.ix.0.queue0.lro_flushed: 0 > dev.ix.0.queue1.interrupt_rate: 41666 > dev.ix.0.queue1.irqs: 7681141208 > dev.ix.0.queue1.txd_head: 219 > dev.ix.0.queue1.txd_tail: 221 > dev.ix.0.queue1.tso_tx: 57 > dev.ix.0.queue1.no_tx_dma_setup: 0 > dev.ix.0.queue1.no_desc_avail: 44334 > dev.ix.0.queue1.tx_packets: 10196891433 > dev.ix.0.queue1.rxd_head: 1988 > dev.ix.0.queue1.rxd_tail: 1987 > dev.ix.0.queue1.rx_packets: 13210132242 > dev.ix.0.queue1.rx_bytes: 4317357059 > dev.ix.0.queue1.rx_copies: 8131936 > dev.ix.0.queue1.lro_queued: 0 > dev.ix.0.queue1.lro_flushed: 0 > dev.ix.0.queue2.interrupt_rate: 5319 > dev.ix.0.queue2.irqs: 7647486080 > dev.ix.0.queue2.txd_head: 761 > dev.ix.0.queue2.txd_tail: 761 > dev.ix.0.queue2.tso_tx: 409 > dev.ix.0.queue2.no_tx_dma_setup: 0 > dev.ix.0.queue2.no_desc_avail: 54207 > dev.ix.0.queue2.tx_packets: 10161246425 > dev.ix.0.queue2.rxd_head: 1874 > dev.ix.0.queue2.rxd_tail: 1872 > dev.ix.0.queue2.rx_packets: 13175551880 > dev.ix.0.queue2.rx_bytes: 4472798418 > dev.ix.0.queue2.rx_copies: 7488876 > dev.ix.0.queue2.lro_queued: 0 > dev.ix.0.queue2.lro_flushed: 0 > dev.ix.0.queue3.interrupt_rate: 500000 > dev.ix.0.queue3.irqs: 7641129521 > dev.ix.0.queue3.txd_head: 2039 > dev.ix.0.queue3.txd_tail: 2039 > dev.ix.0.queue3.tso_tx: 9 > dev.ix.0.queue3.no_tx_dma_setup: 0 > dev.ix.0.queue3.no_desc_avail: 150346 > dev.ix.0.queue3.tx_packets: 10619971896 > dev.ix.0.queue3.rxd_head: 1055 > dev.ix.0.queue3.rxd_tail: 1054 > dev.ix.0.queue3.rx_packets: 13137835529 > dev.ix.0.queue3.rx_bytes: 4063197306 > dev.ix.0.queue3.rx_copies: 8188713 > dev.ix.0.queue3.lro_queued: 0 > dev.ix.0.queue3.lro_flushed: 0 > dev.ix.0.queue4.interrupt_rate: 5319 > dev.ix.0.queue4.irqs: 7439824996 > dev.ix.0.queue4.txd_head: 26 > dev.ix.0.queue4.txd_tail: 26 > dev.ix.0.queue4.tso_tx: 553912 > dev.ix.0.queue4.no_tx_dma_setup: 0 > dev.ix.0.queue4.no_desc_avail: 0 > dev.ix.0.queue4.tx_packets: 10658683718 > dev.ix.0.queue4.rxd_head: 684 > dev.ix.0.queue4.rxd_tail: 681 > dev.ix.0.queue4.rx_packets: 13204786830 > dev.ix.0.queue4.rx_bytes: 3700845239 > dev.ix.0.queue4.rx_copies: 8193379 > dev.ix.0.queue4.lro_queued: 0 > dev.ix.0.queue4.lro_flushed: 0 > dev.ix.0.queue5.interrupt_rate: 15151 > dev.ix.0.queue5.irqs: 7456613396 > dev.ix.0.queue5.txd_head: 603 > dev.ix.0.queue5.txd_tail: 603 > dev.ix.0.queue5.tso_tx: 17 > dev.ix.0.queue5.no_tx_dma_setup: 0 > dev.ix.0.queue5.no_desc_avail: 0 > dev.ix.0.queue5.tx_packets: 10639139790 > dev.ix.0.queue5.rxd_head: 404 > dev.ix.0.queue5.rxd_tail: 403 > dev.ix.0.queue5.rx_packets: 13144301293 > dev.ix.0.queue5.rx_bytes: 3986784766 > dev.ix.0.queue5.rx_copies: 8256195 > dev.ix.0.queue5.lro_queued: 0 > dev.ix.0.queue5.lro_flushed: 0 > dev.ix.0.queue6.interrupt_rate: 125000 > dev.ix.0.queue6.irqs: 7466940576 > dev.ix.0.queue6.txd_head: 1784 > dev.ix.0.queue6.txd_tail: 1784 > dev.ix.0.queue6.tso_tx: 2001 > dev.ix.0.queue6.no_tx_dma_setup: 0 > dev.ix.0.queue6.no_desc_avail: 0 > dev.ix.0.queue6.tx_packets: 9784312967 > dev.ix.0.queue6.rxd_head: 395 > dev.ix.0.queue6.rxd_tail: 394 > dev.ix.0.queue6.rx_packets: 13103079970 > dev.ix.0.queue6.rx_bytes: 3581485264 > dev.ix.0.queue6.rx_copies: 7336569 > dev.ix.0.queue6.lro_queued: 0 > dev.ix.0.queue6.lro_flushed: 0 > dev.ix.0.queue7.interrupt_rate: 5319 > dev.ix.0.queue7.irqs: 7486391989 > dev.ix.0.queue7.txd_head: 1549 > dev.ix.0.queue7.txd_tail: 1549 > dev.ix.0.queue7.tso_tx: 2052 > dev.ix.0.queue7.no_tx_dma_setup: 0 > dev.ix.0.queue7.no_desc_avail: 0 > dev.ix.0.queue7.tx_packets: 9758335509 > dev.ix.0.queue7.rxd_head: 1990 > dev.ix.0.queue7.rxd_tail: 1986 > dev.ix.0.queue7.rx_packets: 13141436559 > dev.ix.0.queue7.rx_bytes: 3877881546 > dev.ix.0.queue7.rx_copies: 8505244 > dev.ix.0.queue7.lro_queued: 0 > dev.ix.0.queue7.lro_flushed: 0 > dev.ix.0.mac_stats.crc_errs: 159 > dev.ix.0.mac_stats.ill_errs: 5 > dev.ix.0.mac_stats.byte_errs: 10 > dev.ix.0.mac_stats.short_discards: 0 > dev.ix.0.mac_stats.local_faults: 213 > dev.ix.0.mac_stats.remote_faults: 37 > dev.ix.0.mac_stats.rec_len_errs: 0 > dev.ix.0.mac_stats.xon_txd: 33623392288 > dev.ix.0.mac_stats.xon_recvd: 0 > dev.ix.0.mac_stats.xoff_txd: 110161 > dev.ix.0.mac_stats.xoff_recvd: 0 > dev.ix.0.mac_stats.total_octets_rcvd: 37967032023930 > dev.ix.0.mac_stats.good_octets_rcvd: 37965112076546 > dev.ix.0.mac_stats.total_pkts_rcvd: 105201314224 > dev.ix.0.mac_stats.good_pkts_rcvd: 18446743122750685300 > dev.ix.0.mac_stats.mcast_pkts_rcvd: 43835 > dev.ix.0.mac_stats.bcast_pkts_rcvd: 960566 > dev.ix.0.mac_stats.rx_frames_64: 454067 > dev.ix.0.mac_stats.rx_frames_65_127: 75350664570 > dev.ix.0.mac_stats.rx_frames_128_255: 4726321485 > dev.ix.0.mac_stats.rx_frames_256_511: 2282382994 > dev.ix.0.mac_stats.rx_frames_512_1023: 3293330145 > dev.ix.0.mac_stats.rx_frames_1024_1522: 19541190234 > dev.ix.0.mac_stats.recv_undersized: 0 > dev.ix.0.mac_stats.recv_fragmented: 0 > dev.ix.0.mac_stats.recv_oversized: 0 > dev.ix.0.mac_stats.recv_jabberd: 4 > dev.ix.0.mac_stats.management_pkts_rcvd: 0 > dev.ix.0.mac_stats.management_pkts_drpd: 0 > dev.ix.0.mac_stats.checksum_errs: 527716650 > dev.ix.0.mac_stats.good_octets_txd: 81355567084088 > dev.ix.0.mac_stats.total_pkts_txd: 81545370171 > dev.ix.0.mac_stats.good_pkts_txd: 47921867720 > dev.ix.0.mac_stats.bcast_pkts_txd: 298646 > dev.ix.0.mac_stats.mcast_pkts_txd: 18446744040086169062 > dev.ix.0.mac_stats.management_pkts_txd: 0 > dev.ix.0.mac_stats.tx_frames_64: 18446744045326220993 > dev.ix.0.mac_stats.tx_frames_65_127: 15629697019 > dev.ix.0.mac_stats.tx_frames_128_255: 4141756940 > dev.ix.0.mac_stats.tx_frames_256_511: 2195917491 > dev.ix.0.mac_stats.tx_frames_512_1023: 2717769904 > dev.ix.0.mac_stats.tx_frames_1024_1522: 51620056990 > > dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - > 2.5.15 > dev.ix.1.%driver: ix > dev.ix.1.%location: slot=0 function=1 handle=\_SB_.PCI1.BR48.S3F1 > dev.ix.1.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 > subdevice=0x7b11 class=0x020000 > dev.ix.1.%parent: pci131 > dev.ix.1.fc: 3 > dev.ix.1.enable_aim: 1 > dev.ix.1.advertise_speed: 0 > dev.ix.1.dropped: 0 > dev.ix.1.mbuf_defrag_failed: 0 > dev.ix.1.watchdog_events: 0 > dev.ix.1.link_irq: 46692 > dev.ix.1.queue0.interrupt_rate: 15151 > dev.ix.1.queue0.irqs: 2634341620 > dev.ix.1.queue0.txd_head: 1101 > dev.ix.1.queue0.txd_tail: 1101 > dev.ix.1.queue0.tso_tx: 0 > dev.ix.1.queue0.no_tx_dma_setup: 0 > dev.ix.1.queue0.no_desc_avail: 4142 > dev.ix.1.queue0.tx_packets: 4749431326 > dev.ix.1.queue0.rxd_head: 812 > dev.ix.1.queue0.rxd_tail: 811 > dev.ix.1.queue0.rx_packets: 21135516 > dev.ix.1.queue0.rx_bytes: 33229682 > dev.ix.1.queue0.rx_copies: 119434 > dev.ix.1.queue0.lro_queued: 0 > dev.ix.1.queue0.lro_flushed: 0 > dev.ix.1.queue1.interrupt_rate: 15151 > dev.ix.1.queue1.irqs: 2616347302 > dev.ix.1.queue1.txd_head: 797 > dev.ix.1.queue1.txd_tail: 797 > dev.ix.1.queue1.tso_tx: 0 > dev.ix.1.queue1.no_tx_dma_setup: 0 > dev.ix.1.queue1.no_desc_avail: 4129 > dev.ix.1.queue1.tx_packets: 4730356550 > dev.ix.1.queue1.rxd_head: 1345 > dev.ix.1.queue1.rxd_tail: 1344 > dev.ix.1.queue1.rx_packets: 20844457 > dev.ix.1.queue1.rx_bytes: 35014827 > dev.ix.1.queue1.rx_copies: 125167 > dev.ix.1.queue1.lro_queued: 0 > dev.ix.1.queue1.lro_flushed: 0 > dev.ix.1.queue2.interrupt_rate: 5555 > dev.ix.1.queue2.irqs: 2603166062 > dev.ix.1.queue2.txd_head: 1428 > dev.ix.1.queue2.txd_tail: 1432 > dev.ix.1.queue2.tso_tx: 0 > dev.ix.1.queue2.no_tx_dma_setup: 0 > dev.ix.1.queue2.no_desc_avail: 34368 > dev.ix.1.queue2.tx_packets: 4681339909 > dev.ix.1.queue2.rxd_head: 1153 > dev.ix.1.queue2.rxd_tail: 1152 > dev.ix.1.queue2.rx_packets: 21263862 > dev.ix.1.queue2.rx_bytes: 46251419 > dev.ix.1.queue2.rx_copies: 125156 > dev.ix.1.queue2.lro_queued: 0 > dev.ix.1.queue2.lro_flushed: 0 > dev.ix.1.queue3.interrupt_rate: 5319 > dev.ix.1.queue3.irqs: 2658483423 > dev.ix.1.queue3.txd_head: 179 > dev.ix.1.queue3.txd_tail: 179 > dev.ix.1.queue3.tso_tx: 0 > dev.ix.1.queue3.no_tx_dma_setup: 0 > dev.ix.1.queue3.no_desc_avail: 4189 > dev.ix.1.queue3.tx_packets: 4811705668 > dev.ix.1.queue3.rxd_head: 583 > dev.ix.1.queue3.rxd_tail: 582 > dev.ix.1.queue3.rx_packets: 20472051 > dev.ix.1.queue3.rx_bytes: 29823943 > dev.ix.1.queue3.rx_copies: 129450 > dev.ix.1.queue3.lro_queued: 0 > dev.ix.1.queue3.lro_flushed: 0 > dev.ix.1.queue4.interrupt_rate: 5319 > dev.ix.1.queue4.irqs: 2629295642 > dev.ix.1.queue4.txd_head: 592 > dev.ix.1.queue4.txd_tail: 594 > dev.ix.1.queue4.tso_tx: 0 > dev.ix.1.queue4.no_tx_dma_setup: 0 > dev.ix.1.queue4.no_desc_avail: 4106 > dev.ix.1.queue4.tx_packets: 4774679355 > dev.ix.1.queue4.rxd_head: 259 > dev.ix.1.queue4.rxd_tail: 258 > dev.ix.1.queue4.rx_packets: 22561633 > dev.ix.1.queue4.rx_bytes: 40196457 > dev.ix.1.queue4.rx_copies: 135966 > dev.ix.1.queue4.lro_queued: 0 > dev.ix.1.queue4.lro_flushed: 0 > dev.ix.1.queue5.interrupt_rate: 18518 > dev.ix.1.queue5.irqs: 2628129098 > dev.ix.1.queue5.txd_head: 1070 > dev.ix.1.queue5.txd_tail: 1072 > dev.ix.1.queue5.tso_tx: 0 > dev.ix.1.queue5.no_tx_dma_setup: 0 > dev.ix.1.queue5.no_desc_avail: 4127 > dev.ix.1.queue5.tx_packets: 4750613588 > dev.ix.1.queue5.rxd_head: 595 > dev.ix.1.queue5.rxd_tail: 594 > dev.ix.1.queue5.rx_packets: 21309940 > dev.ix.1.queue5.rx_bytes: 29264004 > dev.ix.1.queue5.rx_copies: 134128 > dev.ix.1.queue5.lro_queued: 0 > dev.ix.1.queue5.lro_flushed: 0 > dev.ix.1.queue6.interrupt_rate: 125000 > dev.ix.1.queue6.irqs: 2615444123 > dev.ix.1.queue6.txd_head: 1008 > dev.ix.1.queue6.txd_tail: 1008 > dev.ix.1.queue6.tso_tx: 0 > dev.ix.1.queue6.no_tx_dma_setup: 0 > dev.ix.1.queue6.no_desc_avail: 4106 > dev.ix.1.queue6.tx_packets: 4703519800 > dev.ix.1.queue6.rxd_head: 1209 > dev.ix.1.queue6.rxd_tail: 1208 > dev.ix.1.queue6.rx_packets: 22783319 > dev.ix.1.queue6.rx_bytes: 28091721 > dev.ix.1.queue6.rx_copies: 131653 > dev.ix.1.queue6.lro_queued: 0 > dev.ix.1.queue6.lro_flushed: 0 > dev.ix.1.queue7.interrupt_rate: 6024 > dev.ix.1.queue7.irqs: 2644640254 > dev.ix.1.queue7.txd_head: 1313 > dev.ix.1.queue7.txd_tail: 1313 > dev.ix.1.queue7.tso_tx: 0 > dev.ix.1.queue7.no_tx_dma_setup: 0 > dev.ix.1.queue7.no_desc_avail: 4106 > dev.ix.1.queue7.tx_packets: 4824584039 > dev.ix.1.queue7.rxd_head: 1463 > dev.ix.1.queue7.rxd_tail: 1462 > dev.ix.1.queue7.rx_packets: 21464418 > dev.ix.1.queue7.rx_bytes: 38133966 > dev.ix.1.queue7.rx_copies: 163180 > dev.ix.1.queue7.lro_queued: 0 > dev.ix.1.queue7.lro_flushed: 0 > dev.ix.1.mac_stats.crc_errs: 0 > dev.ix.1.mac_stats.ill_errs: 0 > dev.ix.1.mac_stats.byte_errs: 0 > dev.ix.1.mac_stats.short_discards: 0 > dev.ix.1.mac_stats.local_faults: 0 > dev.ix.1.mac_stats.remote_faults: 2 > dev.ix.1.mac_stats.rec_len_errs: 0 > dev.ix.1.mac_stats.xon_txd: 11207820571 > dev.ix.1.mac_stats.xon_recvd: 0 > dev.ix.1.mac_stats.xoff_txd: 3735958948 > dev.ix.1.mac_stats.xoff_recvd: 0 > dev.ix.1.mac_stats.total_octets_rcvd: 47828274751 > dev.ix.1.mac_stats.good_octets_rcvd: 47225376551 > dev.ix.1.mac_stats.total_pkts_rcvd: 177931572 > dev.ix.1.mac_stats.good_pkts_rcvd: 18446743954331576567 > dev.ix.1.mac_stats.mcast_pkts_rcvd: 7410 > dev.ix.1.mac_stats.bcast_pkts_rcvd: 8907 > dev.ix.1.mac_stats.rx_frames_64: 0 > dev.ix.1.mac_stats.rx_frames_65_127: 134619798 > dev.ix.1.mac_stats.rx_frames_128_255: 7996096 > dev.ix.1.mac_stats.rx_frames_256_511: 3996945 > dev.ix.1.mac_stats.rx_frames_512_1023: 5145938 > dev.ix.1.mac_stats.rx_frames_1024_1522: 21331154 > dev.ix.1.mac_stats.recv_undersized: 0 > dev.ix.1.mac_stats.recv_fragmented: 0 > dev.ix.1.mac_stats.recv_oversized: 0 > dev.ix.1.mac_stats.recv_jabberd: 0 > dev.ix.1.mac_stats.management_pkts_rcvd: 0 > dev.ix.1.mac_stats.management_pkts_drpd: 0 > dev.ix.1.mac_stats.checksum_errs: 219681 > dev.ix.1.mac_stats.good_octets_txd: 38874487735713 > dev.ix.1.mac_stats.total_pkts_txd: 38026147390 > dev.ix.1.mac_stats.good_pkts_txd: 27377335167 > dev.ix.1.mac_stats.bcast_pkts_txd: 67411 > dev.ix.1.mac_stats.mcast_pkts_txd: 18446744063060778699 > dev.ix.1.mac_stats.management_pkts_txd: 0 > dev.ix.1.mac_stats.tx_frames_64: 18446744065500352065 > dev.ix.1.mac_stats.tx_frames_65_127: 6755049420 > dev.ix.1.mac_stats.tx_frames_128_255: 1914523761 > dev.ix.1.mac_stats.tx_frames_256_511: 1038553757 > dev.ix.1.mac_stats.tx_frames_512_1023: 1240890942 > dev.ix.1.mac_stats.tx_frames_1024_1522: 24637516838 > > > Cheers, > > >> >>> >>> -a >>> >>> >>> On 5 September 2014 07:35, Marcelo Gondim wrote: >>> >>>> Hi Adrian, >>>> >>>> I confirmed with the support staff of the room where the server is, >>>> that the >>>> ambient temperature was normal. >>>> >>>> >>>> On 04/09/2014 22:46, Marcelo Gondim wrote: >>>> >>>>> On 04/09/2014 20:48, Adrian Chadd wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> The only time this has happened to me is because the card overheated. >>>>>> Can you check that? >>>>>> >>>>> Hi Adrian, >>>>> >>>>> The room where the equipment is located is very cold but I'll check it >>>>> out. >>>>> Also seen at the time of the problem, a lot of dropped packets. >>>>> >>>>> # netstat -idn >>>>> ... >>>>> ix0 1500 a0:36:9f:2a:6d:ac 18446743423829095869 159 >>>>> 750924631703 53285910688 0 0 0 >>>>> ix0 - fe80::a236:9f fe80::a236:9fff:f 0 - - 2 >>>>> - - - >>>>> ix1 1500 a0:36:9f:2a:6d:ae 18446743954328745465 0 >>>>> 119550050209 20178077451 0 0 0 >>>>> ix1 - fe80::a236:9f fe80::a236:9fff:f 0 - - 1 >>>>> - - - >>>>> ... >>>>> >>>>> 119550050209 droped packets on ix1 and 750924631703 droped on ix0 >>>>> >>>>> Could be interesting I upgrade to10.1-PRERELEASE? >>>>> Could there be a problem with the driver? >>>>> >>>>> Traffic on ix0: 1.4Gbps output / 600Mbps input >>>>> Traffic on ix1: 1.2Gbps output >>>>> >>>>> PPS on ix0: 163Kpps output / 215Kpps input >>>>> PPS on ix1: 131Kpps output >>>>> >>>>> Thanks for your help. >>>>> >>>>>> >>>>>> >>>>>> -a >>>>>> >>>>>> >>>>>> On 4 September 2014 16:14, Marcelo Gondim >>>>>> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> I have an Intel X520-SR2and today was working when all traffic >>>>>>> stopped. >>>>>>> I looked in the logs and found this message: >>>>>>> >>>>>>> Sep 4 18:29:53 rt01 kernel: ix1: >>>>>>> Sep 4 18:29:53 rt01 kernel: CRITICAL: ECC ERROR!! Please Reboot!! >>>>>>> >>>>>>> # uname -a >>>>>>> FreeBSD rt01.xxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #10 >>>>>>> r267839: >>>>>>> Thu >>>>>>> Jul 10 15:35:04 BRT 2014 >>>>>>> root@rt01.xxxxx.com.br:/usr/obj/usr/src/sys/GONDIM10 amd64 >>>>>>> >>>>>>> # netstat -m >>>>>>> 98324/53476/151800 mbufs in use (current/cache/total) >>>>>>> 98301/44951/143252/1014370 mbuf clusters in use >>>>>>> (current/cache/total/max) >>>>>>> 98301/44897 mbuf+clusters out of packet secondary zone in use >>>>>>> (current/cache) >>>>>>> 0/421/421/507184 4k (page size) jumbo clusters in use >>>>>>> (current/cache/total/max) >>>>>>> 0/0/0/150276 9k jumbo clusters in use (current/cache/total/max) >>>>>>> 0/0/0/84530 16k jumbo clusters in use (current/cache/total/max) >>>>>>> 221183K/104955K/326138K bytes allocated to network >>>>>>> (current/cache/total) >>>>>>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >>>>>>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >>>>>>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >>>>>>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >>>>>>> 0 requests for sfbufs denied >>>>>>> 0 requests for sfbufs delayed >>>>>>> 0 requests for I/O initiated by sendfile >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>> > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 18:52:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAD39A7E for ; Mon, 8 Sep 2014 18:52:33 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE0771A1D for ; Mon, 8 Sep 2014 18:52:32 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id A6984139CB for ; Mon, 8 Sep 2014 15:53:06 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-type:content-type:in-reply-to:references:subject :subject:to:mime-version:user-agent:from:from:date:date :message-id; s=dkim; t=1410202384; x=1411066385; bh=YBvgccV1KRof I+3gFrH4VmDQ1HkY8X1v0ZNV956Z6ck=; b=Kl3+G3UQgnj1F72M98+ry7yzirSV nnFEC9kOCOpNiZNCfu4h38dCzAq/zMrPkiy94xvOtEBlsAnNHs0XTirl0r5h5Iwl dy6Iw1RYY/ht5to/OIpm/XAz0iIaXbYWhOUaujOwoHbjqRP2GeZQ0Zolio4VCpIg 2bM6KQ/WNxFLOic= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VaAp1zs40Uea for ; Mon, 8 Sep 2014 15:53:04 -0300 (BRT) Received: from [192.168.88.15] (unknown [186.193.48.8]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id D2D7C139CA; Mon, 8 Sep 2014 15:53:02 -0300 (BRT) Message-ID: <540DFAE3.8070001@bsdinfo.com.br> Date: Mon, 08 Sep 2014 15:52:19 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: Eric Joyner Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! References: <5408F23C.2030309@bsdinfo.com.br> <54091607.9010100@bsdinfo.com.br> <5409CA44.8070203@bsdinfo.com.br> <540A1A3E.2040306@bsdinfo.com.br> <540A8200.5010404@bsdinfo.com.br> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Jack F Vogel , FreeBSD Net , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 18:52:34 -0000 On 08/09/2014 15:38, Eric Joyner wrote: > Getting local / remote faults is strange; are these stats from after you > rebooted? When the crash happens I just do: # ifconfig ix0 down; ifconfig ix0 up; ifconfig ix1 down; ifconfig up ixi1 After that the two interface ports return to work. In no time I had to restart the server. # uptime 3:51PM up 11 days, 15:44, 2 users, load averages: 5.36, 5.68, 5.69 Tomorrow will be going to the Datacenter to check on the temperature of the network interface. As the crashes occur when traffic is high, I believe that the temperature increases because processing is high. > > --- > - Eric Joyner > > On Fri, Sep 5, 2014 at 8:39 PM, Marcelo Gondim > wrote: > >> On 05/09/2014 17:17, Marcelo Gondim wrote: >> >>> On 05/09/2014 16:49, Adrian Chadd wrote: >>> >>>> Hi, >>>> >>>> But is the airflow in the unit sufficient? >>>> >>>> I had this problem at a previous job - the box was running fine, the >>>> room was very cold, but the internal fans in the server were set to >>>> "be very quiet". It wasn't enough to keep the ixgbe NICs happy. I had >>>> to change the fan settings to "just always run full speed". >>>> >>>> The fan temperature feedback loop was based on sensors on the CPU, >>>> _not_ on the peripherals. >>>> >>> Hi Adrian, >>> >>> Ummm. I'll check it and improve internal cooling. :) >>> She is not happy and I'm also not. rsrsrsr >>> >>> Cheers, >>> >> Besides the problem of heating of the network interface, I am putting some >> information here. Could you tell me if there is something strange or is it >> normal? >> >> dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - >> 2.5.15 >> dev.ix.0.%driver: ix >> dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0 >> dev.ix.0.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 >> subdevice=0x7b11 class=0x020000 >> dev.ix.0.%parent: pci131 >> dev.ix.0.fc: 3 >> dev.ix.0.enable_aim: 1 >> dev.ix.0.advertise_speed: 0 >> dev.ix.0.dropped: 0 >> dev.ix.0.mbuf_defrag_failed: 0 >> dev.ix.0.watchdog_events: 0 >> dev.ix.0.link_irq: 121769 >> dev.ix.0.queue0.interrupt_rate: 5319 >> dev.ix.0.queue0.irqs: 7900830877 >> dev.ix.0.queue0.txd_head: 1037 >> dev.ix.0.queue0.txd_tail: 1037 >> dev.ix.0.queue0.tso_tx: 142 >> dev.ix.0.queue0.no_tx_dma_setup: 0 >> dev.ix.0.queue0.no_desc_avail: 0 >> dev.ix.0.queue0.tx_packets: 9725701450 >> dev.ix.0.queue0.rxd_head: 1175 >> dev.ix.0.queue0.rxd_tail: 1174 >> dev.ix.0.queue0.rx_packets: 13069276955 >> dev.ix.0.queue0.rx_bytes: 3391061018 >> dev.ix.0.queue0.rx_copies: 8574407 >> dev.ix.0.queue0.lro_queued: 0 >> dev.ix.0.queue0.lro_flushed: 0 >> dev.ix.0.queue1.interrupt_rate: 41666 >> dev.ix.0.queue1.irqs: 7681141208 >> dev.ix.0.queue1.txd_head: 219 >> dev.ix.0.queue1.txd_tail: 221 >> dev.ix.0.queue1.tso_tx: 57 >> dev.ix.0.queue1.no_tx_dma_setup: 0 >> dev.ix.0.queue1.no_desc_avail: 44334 >> dev.ix.0.queue1.tx_packets: 10196891433 >> dev.ix.0.queue1.rxd_head: 1988 >> dev.ix.0.queue1.rxd_tail: 1987 >> dev.ix.0.queue1.rx_packets: 13210132242 >> dev.ix.0.queue1.rx_bytes: 4317357059 >> dev.ix.0.queue1.rx_copies: 8131936 >> dev.ix.0.queue1.lro_queued: 0 >> dev.ix.0.queue1.lro_flushed: 0 >> dev.ix.0.queue2.interrupt_rate: 5319 >> dev.ix.0.queue2.irqs: 7647486080 >> dev.ix.0.queue2.txd_head: 761 >> dev.ix.0.queue2.txd_tail: 761 >> dev.ix.0.queue2.tso_tx: 409 >> dev.ix.0.queue2.no_tx_dma_setup: 0 >> dev.ix.0.queue2.no_desc_avail: 54207 >> dev.ix.0.queue2.tx_packets: 10161246425 >> dev.ix.0.queue2.rxd_head: 1874 >> dev.ix.0.queue2.rxd_tail: 1872 >> dev.ix.0.queue2.rx_packets: 13175551880 >> dev.ix.0.queue2.rx_bytes: 4472798418 >> dev.ix.0.queue2.rx_copies: 7488876 >> dev.ix.0.queue2.lro_queued: 0 >> dev.ix.0.queue2.lro_flushed: 0 >> dev.ix.0.queue3.interrupt_rate: 500000 >> dev.ix.0.queue3.irqs: 7641129521 >> dev.ix.0.queue3.txd_head: 2039 >> dev.ix.0.queue3.txd_tail: 2039 >> dev.ix.0.queue3.tso_tx: 9 >> dev.ix.0.queue3.no_tx_dma_setup: 0 >> dev.ix.0.queue3.no_desc_avail: 150346 >> dev.ix.0.queue3.tx_packets: 10619971896 >> dev.ix.0.queue3.rxd_head: 1055 >> dev.ix.0.queue3.rxd_tail: 1054 >> dev.ix.0.queue3.rx_packets: 13137835529 >> dev.ix.0.queue3.rx_bytes: 4063197306 >> dev.ix.0.queue3.rx_copies: 8188713 >> dev.ix.0.queue3.lro_queued: 0 >> dev.ix.0.queue3.lro_flushed: 0 >> dev.ix.0.queue4.interrupt_rate: 5319 >> dev.ix.0.queue4.irqs: 7439824996 >> dev.ix.0.queue4.txd_head: 26 >> dev.ix.0.queue4.txd_tail: 26 >> dev.ix.0.queue4.tso_tx: 553912 >> dev.ix.0.queue4.no_tx_dma_setup: 0 >> dev.ix.0.queue4.no_desc_avail: 0 >> dev.ix.0.queue4.tx_packets: 10658683718 >> dev.ix.0.queue4.rxd_head: 684 >> dev.ix.0.queue4.rxd_tail: 681 >> dev.ix.0.queue4.rx_packets: 13204786830 >> dev.ix.0.queue4.rx_bytes: 3700845239 >> dev.ix.0.queue4.rx_copies: 8193379 >> dev.ix.0.queue4.lro_queued: 0 >> dev.ix.0.queue4.lro_flushed: 0 >> dev.ix.0.queue5.interrupt_rate: 15151 >> dev.ix.0.queue5.irqs: 7456613396 >> dev.ix.0.queue5.txd_head: 603 >> dev.ix.0.queue5.txd_tail: 603 >> dev.ix.0.queue5.tso_tx: 17 >> dev.ix.0.queue5.no_tx_dma_setup: 0 >> dev.ix.0.queue5.no_desc_avail: 0 >> dev.ix.0.queue5.tx_packets: 10639139790 >> dev.ix.0.queue5.rxd_head: 404 >> dev.ix.0.queue5.rxd_tail: 403 >> dev.ix.0.queue5.rx_packets: 13144301293 >> dev.ix.0.queue5.rx_bytes: 3986784766 >> dev.ix.0.queue5.rx_copies: 8256195 >> dev.ix.0.queue5.lro_queued: 0 >> dev.ix.0.queue5.lro_flushed: 0 >> dev.ix.0.queue6.interrupt_rate: 125000 >> dev.ix.0.queue6.irqs: 7466940576 >> dev.ix.0.queue6.txd_head: 1784 >> dev.ix.0.queue6.txd_tail: 1784 >> dev.ix.0.queue6.tso_tx: 2001 >> dev.ix.0.queue6.no_tx_dma_setup: 0 >> dev.ix.0.queue6.no_desc_avail: 0 >> dev.ix.0.queue6.tx_packets: 9784312967 >> dev.ix.0.queue6.rxd_head: 395 >> dev.ix.0.queue6.rxd_tail: 394 >> dev.ix.0.queue6.rx_packets: 13103079970 >> dev.ix.0.queue6.rx_bytes: 3581485264 >> dev.ix.0.queue6.rx_copies: 7336569 >> dev.ix.0.queue6.lro_queued: 0 >> dev.ix.0.queue6.lro_flushed: 0 >> dev.ix.0.queue7.interrupt_rate: 5319 >> dev.ix.0.queue7.irqs: 7486391989 >> dev.ix.0.queue7.txd_head: 1549 >> dev.ix.0.queue7.txd_tail: 1549 >> dev.ix.0.queue7.tso_tx: 2052 >> dev.ix.0.queue7.no_tx_dma_setup: 0 >> dev.ix.0.queue7.no_desc_avail: 0 >> dev.ix.0.queue7.tx_packets: 9758335509 >> dev.ix.0.queue7.rxd_head: 1990 >> dev.ix.0.queue7.rxd_tail: 1986 >> dev.ix.0.queue7.rx_packets: 13141436559 >> dev.ix.0.queue7.rx_bytes: 3877881546 >> dev.ix.0.queue7.rx_copies: 8505244 >> dev.ix.0.queue7.lro_queued: 0 >> dev.ix.0.queue7.lro_flushed: 0 >> dev.ix.0.mac_stats.crc_errs: 159 >> dev.ix.0.mac_stats.ill_errs: 5 >> dev.ix.0.mac_stats.byte_errs: 10 >> dev.ix.0.mac_stats.short_discards: 0 >> dev.ix.0.mac_stats.local_faults: 213 >> dev.ix.0.mac_stats.remote_faults: 37 >> dev.ix.0.mac_stats.rec_len_errs: 0 >> dev.ix.0.mac_stats.xon_txd: 33623392288 >> dev.ix.0.mac_stats.xon_recvd: 0 >> dev.ix.0.mac_stats.xoff_txd: 110161 >> dev.ix.0.mac_stats.xoff_recvd: 0 >> dev.ix.0.mac_stats.total_octets_rcvd: 37967032023930 >> dev.ix.0.mac_stats.good_octets_rcvd: 37965112076546 >> dev.ix.0.mac_stats.total_pkts_rcvd: 105201314224 >> dev.ix.0.mac_stats.good_pkts_rcvd: 18446743122750685300 >> dev.ix.0.mac_stats.mcast_pkts_rcvd: 43835 >> dev.ix.0.mac_stats.bcast_pkts_rcvd: 960566 >> dev.ix.0.mac_stats.rx_frames_64: 454067 >> dev.ix.0.mac_stats.rx_frames_65_127: 75350664570 >> dev.ix.0.mac_stats.rx_frames_128_255: 4726321485 >> dev.ix.0.mac_stats.rx_frames_256_511: 2282382994 >> dev.ix.0.mac_stats.rx_frames_512_1023: 3293330145 >> dev.ix.0.mac_stats.rx_frames_1024_1522: 19541190234 >> dev.ix.0.mac_stats.recv_undersized: 0 >> dev.ix.0.mac_stats.recv_fragmented: 0 >> dev.ix.0.mac_stats.recv_oversized: 0 >> dev.ix.0.mac_stats.recv_jabberd: 4 >> dev.ix.0.mac_stats.management_pkts_rcvd: 0 >> dev.ix.0.mac_stats.management_pkts_drpd: 0 >> dev.ix.0.mac_stats.checksum_errs: 527716650 >> dev.ix.0.mac_stats.good_octets_txd: 81355567084088 >> dev.ix.0.mac_stats.total_pkts_txd: 81545370171 >> dev.ix.0.mac_stats.good_pkts_txd: 47921867720 >> dev.ix.0.mac_stats.bcast_pkts_txd: 298646 >> dev.ix.0.mac_stats.mcast_pkts_txd: 18446744040086169062 >> dev.ix.0.mac_stats.management_pkts_txd: 0 >> dev.ix.0.mac_stats.tx_frames_64: 18446744045326220993 >> dev.ix.0.mac_stats.tx_frames_65_127: 15629697019 >> dev.ix.0.mac_stats.tx_frames_128_255: 4141756940 >> dev.ix.0.mac_stats.tx_frames_256_511: 2195917491 >> dev.ix.0.mac_stats.tx_frames_512_1023: 2717769904 >> dev.ix.0.mac_stats.tx_frames_1024_1522: 51620056990 >> >> dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - >> 2.5.15 >> dev.ix.1.%driver: ix >> dev.ix.1.%location: slot=0 function=1 handle=\_SB_.PCI1.BR48.S3F1 >> dev.ix.1.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 >> subdevice=0x7b11 class=0x020000 >> dev.ix.1.%parent: pci131 >> dev.ix.1.fc: 3 >> dev.ix.1.enable_aim: 1 >> dev.ix.1.advertise_speed: 0 >> dev.ix.1.dropped: 0 >> dev.ix.1.mbuf_defrag_failed: 0 >> dev.ix.1.watchdog_events: 0 >> dev.ix.1.link_irq: 46692 >> dev.ix.1.queue0.interrupt_rate: 15151 >> dev.ix.1.queue0.irqs: 2634341620 >> dev.ix.1.queue0.txd_head: 1101 >> dev.ix.1.queue0.txd_tail: 1101 >> dev.ix.1.queue0.tso_tx: 0 >> dev.ix.1.queue0.no_tx_dma_setup: 0 >> dev.ix.1.queue0.no_desc_avail: 4142 >> dev.ix.1.queue0.tx_packets: 4749431326 >> dev.ix.1.queue0.rxd_head: 812 >> dev.ix.1.queue0.rxd_tail: 811 >> dev.ix.1.queue0.rx_packets: 21135516 >> dev.ix.1.queue0.rx_bytes: 33229682 >> dev.ix.1.queue0.rx_copies: 119434 >> dev.ix.1.queue0.lro_queued: 0 >> dev.ix.1.queue0.lro_flushed: 0 >> dev.ix.1.queue1.interrupt_rate: 15151 >> dev.ix.1.queue1.irqs: 2616347302 >> dev.ix.1.queue1.txd_head: 797 >> dev.ix.1.queue1.txd_tail: 797 >> dev.ix.1.queue1.tso_tx: 0 >> dev.ix.1.queue1.no_tx_dma_setup: 0 >> dev.ix.1.queue1.no_desc_avail: 4129 >> dev.ix.1.queue1.tx_packets: 4730356550 >> dev.ix.1.queue1.rxd_head: 1345 >> dev.ix.1.queue1.rxd_tail: 1344 >> dev.ix.1.queue1.rx_packets: 20844457 >> dev.ix.1.queue1.rx_bytes: 35014827 >> dev.ix.1.queue1.rx_copies: 125167 >> dev.ix.1.queue1.lro_queued: 0 >> dev.ix.1.queue1.lro_flushed: 0 >> dev.ix.1.queue2.interrupt_rate: 5555 >> dev.ix.1.queue2.irqs: 2603166062 >> dev.ix.1.queue2.txd_head: 1428 >> dev.ix.1.queue2.txd_tail: 1432 >> dev.ix.1.queue2.tso_tx: 0 >> dev.ix.1.queue2.no_tx_dma_setup: 0 >> dev.ix.1.queue2.no_desc_avail: 34368 >> dev.ix.1.queue2.tx_packets: 4681339909 >> dev.ix.1.queue2.rxd_head: 1153 >> dev.ix.1.queue2.rxd_tail: 1152 >> dev.ix.1.queue2.rx_packets: 21263862 >> dev.ix.1.queue2.rx_bytes: 46251419 >> dev.ix.1.queue2.rx_copies: 125156 >> dev.ix.1.queue2.lro_queued: 0 >> dev.ix.1.queue2.lro_flushed: 0 >> dev.ix.1.queue3.interrupt_rate: 5319 >> dev.ix.1.queue3.irqs: 2658483423 >> dev.ix.1.queue3.txd_head: 179 >> dev.ix.1.queue3.txd_tail: 179 >> dev.ix.1.queue3.tso_tx: 0 >> dev.ix.1.queue3.no_tx_dma_setup: 0 >> dev.ix.1.queue3.no_desc_avail: 4189 >> dev.ix.1.queue3.tx_packets: 4811705668 >> dev.ix.1.queue3.rxd_head: 583 >> dev.ix.1.queue3.rxd_tail: 582 >> dev.ix.1.queue3.rx_packets: 20472051 >> dev.ix.1.queue3.rx_bytes: 29823943 >> dev.ix.1.queue3.rx_copies: 129450 >> dev.ix.1.queue3.lro_queued: 0 >> dev.ix.1.queue3.lro_flushed: 0 >> dev.ix.1.queue4.interrupt_rate: 5319 >> dev.ix.1.queue4.irqs: 2629295642 >> dev.ix.1.queue4.txd_head: 592 >> dev.ix.1.queue4.txd_tail: 594 >> dev.ix.1.queue4.tso_tx: 0 >> dev.ix.1.queue4.no_tx_dma_setup: 0 >> dev.ix.1.queue4.no_desc_avail: 4106 >> dev.ix.1.queue4.tx_packets: 4774679355 >> dev.ix.1.queue4.rxd_head: 259 >> dev.ix.1.queue4.rxd_tail: 258 >> dev.ix.1.queue4.rx_packets: 22561633 >> dev.ix.1.queue4.rx_bytes: 40196457 >> dev.ix.1.queue4.rx_copies: 135966 >> dev.ix.1.queue4.lro_queued: 0 >> dev.ix.1.queue4.lro_flushed: 0 >> dev.ix.1.queue5.interrupt_rate: 18518 >> dev.ix.1.queue5.irqs: 2628129098 >> dev.ix.1.queue5.txd_head: 1070 >> dev.ix.1.queue5.txd_tail: 1072 >> dev.ix.1.queue5.tso_tx: 0 >> dev.ix.1.queue5.no_tx_dma_setup: 0 >> dev.ix.1.queue5.no_desc_avail: 4127 >> dev.ix.1.queue5.tx_packets: 4750613588 >> dev.ix.1.queue5.rxd_head: 595 >> dev.ix.1.queue5.rxd_tail: 594 >> dev.ix.1.queue5.rx_packets: 21309940 >> dev.ix.1.queue5.rx_bytes: 29264004 >> dev.ix.1.queue5.rx_copies: 134128 >> dev.ix.1.queue5.lro_queued: 0 >> dev.ix.1.queue5.lro_flushed: 0 >> dev.ix.1.queue6.interrupt_rate: 125000 >> dev.ix.1.queue6.irqs: 2615444123 >> dev.ix.1.queue6.txd_head: 1008 >> dev.ix.1.queue6.txd_tail: 1008 >> dev.ix.1.queue6.tso_tx: 0 >> dev.ix.1.queue6.no_tx_dma_setup: 0 >> dev.ix.1.queue6.no_desc_avail: 4106 >> dev.ix.1.queue6.tx_packets: 4703519800 >> dev.ix.1.queue6.rxd_head: 1209 >> dev.ix.1.queue6.rxd_tail: 1208 >> dev.ix.1.queue6.rx_packets: 22783319 >> dev.ix.1.queue6.rx_bytes: 28091721 >> dev.ix.1.queue6.rx_copies: 131653 >> dev.ix.1.queue6.lro_queued: 0 >> dev.ix.1.queue6.lro_flushed: 0 >> dev.ix.1.queue7.interrupt_rate: 6024 >> dev.ix.1.queue7.irqs: 2644640254 >> dev.ix.1.queue7.txd_head: 1313 >> dev.ix.1.queue7.txd_tail: 1313 >> dev.ix.1.queue7.tso_tx: 0 >> dev.ix.1.queue7.no_tx_dma_setup: 0 >> dev.ix.1.queue7.no_desc_avail: 4106 >> dev.ix.1.queue7.tx_packets: 4824584039 >> dev.ix.1.queue7.rxd_head: 1463 >> dev.ix.1.queue7.rxd_tail: 1462 >> dev.ix.1.queue7.rx_packets: 21464418 >> dev.ix.1.queue7.rx_bytes: 38133966 >> dev.ix.1.queue7.rx_copies: 163180 >> dev.ix.1.queue7.lro_queued: 0 >> dev.ix.1.queue7.lro_flushed: 0 >> dev.ix.1.mac_stats.crc_errs: 0 >> dev.ix.1.mac_stats.ill_errs: 0 >> dev.ix.1.mac_stats.byte_errs: 0 >> dev.ix.1.mac_stats.short_discards: 0 >> dev.ix.1.mac_stats.local_faults: 0 >> dev.ix.1.mac_stats.remote_faults: 2 >> dev.ix.1.mac_stats.rec_len_errs: 0 >> dev.ix.1.mac_stats.xon_txd: 11207820571 >> dev.ix.1.mac_stats.xon_recvd: 0 >> dev.ix.1.mac_stats.xoff_txd: 3735958948 >> dev.ix.1.mac_stats.xoff_recvd: 0 >> dev.ix.1.mac_stats.total_octets_rcvd: 47828274751 >> dev.ix.1.mac_stats.good_octets_rcvd: 47225376551 >> dev.ix.1.mac_stats.total_pkts_rcvd: 177931572 >> dev.ix.1.mac_stats.good_pkts_rcvd: 18446743954331576567 >> dev.ix.1.mac_stats.mcast_pkts_rcvd: 7410 >> dev.ix.1.mac_stats.bcast_pkts_rcvd: 8907 >> dev.ix.1.mac_stats.rx_frames_64: 0 >> dev.ix.1.mac_stats.rx_frames_65_127: 134619798 >> dev.ix.1.mac_stats.rx_frames_128_255: 7996096 >> dev.ix.1.mac_stats.rx_frames_256_511: 3996945 >> dev.ix.1.mac_stats.rx_frames_512_1023: 5145938 >> dev.ix.1.mac_stats.rx_frames_1024_1522: 21331154 >> dev.ix.1.mac_stats.recv_undersized: 0 >> dev.ix.1.mac_stats.recv_fragmented: 0 >> dev.ix.1.mac_stats.recv_oversized: 0 >> dev.ix.1.mac_stats.recv_jabberd: 0 >> dev.ix.1.mac_stats.management_pkts_rcvd: 0 >> dev.ix.1.mac_stats.management_pkts_drpd: 0 >> dev.ix.1.mac_stats.checksum_errs: 219681 >> dev.ix.1.mac_stats.good_octets_txd: 38874487735713 >> dev.ix.1.mac_stats.total_pkts_txd: 38026147390 >> dev.ix.1.mac_stats.good_pkts_txd: 27377335167 >> dev.ix.1.mac_stats.bcast_pkts_txd: 67411 >> dev.ix.1.mac_stats.mcast_pkts_txd: 18446744063060778699 >> dev.ix.1.mac_stats.management_pkts_txd: 0 >> dev.ix.1.mac_stats.tx_frames_64: 18446744065500352065 >> dev.ix.1.mac_stats.tx_frames_65_127: 6755049420 >> dev.ix.1.mac_stats.tx_frames_128_255: 1914523761 >> dev.ix.1.mac_stats.tx_frames_256_511: 1038553757 >> dev.ix.1.mac_stats.tx_frames_512_1023: 1240890942 >> dev.ix.1.mac_stats.tx_frames_1024_1522: 24637516838 >> >> >> Cheers, >> >> >>>> -a >>>> >>>> >>>> On 5 September 2014 07:35, Marcelo Gondim wrote: >>>> >>>>> Hi Adrian, >>>>> >>>>> I confirmed with the support staff of the room where the server is, >>>>> that the >>>>> ambient temperature was normal. >>>>> >>>>> >>>>> On 04/09/2014 22:46, Marcelo Gondim wrote: >>>>> >>>>>> On 04/09/2014 20:48, Adrian Chadd wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> The only time this has happened to me is because the card overheated. >>>>>>> Can you check that? >>>>>>> >>>>>> Hi Adrian, >>>>>> >>>>>> The room where the equipment is located is very cold but I'll check it >>>>>> out. >>>>>> Also seen at the time of the problem, a lot of dropped packets. >>>>>> >>>>>> # netstat -idn >>>>>> ... >>>>>> ix0 1500 a0:36:9f:2a:6d:ac 18446743423829095869 159 >>>>>> 750924631703 53285910688 0 0 0 >>>>>> ix0 - fe80::a236:9f fe80::a236:9fff:f 0 - - 2 >>>>>> - - - >>>>>> ix1 1500 a0:36:9f:2a:6d:ae 18446743954328745465 0 >>>>>> 119550050209 20178077451 0 0 0 >>>>>> ix1 - fe80::a236:9f fe80::a236:9fff:f 0 - - 1 >>>>>> - - - >>>>>> ... >>>>>> >>>>>> 119550050209 droped packets on ix1 and 750924631703 droped on ix0 >>>>>> >>>>>> Could be interesting I upgrade to10.1-PRERELEASE? >>>>>> Could there be a problem with the driver? >>>>>> >>>>>> Traffic on ix0: 1.4Gbps output / 600Mbps input >>>>>> Traffic on ix1: 1.2Gbps output >>>>>> >>>>>> PPS on ix0: 163Kpps output / 215Kpps input >>>>>> PPS on ix1: 131Kpps output >>>>>> >>>>>> Thanks for your help. >>>>>> >>>>>>> >>>>>>> -a >>>>>>> >>>>>>> >>>>>>> On 4 September 2014 16:14, Marcelo Gondim >>>>>>> wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> I have an Intel X520-SR2and today was working when all traffic >>>>>>>> stopped. >>>>>>>> I looked in the logs and found this message: >>>>>>>> >>>>>>>> Sep 4 18:29:53 rt01 kernel: ix1: >>>>>>>> Sep 4 18:29:53 rt01 kernel: CRITICAL: ECC ERROR!! Please Reboot!! >>>>>>>> >>>>>>>> # uname -a >>>>>>>> FreeBSD rt01.xxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #10 >>>>>>>> r267839: >>>>>>>> Thu >>>>>>>> Jul 10 15:35:04 BRT 2014 >>>>>>>> root@rt01.xxxxx.com.br:/usr/obj/usr/src/sys/GONDIM10 amd64 >>>>>>>> >>>>>>>> # netstat -m >>>>>>>> 98324/53476/151800 mbufs in use (current/cache/total) >>>>>>>> 98301/44951/143252/1014370 mbuf clusters in use >>>>>>>> (current/cache/total/max) >>>>>>>> 98301/44897 mbuf+clusters out of packet secondary zone in use >>>>>>>> (current/cache) >>>>>>>> 0/421/421/507184 4k (page size) jumbo clusters in use >>>>>>>> (current/cache/total/max) >>>>>>>> 0/0/0/150276 9k jumbo clusters in use (current/cache/total/max) >>>>>>>> 0/0/0/84530 16k jumbo clusters in use (current/cache/total/max) >>>>>>>> 221183K/104955K/326138K bytes allocated to network >>>>>>>> (current/cache/total) >>>>>>>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >>>>>>>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >>>>>>>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >>>>>>>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >>>>>>>> 0 requests for sfbufs denied >>>>>>>> 0 requests for sfbufs delayed >>>>>>>> 0 requests for I/O initiated by sendfile >>>>>>>> >>>>>>>> Best regards, >>>>>>>> From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 19:09:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A33EB6C4 for ; Mon, 8 Sep 2014 19:09:12 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8470F1BFB for ; Mon, 8 Sep 2014 19:09:12 +0000 (UTC) Received: from [192.168.200.200] (unknown [50.136.155.142]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 16B00192906 for ; Mon, 8 Sep 2014 19:09:11 +0000 (UTC) Subject: ixgbe(4) spin lock held too long From: Sean Bruno Reply-To: sbruno@freebsd.org To: freebsd-net@freebsd.org Content-Type: text/plain; charset="us-ascii" Date: Mon, 08 Sep 2014 12:09:08 -0700 Message-ID: <1410203348.1343.1.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 19:09:12 -0000 This sort of looks like the hardware failed to respond to us in time? Too busy? sean panic: spin lock held too long GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: spin lock 0xffffffff812a0400 (callout) held by 0xfffff800151fe000 (tid 100003) too long panic: spin lock held too long cpuid = 4 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe1f292752b0 panic() at panic+0x155/frame 0xfffffe1f29275330 _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0x243/frame 0xfffffe1f29275390 callout_lock() at callout_lock+0xd4/frame 0xfffffe1f292753d0 callout_reset_sbt_on() at callout_reset_sbt_on+0x10b/frame 0xfffffe1f29275450 tcp_timer_activate() at tcp_timer_activate+0xe7/frame 0xfffffe1f29275470 tcp_do_segment() at tcp_do_segment+0x96/frame 0xfffffe1f292755b0 tcp_input() at tcp_input+0xeed/frame 0xfffffe1f292756f0 ip_input() at ip_input+0x97/frame 0xfffffe1f29275740 netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe1f292757b0 ether_demux() at ether_demux+0x126/frame 0xfffffe1f292757e0 ether_nh_input() at ether_nh_input+0x349/frame 0xfffffe1f29275840 netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe1f292758b0 tcp_lro_flush() at tcp_lro_flush+0x198/frame 0xfffffe1f292758d0 ixgbe_rxeof() at ixgbe_rxeof+0x6b3/frame 0xfffffe1f29275990 ixgbe_msix_que() at ixgbe_msix_que+0xba/frame 0xfffffe1f292759e0 intr_event_execute_handlers() at intr_event_execute_handlers+0xab/frame 0xfffffe1f29275a20 ithread_loop() at ithread_loop+0x96/frame 0xfffffe1f29275a70 fork_exit() at fork_exit+0x9a/frame 0xfffffe1f29275ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe1f29275ab0 --- trap 0, rip = 0, rsp = 0xfffffe1f29275b70, rbp = 0 --- Uptime: 8d20h4m58s From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 19:19:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71119D8D for ; Mon, 8 Sep 2014 19:19:27 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FF9F1D29 for ; Mon, 8 Sep 2014 19:19:27 +0000 (UTC) Received: from [192.168.200.200] (unknown [50.136.155.142]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 5E08A192906 for ; Mon, 8 Sep 2014 19:19:26 +0000 (UTC) Subject: Re: ixgbe(4) spin lock held too long From: Sean Bruno Reply-To: sbruno@freebsd.org To: FreeBSD Net In-Reply-To: <1410203348.1343.1.camel@bruno> References: <1410203348.1343.1.camel@bruno> Content-Type: text/plain; charset="us-ascii" Date: Mon, 08 Sep 2014 12:19:25 -0700 Message-ID: <1410203965.1343.3.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 19:19:27 -0000 On Mon, 2014-09-08 at 12:09 -0700, Sean Bruno wrote: > This sort of looks like the hardware failed to respond to us in time? > Too busy? > > sean > This seems to be affecting my 10/stable machines from 15Aug2014. Not a lot of churn in the code so I don't think this is new. The afflicted machines, quite a few by my count, appear to have not been super busy (pushing about 200 Mb/s). sean > > panic: spin lock held too long > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > spin lock 0xffffffff812a0400 (callout) held by 0xfffff800151fe000 (tid > 100003) too long > panic: spin lock held too long > cpuid = 4 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe1f292752b0 > panic() at panic+0x155/frame 0xfffffe1f29275330 > _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0x243/frame > 0xfffffe1f29275390 > callout_lock() at callout_lock+0xd4/frame 0xfffffe1f292753d0 > callout_reset_sbt_on() at callout_reset_sbt_on+0x10b/frame > 0xfffffe1f29275450 > tcp_timer_activate() at tcp_timer_activate+0xe7/frame 0xfffffe1f29275470 > tcp_do_segment() at tcp_do_segment+0x96/frame 0xfffffe1f292755b0 > tcp_input() at tcp_input+0xeed/frame 0xfffffe1f292756f0 > ip_input() at ip_input+0x97/frame 0xfffffe1f29275740 > netisr_dispatch_src() at netisr_dispatch_src+0x62/frame > 0xfffffe1f292757b0 > ether_demux() at ether_demux+0x126/frame 0xfffffe1f292757e0 > ether_nh_input() at ether_nh_input+0x349/frame 0xfffffe1f29275840 > netisr_dispatch_src() at netisr_dispatch_src+0x62/frame > 0xfffffe1f292758b0 > tcp_lro_flush() at tcp_lro_flush+0x198/frame 0xfffffe1f292758d0 > ixgbe_rxeof() at ixgbe_rxeof+0x6b3/frame 0xfffffe1f29275990 > ixgbe_msix_que() at ixgbe_msix_que+0xba/frame 0xfffffe1f292759e0 > intr_event_execute_handlers() at intr_event_execute_handlers+0xab/frame > 0xfffffe1f29275a20 > ithread_loop() at ithread_loop+0x96/frame 0xfffffe1f29275a70 > fork_exit() at fork_exit+0x9a/frame 0xfffffe1f29275ab0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe1f29275ab0 > --- trap 0, rip = 0, rsp = 0xfffffe1f29275b70, rbp = 0 --- > Uptime: 8d20h4m58s > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 19:34:05 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 049D17E1; Mon, 8 Sep 2014 19:34:05 +0000 (UTC) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id DB06B1F12; Mon, 8 Sep 2014 19:34:04 +0000 (UTC) Received: from marvin.lab.vangyzen.net (c-24-125-214-90.hsd1.va.comcast.net [24.125.214.90]) by smtp.vangyzen.net (Postfix) with ESMTPSA id 90BF656444; Mon, 8 Sep 2014 14:34:03 -0500 (CDT) Message-ID: <540E04AA.80806@vangyzen.net> Date: Mon, 08 Sep 2014 15:34:02 -0400 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: sbruno@freebsd.org, FreeBSD Net Subject: Re: ixgbe(4) spin lock held too long References: <1410203348.1343.1.camel@bruno> <1410203965.1343.3.camel@bruno> In-Reply-To: <1410203965.1343.3.camel@bruno> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 19:34:05 -0000 On 09/08/2014 15:19, Sean Bruno wrote: > On Mon, 2014-09-08 at 12:09 -0700, Sean Bruno wrote: >> This sort of looks like the hardware failed to respond to us in time? >> Too busy? >> >> sean >> > This seems to be affecting my 10/stable machines from 15Aug2014. > > Not a lot of churn in the code so I don't think this is new. The > afflicted machines, quite a few by my count, appear to have not been > super busy (pushing about 200 Mb/s). > > sean > > > >> panic: spin lock held too long >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you >> are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> spin lock 0xffffffff812a0400 (callout) held by 0xfffff800151fe000 (tid >> 100003) too long TID 100003 is usually a kernel idle thread, which would seem to indicate a dangling lock. Can you enable WITNESS (without WITNESS_SKIPSPIN) on this box? >> panic: spin lock held too long >> cpuid = 4 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe1f292752b0 >> panic() at panic+0x155/frame 0xfffffe1f29275330 >> _mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0x243/frame >> 0xfffffe1f29275390 >> callout_lock() at callout_lock+0xd4/frame 0xfffffe1f292753d0 >> callout_reset_sbt_on() at callout_reset_sbt_on+0x10b/frame >> 0xfffffe1f29275450 >> tcp_timer_activate() at tcp_timer_activate+0xe7/frame 0xfffffe1f29275470 >> tcp_do_segment() at tcp_do_segment+0x96/frame 0xfffffe1f292755b0 >> tcp_input() at tcp_input+0xeed/frame 0xfffffe1f292756f0 >> ip_input() at ip_input+0x97/frame 0xfffffe1f29275740 >> netisr_dispatch_src() at netisr_dispatch_src+0x62/frame >> 0xfffffe1f292757b0 >> ether_demux() at ether_demux+0x126/frame 0xfffffe1f292757e0 >> ether_nh_input() at ether_nh_input+0x349/frame 0xfffffe1f29275840 >> netisr_dispatch_src() at netisr_dispatch_src+0x62/frame >> 0xfffffe1f292758b0 >> tcp_lro_flush() at tcp_lro_flush+0x198/frame 0xfffffe1f292758d0 >> ixgbe_rxeof() at ixgbe_rxeof+0x6b3/frame 0xfffffe1f29275990 >> ixgbe_msix_que() at ixgbe_msix_que+0xba/frame 0xfffffe1f292759e0 >> intr_event_execute_handlers() at intr_event_execute_handlers+0xab/frame >> 0xfffffe1f29275a20 >> ithread_loop() at ithread_loop+0x96/frame 0xfffffe1f29275a70 >> fork_exit() at fork_exit+0x9a/frame 0xfffffe1f29275ab0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe1f29275ab0 >> --- trap 0, rip = 0, rsp = 0xfffffe1f29275b70, rbp = 0 --- >> Uptime: 8d20h4m58s >> > From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 19:56:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C5A6351 for ; Mon, 8 Sep 2014 19:56:00 +0000 (UTC) Received: from webmail2.jnielsen.net (webmail2.jnielsen.net [50.114.224.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "webmail2.jnielsen.net", Issuer "freebsdsolutions.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D537F125A for ; Mon, 8 Sep 2014 19:55:59 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by webmail2.jnielsen.net (8.14.9/8.14.9) with ESMTP id s88JtoD1054290 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 8 Sep 2014 13:55:52 -0600 (MDT) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail2.jnielsen.net: Host office.betterlinux.com [199.58.199.60] claimed to be [10.10.1.198] Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: When to use and not use divert/natd ... From: John Nielsen In-Reply-To: Date: Mon, 8 Sep 2014 13:55:49 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <366D397B-4521-4E5B-8AB0-2E218192C2AD@jnielsen.net> References: To: John Case X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 19:56:00 -0000 On Sep 5, 2014, at 9:15 PM, John Case wrote: > For many years I would build FreeBSD firewalls and they would be very, = very simple - I just set gateway_enable=3D"yes" in rc.conf and = everything just worked. >=20 > However, these firewalls *always* had real, routable IPs no both = sides. Both interfaces had real, routable IPs. >=20 > Now I have a firewall that has two non-routable IPs for its = interfaces, and is connected to a internet router with the real IP. = When I try to builda very simple firewall it does not work, and I am = forced to use ipdivert and natd. >=20 > If I use ipdivert and natd, it works just fine. >=20 > So, am I correct that I can create a simple gateway without = natd/divert as long as both interfaces are real IPs, but if both = interfaces are non-routable IPs, I am forced to use divert/natd ? Just think about the 'routing' aspect. In your current scenario it = sounds like the Internet-connected device is doing NAT. It knows about = its public IP and its private subnet. It sounds like you have a second = private subnet behind your FreeBSD machine about which the = Internet-connected device knows nothing. For packets to get from the = Internet-connected device to your second subnet one of two things needs = to happen: 1) The Internet-connected device has a static route to the second = subnet (so it knows to use your FreeBSD machine as the gateway), or 2) The FreeBSD machine performs NAT (a second time), so the = Internet-connected device send traffic to it even though it knows = nothing about the subnet behind it. I would prefer 1) as it's simpler and double-NAT isn't generally a good = thing. However, if you don't have a way to add a route to the = Internet-connected device then 2) isn't necessarily bad. In your previous all-routable-IPs setups something was presumably = advertising the route for you. The new setup isn't much different in = principle. JN PS: Using the in-kernel NAT with IPFW is simpler and more efficient than = using natd... From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 20:13:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9DCB8AB3; Mon, 8 Sep 2014 20:13:59 +0000 (UTC) Received: from ns2.videon-znojmo.cz (ns2.videon-znojmo.cz [212.96.160.50]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E8C11501; Mon, 8 Sep 2014 20:13:58 +0000 (UTC) Received: from adminPC ([213.211.53.98]) by ns2.videon-znojmo.cz (8.14.9/8.14.9) with ESMTP id s88K4YnJ059271; Mon, 8 Sep 2014 22:04:34 +0200 (CEST) (envelope-from alesnechvatal@videon-znojmo.cz) From: "=?iso-8859-2?Q?Ing._Ale=B9_Nechv=E1tal?=" To: "'Marcelo Gondim'" , "'Eric Joyner'" References: <5408F23C.2030309@bsdinfo.com.br> <54091607.9010100@bsdinfo.com.br> <5409CA44.8070203@bsdinfo.com.br> <540A1A3E.2040306@bsdinfo.com.br> <540A8200.5010404@bsdinfo.com.br> <540DFAE3.8070001@bsdinfo.com.br> In-Reply-To: <540DFAE3.8070001@bsdinfo.com.br> Subject: RE: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! Date: Mon, 8 Sep 2014 22:05:42 +0200 Message-ID: <002001cfcba0$4812c7f0$d83857d0$@videon-znojmo.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQHDzs3aBwnJinWaiqhrSHzcS0agVQImyzsCAfXrMZkBv17xzgFuqIsBAgqTvuICj/5ChAGAeC2EAiuTOmabkwyP0A== Content-Language: cs Cc: 'Jack F Vogel' , 'FreeBSD Net' , 'Adrian Chadd' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 20:13:59 -0000 I was experiencing similar behaviour on a machine acting as a gateway. It has one X520-SR2, both SPF+ ports used to connet to another machines with the same NICs. One of the NICs suddenly stopped to pass data through and the the ifconfing down up cycle was needed to make it work again, that in better case. In the worse case, the machine rebooted without leaving any message in syslog (it's on a remote site). This happened in intervals of 5 mins. up to 1 hour (I am not sure about the exact correlation between traffic going through and the number of these events, but during traffic peaks, it seems to have happened more often). First I checked the mbufs, but didn't see any problem. Then I tried to disable TSO (on both sides of the links), but without observing any improvement. I was playing with different versions of the kernel, the newer the kernel (I went up to 9.3 STABLE, didn't try 10 or HEAD) the more instability. It seemed, that the most stable was 9.1 PRERELEASE with 2.4.8 driver (at least I didn't see reboots, a cron script could keep the gateway passing data by shutting down and bringing up the interfaces). Now I am running 9.3 RELEASE r271227M with the 2.5.25 driver from Intel's website. This configuration was not stable either, until I discovered high interrupt rate on each queue (the sum on all queues reached over 400k in some moments). I turned off the limit of interrupts by setting hw.intr_storm_threshold=0 (previously set to 150000). Since the following reboot (involuntary, btw.) the gateway keeps running smoothly for 7 hours now with peak throughput of approx. 1.6 Gbit/s. The sum of interrupt rates on all your queues looks quite high too, so there could be some similarity and disabling the mentioned limit could help. My sysctl.conf: kern.ipc.somaxconn=1024 net.inet.icmp.icmplim=100 net.inet.tcp.blackhole=2 net.inet.udp.blackhole=1 net.inet.tcp.sendspace=262144 net.inet.tcp.recvspace=262144 net.inet.ip.fastforwarding=1 kern.ipc.maxsockbuf=16777216 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 hw.intr_storm_threshold=0 #sysctl -a dev.ix.1 dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.5.25 dev.ix.1.%driver: ix dev.ix.1.%location: slot=0 function=1 dev.ix.1.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 subdevice=0x7a11 class=0x020000 dev.ix.1.%parent: pci1 dev.ix.1.fc: 3 dev.ix.1.enable_aim: 1 dev.ix.1.advertise_speed: 0 dev.ix.1.dropped: 0 dev.ix.1.mbuf_defrag_failed: 0 dev.ix.1.watchdog_events: 0 dev.ix.1.link_irq: 2 dev.ix.1.queue0.interrupt_rate: 31250 dev.ix.1.queue0.irqs: 152182132 dev.ix.1.queue0.txd_head: 526 dev.ix.1.queue0.txd_tail: 526 dev.ix.1.queue0.tso_tx: 2 dev.ix.1.queue0.no_tx_dma_setup: 0 dev.ix.1.queue0.no_desc_avail: 119373 dev.ix.1.queue0.tx_packets: 274418209 dev.ix.1.queue0.rxd_head: 441 dev.ix.1.queue0.rxd_tail: 440 dev.ix.1.queue0.rx_packets: 103852222 dev.ix.1.queue0.rx_bytes: 25483398737 dev.ix.1.queue0.rx_copies: 87237192 dev.ix.1.queue0.lro_queued: 0 dev.ix.1.queue0.lro_flushed: 0 dev.ix.1.queue1.interrupt_rate: 31250 dev.ix.1.queue1.irqs: 152973172 dev.ix.1.queue1.txd_head: 1282 dev.ix.1.queue1.txd_tail: 1282 dev.ix.1.queue1.tso_tx: 50 dev.ix.1.queue1.no_tx_dma_setup: 0 dev.ix.1.queue1.no_desc_avail: 114840 dev.ix.1.queue1.tx_packets: 271827311 dev.ix.1.queue1.rxd_head: 495 dev.ix.1.queue1.rxd_tail: 494 dev.ix.1.queue1.rx_packets: 103292003 dev.ix.1.queue1.rx_bytes: 20681070454 dev.ix.1.queue1.rx_copies: 89490795 dev.ix.1.queue1.lro_queued: 0 dev.ix.1.queue1.lro_flushed: 0 dev.ix.1.queue2.interrupt_rate: 21739 dev.ix.1.queue2.irqs: 81502157 dev.ix.1.queue2.txd_head: 313 dev.ix.1.queue2.txd_tail: 313 dev.ix.1.queue2.tso_tx: 50 dev.ix.1.queue2.no_tx_dma_setup: 0 dev.ix.1.queue2.no_desc_avail: 111923 dev.ix.1.queue2.tx_packets: 23189156 dev.ix.1.queue2.rxd_head: 349 dev.ix.1.queue2.rxd_tail: 348 dev.ix.1.queue2.rx_packets: 94457797 dev.ix.1.queue2.rx_bytes: 18053240855 dev.ix.1.queue2.rx_copies: 82430878 dev.ix.1.queue2.lro_queued: 0 dev.ix.1.queue2.lro_flushed: 0 dev.ix.1.queue3.interrupt_rate: 83333 dev.ix.1.queue3.irqs: 81678068 dev.ix.1.queue3.txd_head: 1990 dev.ix.1.queue3.txd_tail: 1990 dev.ix.1.queue3.tso_tx: 3 dev.ix.1.queue3.no_tx_dma_setup: 0 dev.ix.1.queue3.no_desc_avail: 110545 dev.ix.1.queue3.tx_packets: 21258249 dev.ix.1.queue3.rxd_head: 1544 dev.ix.1.queue3.rxd_tail: 1543 dev.ix.1.queue3.rx_packets: 99140619 dev.ix.1.queue3.rx_bytes: 19476001717 dev.ix.1.queue3.rx_copies: 86228489 dev.ix.1.queue3.lro_queued: 0 dev.ix.1.queue3.lro_flushed: 0 dev.ix.1.mac_stats.crc_errs: 0 dev.ix.1.mac_stats.ill_errs: 0 dev.ix.1.mac_stats.byte_errs: 0 dev.ix.1.mac_stats.short_discards: 0 dev.ix.1.mac_stats.local_faults: 0 dev.ix.1.mac_stats.remote_faults: 2 dev.ix.1.mac_stats.rec_len_errs: 0 dev.ix.1.mac_stats.xon_txd: 0 dev.ix.1.mac_stats.xon_recvd: 0 dev.ix.1.mac_stats.xoff_txd: 0 dev.ix.1.mac_stats.xoff_recvd: 0 dev.ix.1.mac_stats.total_octets_rcvd: 85345731961 dev.ix.1.mac_stats.good_octets_rcvd: 85345731961 dev.ix.1.mac_stats.total_pkts_rcvd: 400734734 dev.ix.1.mac_stats.good_pkts_rcvd: 400734734 dev.ix.1.mac_stats.mcast_pkts_rcvd: 0 dev.ix.1.mac_stats.bcast_pkts_rcvd: 0 dev.ix.1.mac_stats.rx_frames_64: 190985269 dev.ix.1.mac_stats.rx_frames_65_127: 142879679 dev.ix.1.mac_stats.rx_frames_128_255: 18114498 dev.ix.1.mac_stats.rx_frames_256_511: 5882972 dev.ix.1.mac_stats.rx_frames_512_1023: 7407080 dev.ix.1.mac_stats.rx_frames_1024_1522: 35465236 dev.ix.1.mac_stats.recv_undersized: 0 dev.ix.1.mac_stats.recv_fragmented: 0 dev.ix.1.mac_stats.recv_oversized: 0 dev.ix.1.mac_stats.recv_jabberd: 0 dev.ix.1.mac_stats.management_pkts_rcvd: 0 dev.ix.1.mac_stats.management_pkts_drpd: 0 dev.ix.1.mac_stats.checksum_errs: 462621 dev.ix.1.mac_stats.good_octets_txd: 705061464268 dev.ix.1.mac_stats.total_pkts_txd: 590658400 dev.ix.1.mac_stats.good_pkts_txd: 590658400 dev.ix.1.mac_stats.bcast_pkts_txd: 2907 dev.ix.1.mac_stats.mcast_pkts_txd: 0 dev.ix.1.mac_stats.management_pkts_txd: 0 dev.ix.1.mac_stats.tx_frames_64: 24533923 dev.ix.1.mac_stats.tx_frames_65_127: 65221733 dev.ix.1.mac_stats.tx_frames_128_255: 24274012 dev.ix.1.mac_stats.tx_frames_256_511: 12092320 dev.ix.1.mac_stats.tx_frames_512_1023: 10358523 dev.ix.1.mac_stats.tx_frames_1024_1522: 454177889 > -----Original Message----- > From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd- > net@freebsd.org] On Behalf Of Marcelo Gondim > Sent: Monday, September 08, 2014 8:52 PM > To: Eric Joyner > Cc: Jack F Vogel; FreeBSD Net; Adrian Chadd > Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! > > On 08/09/2014 15:38, Eric Joyner wrote: > > Getting local / remote faults is strange; are these stats from after you > > rebooted? > > When the crash happens I just do: > > # ifconfig ix0 down; ifconfig ix0 up; ifconfig ix1 down; ifconfig up ixi1 > > After that the two interface ports return to work. > > In no time I had to restart the server. > > # uptime > 3:51PM up 11 days, 15:44, 2 users, load averages: 5.36, 5.68, 5.69 > > Tomorrow will be going to the Datacenter to check on the temperature of > the network interface. As the crashes occur when traffic is high, I > believe that the temperature increases because processing is high. > > > > > > --- > > - Eric Joyner > > > > On Fri, Sep 5, 2014 at 8:39 PM, Marcelo Gondim > > wrote: > > > >> On 05/09/2014 17:17, Marcelo Gondim wrote: > >> > >>> On 05/09/2014 16:49, Adrian Chadd wrote: > >>> > >>>> Hi, > >>>> > >>>> But is the airflow in the unit sufficient? > >>>> > >>>> I had this problem at a previous job - the box was running fine, the > >>>> room was very cold, but the internal fans in the server were set to > >>>> "be very quiet". It wasn't enough to keep the ixgbe NICs happy. I had > >>>> to change the fan settings to "just always run full speed". > >>>> > >>>> The fan temperature feedback loop was based on sensors on the CPU, > >>>> _not_ on the peripherals. > >>>> > >>> Hi Adrian, > >>> > >>> Ummm. I'll check it and improve internal cooling. :) > >>> She is not happy and I'm also not. rsrsrsr > >>> > >>> Cheers, > >>> > >> Besides the problem of heating of the network interface, I am putting > some > >> information here. Could you tell me if there is something strange or is it > >> normal? > >> > >> dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version > - > >> 2.5.15 > >> dev.ix.0.%driver: ix > >> dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0 > >> dev.ix.0.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 > >> subdevice=0x7b11 class=0x020000 > >> dev.ix.0.%parent: pci131 > >> dev.ix.0.fc: 3 > >> dev.ix.0.enable_aim: 1 > >> dev.ix.0.advertise_speed: 0 > >> dev.ix.0.dropped: 0 > >> dev.ix.0.mbuf_defrag_failed: 0 > >> dev.ix.0.watchdog_events: 0 > >> dev.ix.0.link_irq: 121769 > >> dev.ix.0.queue0.interrupt_rate: 5319 > >> dev.ix.0.queue0.irqs: 7900830877 > >> dev.ix.0.queue0.txd_head: 1037 > >> dev.ix.0.queue0.txd_tail: 1037 > >> dev.ix.0.queue0.tso_tx: 142 > >> dev.ix.0.queue0.no_tx_dma_setup: 0 > >> dev.ix.0.queue0.no_desc_avail: 0 > >> dev.ix.0.queue0.tx_packets: 9725701450 > >> dev.ix.0.queue0.rxd_head: 1175 > >> dev.ix.0.queue0.rxd_tail: 1174 > >> dev.ix.0.queue0.rx_packets: 13069276955 > >> dev.ix.0.queue0.rx_bytes: 3391061018 > >> dev.ix.0.queue0.rx_copies: 8574407 > >> dev.ix.0.queue0.lro_queued: 0 > >> dev.ix.0.queue0.lro_flushed: 0 > >> dev.ix.0.queue1.interrupt_rate: 41666 > >> dev.ix.0.queue1.irqs: 7681141208 > >> dev.ix.0.queue1.txd_head: 219 > >> dev.ix.0.queue1.txd_tail: 221 > >> dev.ix.0.queue1.tso_tx: 57 > >> dev.ix.0.queue1.no_tx_dma_setup: 0 > >> dev.ix.0.queue1.no_desc_avail: 44334 > >> dev.ix.0.queue1.tx_packets: 10196891433 > >> dev.ix.0.queue1.rxd_head: 1988 > >> dev.ix.0.queue1.rxd_tail: 1987 > >> dev.ix.0.queue1.rx_packets: 13210132242 > >> dev.ix.0.queue1.rx_bytes: 4317357059 > >> dev.ix.0.queue1.rx_copies: 8131936 > >> dev.ix.0.queue1.lro_queued: 0 > >> dev.ix.0.queue1.lro_flushed: 0 > >> dev.ix.0.queue2.interrupt_rate: 5319 > >> dev.ix.0.queue2.irqs: 7647486080 > >> dev.ix.0.queue2.txd_head: 761 > >> dev.ix.0.queue2.txd_tail: 761 > >> dev.ix.0.queue2.tso_tx: 409 > >> dev.ix.0.queue2.no_tx_dma_setup: 0 > >> dev.ix.0.queue2.no_desc_avail: 54207 > >> dev.ix.0.queue2.tx_packets: 10161246425 > >> dev.ix.0.queue2.rxd_head: 1874 > >> dev.ix.0.queue2.rxd_tail: 1872 > >> dev.ix.0.queue2.rx_packets: 13175551880 > >> dev.ix.0.queue2.rx_bytes: 4472798418 > >> dev.ix.0.queue2.rx_copies: 7488876 > >> dev.ix.0.queue2.lro_queued: 0 > >> dev.ix.0.queue2.lro_flushed: 0 > >> dev.ix.0.queue3.interrupt_rate: 500000 > >> dev.ix.0.queue3.irqs: 7641129521 > >> dev.ix.0.queue3.txd_head: 2039 > >> dev.ix.0.queue3.txd_tail: 2039 > >> dev.ix.0.queue3.tso_tx: 9 > >> dev.ix.0.queue3.no_tx_dma_setup: 0 > >> dev.ix.0.queue3.no_desc_avail: 150346 > >> dev.ix.0.queue3.tx_packets: 10619971896 > >> dev.ix.0.queue3.rxd_head: 1055 > >> dev.ix.0.queue3.rxd_tail: 1054 > >> dev.ix.0.queue3.rx_packets: 13137835529 > >> dev.ix.0.queue3.rx_bytes: 4063197306 > >> dev.ix.0.queue3.rx_copies: 8188713 > >> dev.ix.0.queue3.lro_queued: 0 > >> dev.ix.0.queue3.lro_flushed: 0 > >> dev.ix.0.queue4.interrupt_rate: 5319 > >> dev.ix.0.queue4.irqs: 7439824996 > >> dev.ix.0.queue4.txd_head: 26 > >> dev.ix.0.queue4.txd_tail: 26 > >> dev.ix.0.queue4.tso_tx: 553912 > >> dev.ix.0.queue4.no_tx_dma_setup: 0 > >> dev.ix.0.queue4.no_desc_avail: 0 > >> dev.ix.0.queue4.tx_packets: 10658683718 > >> dev.ix.0.queue4.rxd_head: 684 > >> dev.ix.0.queue4.rxd_tail: 681 > >> dev.ix.0.queue4.rx_packets: 13204786830 > >> dev.ix.0.queue4.rx_bytes: 3700845239 > >> dev.ix.0.queue4.rx_copies: 8193379 > >> dev.ix.0.queue4.lro_queued: 0 > >> dev.ix.0.queue4.lro_flushed: 0 > >> dev.ix.0.queue5.interrupt_rate: 15151 > >> dev.ix.0.queue5.irqs: 7456613396 > >> dev.ix.0.queue5.txd_head: 603 > >> dev.ix.0.queue5.txd_tail: 603 > >> dev.ix.0.queue5.tso_tx: 17 > >> dev.ix.0.queue5.no_tx_dma_setup: 0 > >> dev.ix.0.queue5.no_desc_avail: 0 > >> dev.ix.0.queue5.tx_packets: 10639139790 > >> dev.ix.0.queue5.rxd_head: 404 > >> dev.ix.0.queue5.rxd_tail: 403 > >> dev.ix.0.queue5.rx_packets: 13144301293 > >> dev.ix.0.queue5.rx_bytes: 3986784766 > >> dev.ix.0.queue5.rx_copies: 8256195 > >> dev.ix.0.queue5.lro_queued: 0 > >> dev.ix.0.queue5.lro_flushed: 0 > >> dev.ix.0.queue6.interrupt_rate: 125000 > >> dev.ix.0.queue6.irqs: 7466940576 > >> dev.ix.0.queue6.txd_head: 1784 > >> dev.ix.0.queue6.txd_tail: 1784 > >> dev.ix.0.queue6.tso_tx: 2001 > >> dev.ix.0.queue6.no_tx_dma_setup: 0 > >> dev.ix.0.queue6.no_desc_avail: 0 > >> dev.ix.0.queue6.tx_packets: 9784312967 > >> dev.ix.0.queue6.rxd_head: 395 > >> dev.ix.0.queue6.rxd_tail: 394 > >> dev.ix.0.queue6.rx_packets: 13103079970 > >> dev.ix.0.queue6.rx_bytes: 3581485264 > >> dev.ix.0.queue6.rx_copies: 7336569 > >> dev.ix.0.queue6.lro_queued: 0 > >> dev.ix.0.queue6.lro_flushed: 0 > >> dev.ix.0.queue7.interrupt_rate: 5319 > >> dev.ix.0.queue7.irqs: 7486391989 > >> dev.ix.0.queue7.txd_head: 1549 > >> dev.ix.0.queue7.txd_tail: 1549 > >> dev.ix.0.queue7.tso_tx: 2052 > >> dev.ix.0.queue7.no_tx_dma_setup: 0 > >> dev.ix.0.queue7.no_desc_avail: 0 > >> dev.ix.0.queue7.tx_packets: 9758335509 > >> dev.ix.0.queue7.rxd_head: 1990 > >> dev.ix.0.queue7.rxd_tail: 1986 > >> dev.ix.0.queue7.rx_packets: 13141436559 > >> dev.ix.0.queue7.rx_bytes: 3877881546 > >> dev.ix.0.queue7.rx_copies: 8505244 > >> dev.ix.0.queue7.lro_queued: 0 > >> dev.ix.0.queue7.lro_flushed: 0 > >> dev.ix.0.mac_stats.crc_errs: 159 > >> dev.ix.0.mac_stats.ill_errs: 5 > >> dev.ix.0.mac_stats.byte_errs: 10 > >> dev.ix.0.mac_stats.short_discards: 0 > >> dev.ix.0.mac_stats.local_faults: 213 > >> dev.ix.0.mac_stats.remote_faults: 37 > >> dev.ix.0.mac_stats.rec_len_errs: 0 > >> dev.ix.0.mac_stats.xon_txd: 33623392288 > >> dev.ix.0.mac_stats.xon_recvd: 0 > >> dev.ix.0.mac_stats.xoff_txd: 110161 > >> dev.ix.0.mac_stats.xoff_recvd: 0 > >> dev.ix.0.mac_stats.total_octets_rcvd: 37967032023930 > >> dev.ix.0.mac_stats.good_octets_rcvd: 37965112076546 > >> dev.ix.0.mac_stats.total_pkts_rcvd: 105201314224 > >> dev.ix.0.mac_stats.good_pkts_rcvd: 18446743122750685300 > >> dev.ix.0.mac_stats.mcast_pkts_rcvd: 43835 > >> dev.ix.0.mac_stats.bcast_pkts_rcvd: 960566 > >> dev.ix.0.mac_stats.rx_frames_64: 454067 > >> dev.ix.0.mac_stats.rx_frames_65_127: 75350664570 > >> dev.ix.0.mac_stats.rx_frames_128_255: 4726321485 > >> dev.ix.0.mac_stats.rx_frames_256_511: 2282382994 > >> dev.ix.0.mac_stats.rx_frames_512_1023: 3293330145 > >> dev.ix.0.mac_stats.rx_frames_1024_1522: 19541190234 > >> dev.ix.0.mac_stats.recv_undersized: 0 > >> dev.ix.0.mac_stats.recv_fragmented: 0 > >> dev.ix.0.mac_stats.recv_oversized: 0 > >> dev.ix.0.mac_stats.recv_jabberd: 4 > >> dev.ix.0.mac_stats.management_pkts_rcvd: 0 > >> dev.ix.0.mac_stats.management_pkts_drpd: 0 > >> dev.ix.0.mac_stats.checksum_errs: 527716650 > >> dev.ix.0.mac_stats.good_octets_txd: 81355567084088 > >> dev.ix.0.mac_stats.total_pkts_txd: 81545370171 > >> dev.ix.0.mac_stats.good_pkts_txd: 47921867720 > >> dev.ix.0.mac_stats.bcast_pkts_txd: 298646 > >> dev.ix.0.mac_stats.mcast_pkts_txd: 18446744040086169062 > >> dev.ix.0.mac_stats.management_pkts_txd: 0 > >> dev.ix.0.mac_stats.tx_frames_64: 18446744045326220993 > >> dev.ix.0.mac_stats.tx_frames_65_127: 15629697019 > >> dev.ix.0.mac_stats.tx_frames_128_255: 4141756940 > >> dev.ix.0.mac_stats.tx_frames_256_511: 2195917491 > >> dev.ix.0.mac_stats.tx_frames_512_1023: 2717769904 > >> dev.ix.0.mac_stats.tx_frames_1024_1522: 51620056990 > >> > >> dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version > - > >> 2.5.15 > >> dev.ix.1.%driver: ix > >> dev.ix.1.%location: slot=0 function=1 handle=\_SB_.PCI1.BR48.S3F1 > >> dev.ix.1.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 > >> subdevice=0x7b11 class=0x020000 > >> dev.ix.1.%parent: pci131 > >> dev.ix.1.fc: 3 > >> dev.ix.1.enable_aim: 1 > >> dev.ix.1.advertise_speed: 0 > >> dev.ix.1.dropped: 0 > >> dev.ix.1.mbuf_defrag_failed: 0 > >> dev.ix.1.watchdog_events: 0 > >> dev.ix.1.link_irq: 46692 > >> dev.ix.1.queue0.interrupt_rate: 15151 > >> dev.ix.1.queue0.irqs: 2634341620 > >> dev.ix.1.queue0.txd_head: 1101 > >> dev.ix.1.queue0.txd_tail: 1101 > >> dev.ix.1.queue0.tso_tx: 0 > >> dev.ix.1.queue0.no_tx_dma_setup: 0 > >> dev.ix.1.queue0.no_desc_avail: 4142 > >> dev.ix.1.queue0.tx_packets: 4749431326 > >> dev.ix.1.queue0.rxd_head: 812 > >> dev.ix.1.queue0.rxd_tail: 811 > >> dev.ix.1.queue0.rx_packets: 21135516 > >> dev.ix.1.queue0.rx_bytes: 33229682 > >> dev.ix.1.queue0.rx_copies: 119434 > >> dev.ix.1.queue0.lro_queued: 0 > >> dev.ix.1.queue0.lro_flushed: 0 > >> dev.ix.1.queue1.interrupt_rate: 15151 > >> dev.ix.1.queue1.irqs: 2616347302 > >> dev.ix.1.queue1.txd_head: 797 > >> dev.ix.1.queue1.txd_tail: 797 > >> dev.ix.1.queue1.tso_tx: 0 > >> dev.ix.1.queue1.no_tx_dma_setup: 0 > >> dev.ix.1.queue1.no_desc_avail: 4129 > >> dev.ix.1.queue1.tx_packets: 4730356550 > >> dev.ix.1.queue1.rxd_head: 1345 > >> dev.ix.1.queue1.rxd_tail: 1344 > >> dev.ix.1.queue1.rx_packets: 20844457 > >> dev.ix.1.queue1.rx_bytes: 35014827 > >> dev.ix.1.queue1.rx_copies: 125167 > >> dev.ix.1.queue1.lro_queued: 0 > >> dev.ix.1.queue1.lro_flushed: 0 > >> dev.ix.1.queue2.interrupt_rate: 5555 > >> dev.ix.1.queue2.irqs: 2603166062 > >> dev.ix.1.queue2.txd_head: 1428 > >> dev.ix.1.queue2.txd_tail: 1432 > >> dev.ix.1.queue2.tso_tx: 0 > >> dev.ix.1.queue2.no_tx_dma_setup: 0 > >> dev.ix.1.queue2.no_desc_avail: 34368 > >> dev.ix.1.queue2.tx_packets: 4681339909 > >> dev.ix.1.queue2.rxd_head: 1153 > >> dev.ix.1.queue2.rxd_tail: 1152 > >> dev.ix.1.queue2.rx_packets: 21263862 > >> dev.ix.1.queue2.rx_bytes: 46251419 > >> dev.ix.1.queue2.rx_copies: 125156 > >> dev.ix.1.queue2.lro_queued: 0 > >> dev.ix.1.queue2.lro_flushed: 0 > >> dev.ix.1.queue3.interrupt_rate: 5319 > >> dev.ix.1.queue3.irqs: 2658483423 > >> dev.ix.1.queue3.txd_head: 179 > >> dev.ix.1.queue3.txd_tail: 179 > >> dev.ix.1.queue3.tso_tx: 0 > >> dev.ix.1.queue3.no_tx_dma_setup: 0 > >> dev.ix.1.queue3.no_desc_avail: 4189 > >> dev.ix.1.queue3.tx_packets: 4811705668 > >> dev.ix.1.queue3.rxd_head: 583 > >> dev.ix.1.queue3.rxd_tail: 582 > >> dev.ix.1.queue3.rx_packets: 20472051 > >> dev.ix.1.queue3.rx_bytes: 29823943 > >> dev.ix.1.queue3.rx_copies: 129450 > >> dev.ix.1.queue3.lro_queued: 0 > >> dev.ix.1.queue3.lro_flushed: 0 > >> dev.ix.1.queue4.interrupt_rate: 5319 > >> dev.ix.1.queue4.irqs: 2629295642 > >> dev.ix.1.queue4.txd_head: 592 > >> dev.ix.1.queue4.txd_tail: 594 > >> dev.ix.1.queue4.tso_tx: 0 > >> dev.ix.1.queue4.no_tx_dma_setup: 0 > >> dev.ix.1.queue4.no_desc_avail: 4106 > >> dev.ix.1.queue4.tx_packets: 4774679355 > >> dev.ix.1.queue4.rxd_head: 259 > >> dev.ix.1.queue4.rxd_tail: 258 > >> dev.ix.1.queue4.rx_packets: 22561633 > >> dev.ix.1.queue4.rx_bytes: 40196457 > >> dev.ix.1.queue4.rx_copies: 135966 > >> dev.ix.1.queue4.lro_queued: 0 > >> dev.ix.1.queue4.lro_flushed: 0 > >> dev.ix.1.queue5.interrupt_rate: 18518 > >> dev.ix.1.queue5.irqs: 2628129098 > >> dev.ix.1.queue5.txd_head: 1070 > >> dev.ix.1.queue5.txd_tail: 1072 > >> dev.ix.1.queue5.tso_tx: 0 > >> dev.ix.1.queue5.no_tx_dma_setup: 0 > >> dev.ix.1.queue5.no_desc_avail: 4127 > >> dev.ix.1.queue5.tx_packets: 4750613588 > >> dev.ix.1.queue5.rxd_head: 595 > >> dev.ix.1.queue5.rxd_tail: 594 > >> dev.ix.1.queue5.rx_packets: 21309940 > >> dev.ix.1.queue5.rx_bytes: 29264004 > >> dev.ix.1.queue5.rx_copies: 134128 > >> dev.ix.1.queue5.lro_queued: 0 > >> dev.ix.1.queue5.lro_flushed: 0 > >> dev.ix.1.queue6.interrupt_rate: 125000 > >> dev.ix.1.queue6.irqs: 2615444123 > >> dev.ix.1.queue6.txd_head: 1008 > >> dev.ix.1.queue6.txd_tail: 1008 > >> dev.ix.1.queue6.tso_tx: 0 > >> dev.ix.1.queue6.no_tx_dma_setup: 0 > >> dev.ix.1.queue6.no_desc_avail: 4106 > >> dev.ix.1.queue6.tx_packets: 4703519800 > >> dev.ix.1.queue6.rxd_head: 1209 > >> dev.ix.1.queue6.rxd_tail: 1208 > >> dev.ix.1.queue6.rx_packets: 22783319 > >> dev.ix.1.queue6.rx_bytes: 28091721 > >> dev.ix.1.queue6.rx_copies: 131653 > >> dev.ix.1.queue6.lro_queued: 0 > >> dev.ix.1.queue6.lro_flushed: 0 > >> dev.ix.1.queue7.interrupt_rate: 6024 > >> dev.ix.1.queue7.irqs: 2644640254 > >> dev.ix.1.queue7.txd_head: 1313 > >> dev.ix.1.queue7.txd_tail: 1313 > >> dev.ix.1.queue7.tso_tx: 0 > >> dev.ix.1.queue7.no_tx_dma_setup: 0 > >> dev.ix.1.queue7.no_desc_avail: 4106 > >> dev.ix.1.queue7.tx_packets: 4824584039 > >> dev.ix.1.queue7.rxd_head: 1463 > >> dev.ix.1.queue7.rxd_tail: 1462 > >> dev.ix.1.queue7.rx_packets: 21464418 > >> dev.ix.1.queue7.rx_bytes: 38133966 > >> dev.ix.1.queue7.rx_copies: 163180 > >> dev.ix.1.queue7.lro_queued: 0 > >> dev.ix.1.queue7.lro_flushed: 0 > >> dev.ix.1.mac_stats.crc_errs: 0 > >> dev.ix.1.mac_stats.ill_errs: 0 > >> dev.ix.1.mac_stats.byte_errs: 0 > >> dev.ix.1.mac_stats.short_discards: 0 > >> dev.ix.1.mac_stats.local_faults: 0 > >> dev.ix.1.mac_stats.remote_faults: 2 > >> dev.ix.1.mac_stats.rec_len_errs: 0 > >> dev.ix.1.mac_stats.xon_txd: 11207820571 > >> dev.ix.1.mac_stats.xon_recvd: 0 > >> dev.ix.1.mac_stats.xoff_txd: 3735958948 > >> dev.ix.1.mac_stats.xoff_recvd: 0 > >> dev.ix.1.mac_stats.total_octets_rcvd: 47828274751 > >> dev.ix.1.mac_stats.good_octets_rcvd: 47225376551 > >> dev.ix.1.mac_stats.total_pkts_rcvd: 177931572 > >> dev.ix.1.mac_stats.good_pkts_rcvd: 18446743954331576567 > >> dev.ix.1.mac_stats.mcast_pkts_rcvd: 7410 > >> dev.ix.1.mac_stats.bcast_pkts_rcvd: 8907 > >> dev.ix.1.mac_stats.rx_frames_64: 0 > >> dev.ix.1.mac_stats.rx_frames_65_127: 134619798 > >> dev.ix.1.mac_stats.rx_frames_128_255: 7996096 > >> dev.ix.1.mac_stats.rx_frames_256_511: 3996945 > >> dev.ix.1.mac_stats.rx_frames_512_1023: 5145938 > >> dev.ix.1.mac_stats.rx_frames_1024_1522: 21331154 > >> dev.ix.1.mac_stats.recv_undersized: 0 > >> dev.ix.1.mac_stats.recv_fragmented: 0 > >> dev.ix.1.mac_stats.recv_oversized: 0 > >> dev.ix.1.mac_stats.recv_jabberd: 0 > >> dev.ix.1.mac_stats.management_pkts_rcvd: 0 > >> dev.ix.1.mac_stats.management_pkts_drpd: 0 > >> dev.ix.1.mac_stats.checksum_errs: 219681 > >> dev.ix.1.mac_stats.good_octets_txd: 38874487735713 > >> dev.ix.1.mac_stats.total_pkts_txd: 38026147390 > >> dev.ix.1.mac_stats.good_pkts_txd: 27377335167 > >> dev.ix.1.mac_stats.bcast_pkts_txd: 67411 > >> dev.ix.1.mac_stats.mcast_pkts_txd: 18446744063060778699 > >> dev.ix.1.mac_stats.management_pkts_txd: 0 > >> dev.ix.1.mac_stats.tx_frames_64: 18446744065500352065 > >> dev.ix.1.mac_stats.tx_frames_65_127: 6755049420 > >> dev.ix.1.mac_stats.tx_frames_128_255: 1914523761 > >> dev.ix.1.mac_stats.tx_frames_256_511: 1038553757 > >> dev.ix.1.mac_stats.tx_frames_512_1023: 1240890942 > >> dev.ix.1.mac_stats.tx_frames_1024_1522: 24637516838 > >> > >> > >> Cheers, > >> > >> > >>>> -a > >>>> > >>>> > >>>> On 5 September 2014 07:35, Marcelo Gondim > wrote: > >>>> > >>>>> Hi Adrian, > >>>>> > >>>>> I confirmed with the support staff of the room where the server is, > >>>>> that the > >>>>> ambient temperature was normal. > >>>>> > >>>>> > >>>>> On 04/09/2014 22:46, Marcelo Gondim wrote: > >>>>> > >>>>>> On 04/09/2014 20:48, Adrian Chadd wrote: > >>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> The only time this has happened to me is because the card > overheated. > >>>>>>> Can you check that? > >>>>>>> > >>>>>> Hi Adrian, > >>>>>> > >>>>>> The room where the equipment is located is very cold but I'll check it > >>>>>> out. > >>>>>> Also seen at the time of the problem, a lot of dropped packets. > >>>>>> > >>>>>> # netstat -idn > >>>>>> ... > >>>>>> ix0 1500 a0:36:9f:2a:6d:ac 18446743423829095869 159 > >>>>>> 750924631703 53285910688 0 0 0 > >>>>>> ix0 - fe80::a236:9f fe80::a236:9fff:f 0 - - 2 > >>>>>> - - - > >>>>>> ix1 1500 a0:36:9f:2a:6d:ae 18446743954328745465 0 > >>>>>> 119550050209 20178077451 0 0 0 > >>>>>> ix1 - fe80::a236:9f fe80::a236:9fff:f 0 - - 1 > >>>>>> - - - > >>>>>> ... > >>>>>> > >>>>>> 119550050209 droped packets on ix1 and 750924631703 droped on ix0 > >>>>>> > >>>>>> Could be interesting I upgrade to10.1-PRERELEASE? > >>>>>> Could there be a problem with the driver? > >>>>>> > >>>>>> Traffic on ix0: 1.4Gbps output / 600Mbps input > >>>>>> Traffic on ix1: 1.2Gbps output > >>>>>> > >>>>>> PPS on ix0: 163Kpps output / 215Kpps input > >>>>>> PPS on ix1: 131Kpps output > >>>>>> > >>>>>> Thanks for your help. > >>>>>> > >>>>>>> > >>>>>>> -a > >>>>>>> > >>>>>>> > >>>>>>> On 4 September 2014 16:14, Marcelo Gondim > > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hi All, > >>>>>>>> > >>>>>>>> I have an Intel X520-SR2and today was working when all traffic > >>>>>>>> stopped. > >>>>>>>> I looked in the logs and found this message: > >>>>>>>> > >>>>>>>> Sep 4 18:29:53 rt01 kernel: ix1: > >>>>>>>> Sep 4 18:29:53 rt01 kernel: CRITICAL: ECC ERROR!! Please Reboot!! > >>>>>>>> > >>>>>>>> # uname -a > >>>>>>>> FreeBSD rt01.xxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #10 > >>>>>>>> r267839: > >>>>>>>> Thu > >>>>>>>> Jul 10 15:35:04 BRT 2014 > >>>>>>>> root@rt01.xxxxx.com.br:/usr/obj/usr/src/sys/GONDIM10 amd64 > >>>>>>>> > >>>>>>>> # netstat -m > >>>>>>>> 98324/53476/151800 mbufs in use (current/cache/total) > >>>>>>>> 98301/44951/143252/1014370 mbuf clusters in use > >>>>>>>> (current/cache/total/max) > >>>>>>>> 98301/44897 mbuf+clusters out of packet secondary zone in use > >>>>>>>> (current/cache) > >>>>>>>> 0/421/421/507184 4k (page size) jumbo clusters in use > >>>>>>>> (current/cache/total/max) > >>>>>>>> 0/0/0/150276 9k jumbo clusters in use (current/cache/total/max) > >>>>>>>> 0/0/0/84530 16k jumbo clusters in use (current/cache/total/max) > >>>>>>>> 221183K/104955K/326138K bytes allocated to network > >>>>>>>> (current/cache/total) > >>>>>>>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > >>>>>>>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > >>>>>>>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > >>>>>>>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > >>>>>>>> 0 requests for sfbufs denied > >>>>>>>> 0 requests for sfbufs delayed > >>>>>>>> 0 requests for I/O initiated by sendfile > >>>>>>>> > >>>>>>>> Best regards, > >>>>>>>> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 20:28:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2FA8258 for ; Mon, 8 Sep 2014 20:28:08 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A341317DD for ; Mon, 8 Sep 2014 20:28:08 +0000 (UTC) Received: from [192.168.200.200] (unknown [50.136.155.142]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 82E3F192906; Mon, 8 Sep 2014 20:28:01 +0000 (UTC) Subject: Re: ixgbe(4) spin lock held too long From: Sean Bruno Reply-To: sbruno@freebsd.org To: Eric van Gyzen In-Reply-To: <540E04AA.80806@vangyzen.net> References: <1410203348.1343.1.camel@bruno> <1410203965.1343.3.camel@bruno> <540E04AA.80806@vangyzen.net> Content-Type: text/plain; charset="us-ascii" Date: Mon, 08 Sep 2014 13:27:59 -0700 Message-ID: <1410208079.1343.5.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 20:28:08 -0000 On Mon, 2014-09-08 at 15:34 -0400, Eric van Gyzen wrote: > >> Unread portion of the kernel message buffer: > >> spin lock 0xffffffff812a0400 (callout) held by 0xfffff800151fe000 > (tid > >> 100003) too long > > TID 100003 is usually a kernel idle thread, which would seem to > indicate > a dangling lock. Can you enable WITNESS (without WITNESS_SKIPSPIN) on > this box? Will do. I'll report back when we get a crash with WITNESS sean From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 20:43:32 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90D7EF13; Mon, 8 Sep 2014 20:43:32 +0000 (UTC) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED17C1A0E; Mon, 8 Sep 2014 20:43:31 +0000 (UTC) Received: by mail-wi0-f178.google.com with SMTP id hi2so98493wib.17 for ; Mon, 08 Sep 2014 13:43:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:hackerspace:user-agent; bh=FHknFpHb3+jh+CQmg20xIxnvAWi/20B2OEO+rjmqzs4=; b=NB8dE+Gw0NiZldM85D7mBtt2V+tfzINtYY5yM1irhcAx0zdewQ2BWafw/NKyfv5a1b 4Jhjzt+qidDqVWlNeH/HMzc2k8fRHvMdCuFVAIKRRuO/UtK0P7JWFP061owR5BqOJ9ir 5cBrQOEhzrjfPAXCyUPA1ECrlpN3sW9txeyyEXsDOrIzEh4oQMj82NHsk1N5JlzV8dLd vQy+ro91oTKvOfhfNhjdWjHOEPY8T1gJNpSgOUDeWiSmOopnWxKX4dOTTcowvQ3giINU +U4DdV51SE1uqe4wqdmobQaXw8IW/o2V50eSe9tiCEIM1eQdN4sp+19UCRuv83RwMiIZ 7hxw== X-Received: by 10.194.84.69 with SMTP id w5mr38655242wjy.0.1410209010115; Mon, 08 Sep 2014 13:43:30 -0700 (PDT) Received: from gmail.com (host109-150-128-47.range109-150.btcentralplus.com. [109.150.128.47]) by mx.google.com with ESMTPSA id u8sm12981318wia.24.2014.09.08.13.43.28 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Sep 2014 13:43:29 -0700 (PDT) Sender: Tom Jones Date: Mon, 8 Sep 2014 21:43:24 +0100 From: Tom Jones To: George Neville-Neil Subject: Re: Patches for RFC6937 and draft-ietf-tcpm-newcwv-00 Message-ID: <20140908204322.GA16129@gmail.com> References: <814E0886-1B6B-4316-8BAB-684DAFDE1983@netapp.com> <20140826145517.GD12732@gmail.com> <76D986F7-72A8-4ABE-8731-064C6C77A56F@netapp.com> <3ABE6D0D-1D98-425B-BDAD-8D1B9026AD8F@netapp.com> <4F10C5BC-03B2-486D-A5EF-C55582869ED1@netapp.com> <1BF6E077-0B47-41C9-8B47-848E048517B4@neville-neil.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="C7zPtVaVf+AK4Oqc" Content-Disposition: inline In-Reply-To: <1BF6E077-0B47-41C9-8B47-848E048517B4@neville-neil.com> Hackerspace: 57North Hacklab User-Agent: Mutt/1.5.23 (2014-03-12) Cc: FreeBSD Net , Adrian Chadd , "Eggert, Lars" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 20:43:32 -0000 --C7zPtVaVf+AK4Oqc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Folks, On Thu, Aug 28, 2014 at 12:39:09PM -0400, George Neville-Neil wrote: > Adrian, >=20 > Can you put this into a Phabricator for review? >=20 > Lars, >=20 > How have you been testing this? Did the newcwv patch make its way into Phabricator? I don't think I would h= ave seen if it did. >=20 > On 27 Aug 2014, at 4:01, Eggert, Lars wrote: >=20 > > Yep > > > > On 2014-8-27, at 9:53, Adrian Chadd wrote: > > > >> Ok. Is it the same patch you sent out in Feb? > >> > >> > >> -a > >> > >> > >> On 27 August 2014 00:43, Eggert, Lars wrote: > >>> Not as far as I know. > >>> > >>> Lars > >>> > >>> On 2014-8-27, at 9:39, Adrian Chadd wrote: > >>> > >>>> Is there a PR for it? > >>>> > >>>> > >>>> -a > >>>> > >>>> > >>>> On 27 August 2014 00:23, Eggert, Lars wrote: > >>>>> It would be great if people could also review Aris' PRR patch - RFC= 6937 has been out for a while. > >>>>> > >>>>> Lars > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On 2014-8-26, at 20:09, Adrian Chadd wrote: > >>>>> > >>>>>> Hi! > >>>>>> > >>>>>> I'm going to merge Tom's work in a week unless someone gives me a > >>>>>> really good reason not to. > >>>>>> > >>>>>> I think there's been enough work and discussion about it since the > >>>>>> first post from Lars in Feburary and enough review opportunity. > >>>>>> > >>>>>> > >>>>>> -a > >>>>>> > >>>>>> > >>>>>> On 26 August 2014 07:55, Tom Jones wrote: > >>>>>>> On Tue, Aug 26, 2014 at 02:43:49PM +0000, Eggert, Lars wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> the newcwv patch is probably stale now with Tom Jones' recent pa= tch based on > >>>>>>>> a more up-to-date version of the Internet-Draft, but the PRR pat= ch should > >>>>>>>> still be useful? > >>>>>>> > >>>>>>> My newcwv patch is much more up to date than Aris's, but it is sl= ightly > >>>>>>> different in implementation. I have had a few suggestions from Ad= rian, but he > >>>>>>> couldn't comment on how it relates to the tcp internals. > >>>>>>> > >>>>>>> There is a PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id= =3D191520 > >>>>>>> > >>>>>>> The biggest difference in structure between mine and Aris's patch= is the use of > >>>>>>> tcp timers. It would be good to hear if my approach or Aris's is = prefered. > >>>>>>> > >>>>>>>> On 2014-6-19, at 23:35, George Neville-Neil wrote: > >>>>>>>> > >>>>>>>>> On 4 Feb 2014, at 1:38, Eggert, Lars wrote: > >>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> below are two patches that implement RFC6937 ("Proportional Ra= te Reduction for TCP") and draft-ietf-tcpm-newcwv-00 ("Updating TCP to supp= ort Rate-Limited Traffic"). They were done by Aris Angelogiannopoulos for h= is MS thesis, which is at https://eggert.org/students/angelogiannopoulos-th= esis.pdf. > >>>>>>>>>> > >>>>>>>>>> The patches should apply to -CURRENT as of Sep 17, 2013. (Sorr= y for the delay in sending them, we'd been trying to get some feedback from= committers first, without luck.) > >>>>>>>>>> > >>>>>>>>>> Please note that newcwv is still a work in progress in the IET= F, and the patch has some limitations with regards to the "pipeACK Sampling= Period" mentioned in the Internet-Draft. Aris says this in his thesis abou= t what exactly he implemented: > >>>>>>>>>> > >>>>>>>>>> "The second implementation choice, is in regards with the meas= urement of pipeACK. This variable is the most important introduced by the m= ethod and is used to compute the phase that the sender currently lies in. I= n order to compute pipeACK the approach suggested by the Internet Draft (ID= ) is followed [ncwv]. During initialization, pipeACK is set to the maximum = possible value. A helper variable prevHighACK is introduced that is initial= ized to the initial sequence number (iss). prevHighACK holds the value of t= he highest acknowledged byte so far. pipeACK is measured once per RTT meani= ng that when an ACK covering prevHighACK is received, pipeACK becomes the d= ifference between the current ACK and prevHighACK. This is called a pipeACK= sample. A newer version of the draft suggests that multiple pipeACK sampl= es can be used during the pipeACK sampling period." > >>>>>>>>>> > >>>>>>>>>> Lars > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> [prr.patch] > >>>>>>>>>> > >>>>>>>>>> [newcwv.patch] > >>>>>>>>> > >>>>>>>>> Apologies for not looking at this as yet. It is now closer to = the top of my list. > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> George > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Tom > >>>>>>> @adventureloop > >>>>>>> adventurist.me > >>>>>>> > >>>>>>> :wq > >>>>>>> _______________________________________________ > >>>>>>> freebsd-net@freebsd.org mailing list > >>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd= =2Eorg" > >>>>> > >>>>> > >>> --=20 Tom @adventureloop adventurist.me #pragma summon cthulhu :wq --C7zPtVaVf+AK4Oqc Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - https://gpgtools.org iQIcBAABCgAGBQJUDhTqAAoJEDetXgJOHpB8fzwP/AqV5GMjRMm5YJfTeUSYue2d TjJ9cMl+7tB64WDOM07Ugs6doO4dplpKHy4Ek9vdfhg5MVt5ZarW+bNT6tAfmcP/ rBCsltMgGqS5c4CtHP0knHA8ekfSpsGtOln+sfqwVyezl2Lbw+BDGWYZ1eKnN5sW cawm8ICTqOA9VSbRRYVPYeJdmia2SIcySz+DC/1g3o2qx8Dh2kDI6czli4XR/vLR mVBxXf2czo94cfpRxxQkCmEiSD7JiXQjGeglGYvmtAoQraM1ocEuTbbTtuAVNMN2 mTjOrEIW6eSKNX6HN/HNeGSH4Idv4eYoYdDsyz/RhydycutOdXjyvM5jaW5MMDee pAumFxoX+YNOuujUjzDiXVFnwpXyI7jEtOgpFUfY2jVXl03QfnJszsQNrucm/isT tkVAp96Pzx+wNhoZQw9+ghA77nZZRk1/klDtMwLi7F3ACtKg8SP1WQg8yyUt+3aO vSLQcb+MMabGofNRRpHtcG3ZnRT3nc3eSeUvPGxVo8CQpPNE2kaB2TNNYkqnpYab 5Or4oh/u+ph2FKSESDURyRvNEf6epzFUU1WmyCvobADDJw5hYKF9zi7+QgZKEb2O ga5ASpIO4kZi5XZGabkP8R+MQ22LHoCTtqKDFIp4DKvFLPHhOw5/Tw/3iB9SCf1k bh5kfnvX0OUTqxNMtEOu =8JFh -----END PGP SIGNATURE----- --C7zPtVaVf+AK4Oqc-- From owner-freebsd-net@FreeBSD.ORG Mon Sep 8 23:33:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA14623A; Mon, 8 Sep 2014 23:33:12 +0000 (UTC) Received: from mail-pa0-x236.google.com (mail-pa0-x236.google.com [IPv6:2607:f8b0:400e:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9347A6D5; Mon, 8 Sep 2014 23:33:12 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id lj1so4892951pab.27 for ; Mon, 08 Sep 2014 16:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=GyaN/nDGqIp/YKJOHkCqeBf4ynQViDu25B9P/rD03v8=; b=GMnkyKufNt3tTvsYj5bILeiv0kKft7URJqI6VsODXwCX0pQrk/Ib3lw2CU5GXwsh6u 4cToEQKSPRV3pZ845Fi3TJTu+9H0/o5iGK/pvZC9dZVyHCnkH/tQUBvZre0gPesXfR99 ZVNS30HTsDqSHYcOwMgni+9y61dAcokPuiwNjbgqBxIvDk86RE747bvn3/RIkwkr7JUb q3aJwRpwzHZzsJWoxSWP41x4KUvgQpB1RshOV19yEbojgq5FLb+67J87Wgb6/XguqSAl M+hF3uwg9ueQ+Jz/hpJGknrvOy0B7P7HfCIL5Yu3vvEqPI3duxafa6XJGTfwaGet6SnG IKFA== X-Received: by 10.70.47.42 with SMTP id a10mr12368287pdn.14.1410219192067; Mon, 08 Sep 2014 16:33:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.70.55.162 with HTTP; Mon, 8 Sep 2014 16:32:31 -0700 (PDT) In-Reply-To: <1882852102.33387181.1410177939640.JavaMail.root@uoguelph.ca> References: <540D444C.6090904@selasky.org> <1882852102.33387181.1410177939640.JavaMail.root@uoguelph.ca> From: Eric Joyner Date: Mon, 8 Sep 2014 16:32:31 -0700 Message-ID: Subject: Re: [RFC] Patch to improve TSO limitation formula in general To: Rick Macklem Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Hans Petter Selasky , "freebsd-net@freebsd.org" , FreeBSD Current , Scott Long , Jack F Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 23:33:13 -0000 Let me remove my concerns earlier in the thread -- this patch won't negatively affect any of our drivers; and the problem I mentioned with ixl would require a change somewhere further up the stack. --- - Eric Joyner On Mon, Sep 8, 2014 at 5:05 AM, Rick Macklem wrote: > Hans Petter Selasky wrote: > > On 09/06/14 00:09, Rick Macklem wrote: > > > Hans Petter Selesky wrote: > > >> On 09/05/14 23:19, Eric Joyner wrote: > > >>> There are some concerns if we use this with devices that ixl > > >>> supports: > > >>> > > >>> - The maximum fragment size is 16KB-1, which isn't a power of 2. > > >>> > > >> > > >> Hi Eric, > > >> > > >> Multiplying by powers of two are more fast, than non-powers of > > >> two. > > >> So > > >> in this case you would have to use 8KB as a maximum. > > >> > > > Well, I'm no architecture expert, but I really doubt the CPU delay > > > of a > > > non-power of 2 multiply/divide is significant related to doing > > > smaller > > > TSO segments. Long ago (as in 1970s) I did work on machines where > > > shifts > > > for power of 2 multiply/divide was preferable, but these days I > > > doubt it > > > is going to matter?? > > > > > >>> - You can't get the maximum TSO size for ixl devices by > > >>> multiplying > > >>> the > > >>> maximum number of fragments by the maximum size. > > >>> Instead the number of fragments is AFAIK unlimited, but a segment > > >>> can only > > >>> span 8 mbufs (including the [up to 3] mbufs containing the > > >>> header), > > >>> and the > > >>> maximum TSO size is 256KB. > > > > Hi, > > > > Maybe that can be a separate parameter? > > > > I see that your patch assumes that a segment can be any-length. That > > is > > not always the case. Remember there are JUMBO mbufs too. > > > I thought JUMBO mbufs were only going to be used on the receive side, > however I suppose if a packet is received into a JUMBO mbuf and then > forwarded on another interface... > > > With my patch, the maximum segment size is a separate parameter. The > > total number of TSO bytes is then not so useful. > > > Well, if you are referring to if_hw_tsomax, I'm not sure it was the > best plan to begin with. It was implemented for xen and I'm not sure > that any other driver uses it as of now. > > However... > I'm not a network/hardware guy, but it seems some devices do have > the IP_MAXPACKET limit (use the ip_len field in the ip header to > know how large the TSO segment is). There is also at least one device > (82598 chip for "ix" driver) that can handle more that IP_MAXPACKET, > so it seems appropriate to have a value that the driver can set. > > Since the maximum size of the gather list for transmit does seem to > vary a lot between devices, with several handling less than 35, it > does seem appropriate to allow drivers to specify that. > > If devices can't handle a single gather entry over a certain size, > I think that does need to be specified along with the max size of > the gather list, since the driver will need to use multiple gather > entries for a single mbuf and tcp_output() should take that into > account. > > In summary, yep, I basically agree. > > rick > ps: Who will look at your patch soon. > > > --HPS > > > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to > > "freebsd-current-unsubscribe@freebsd.org" > > > From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 01:51:50 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DE6C772 for ; Tue, 9 Sep 2014 01:51:50 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2CB81142 for ; Tue, 9 Sep 2014 01:14:44 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id 294D6139D0 for ; Mon, 8 Sep 2014 22:15:24 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1410225322; x=1411089323; bh=/FDBSed5T92sHTSu62qLzHZBq+6gBS789AC 61GMfYsA=; b=HsipSY0cUhMXRV35CfnhJ1i3xv52M8qZNojI/mRnEXkPQxcS5N8 tolpGauwpOCItc5fe5tPYUV85jzd5lMuKJsQrDnvDnU7+WZhOgyI9VWQWMp+HCzv vBpdzNVo7SX333q8DvmtO3xfdZr0AzPr8SVKcCBftRRI4vQfz5xaAovM= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7acUUAEhO2sv for ; Mon, 8 Sep 2014 22:15:22 -0300 (BRT) Received: from [192.168.10.208] (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id F077B139CA; Mon, 8 Sep 2014 22:15:18 -0300 (BRT) Message-ID: <540E547C.9030703@bsdinfo.com.br> Date: Mon, 08 Sep 2014 22:14:36 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: =?UTF-8?B?IkluZy4gQWxlxaEgTmVjaHbDoXRhbCI=?= , 'Eric Joyner' Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! References: <5408F23C.2030309@bsdinfo.com.br> <54091607.9010100@bsdinfo.com.br> <5409CA44.8070203@bsdinfo.com.br> <540A1A3E.2040306@bsdinfo.com.br> <540A8200.5010404@bsdinfo.com.br> <540DFAE3.8070001@bsdinfo.com.br> <002001cfcba0$4812c7f0$d83857d0$@videon-znojmo.cz> In-Reply-To: <002001cfcba0$4812c7f0$d83857d0$@videon-znojmo.cz> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: 'Jack F Vogel' , 'FreeBSD Net' , 'Adrian Chadd' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 01:51:50 -0000 On 08/09/2014 17:05, Ing. AleÅ¡ Nechvátal wrote: > I was experiencing similar behaviour on a machine acting as a gateway. It > has one X520-SR2, both SPF+ ports used to connet to another machines with > the same NICs. One of the NICs suddenly stopped to pass data through and the > the ifconfing down up cycle was needed to make it work again, that in better > case. In the worse case, the machine rebooted without leaving any message in > syslog (it's on a remote site). This happened in intervals of 5 mins. up to > 1 hour (I am not sure about the exact correlation between traffic going > through and the number of these events, but during traffic peaks, it seems > to have happened more often). > > First I checked the mbufs, but didn't see any problem. Then I tried to > disable TSO (on both sides of the links), but without observing any > improvement. > I was playing with different versions of the kernel, the newer the kernel (I > went up to 9.3 STABLE, didn't try 10 or HEAD) the more instability. It > seemed, that the most stable was 9.1 PRERELEASE with 2.4.8 driver (at least > I didn't see reboots, a cron script could keep the gateway passing data by > shutting down and bringing up the interfaces). Now I am running 9.3 RELEASE > r271227M with the 2.5.25 driver from Intel's website. This configuration was > not stable either, until I discovered high interrupt rate on each queue (the > sum on all queues reached over 400k in some moments). I turned off the > limit of interrupts by setting hw.intr_storm_threshold=0 (previously set to > 150000). Since the following reboot (involuntary, btw.) the gateway keeps > running smoothly for 7 hours now with peak throughput of approx. 1.6 Gbit/s. > > The sum of interrupt rates on all your queues looks quite high too, so there > could be some similarity and disabling the mentioned limit could help. > > My sysctl.conf: > kern.ipc.somaxconn=1024 > net.inet.icmp.icmplim=100 > net.inet.tcp.blackhole=2 > net.inet.udp.blackhole=1 > net.inet.tcp.sendspace=262144 > net.inet.tcp.recvspace=262144 > net.inet.ip.fastforwarding=1 > kern.ipc.maxsockbuf=16777216 > net.inet.tcp.sendbuf_max=16777216 > net.inet.tcp.recvbuf_max=16777216 > hw.intr_storm_threshold=0 > > #sysctl -a dev.ix.1 I changed the value of "hw.intr_storm_threshold" to 0 and yet the interface stopped working. :( my sysctl.conf: net.inet.ip.forwarding=1 net.inet.ip.fastforwarding=1 net.inet6.ip6.forwarding=1 kern.ipc.somaxconn=4096 net.inet.tcp.syncookies=1 net.inet.ip.redirect=1 net.inet.ip.accept_sourceroute=0 net.inet.ip.sourceroute=0 net.inet.tcp.drop_synfin=1 net.inet.udp.blackhole=1 net.inet.tcp.blackhole=2 security.bsd.see_other_uids=0 net.inet.ip.fw.dyn_buckets=65536 net.inet.ip.fw.dyn_max=65536 hw.intr_storm_threshold=0 net.inet.ip.dummynet.pipe_slot_limit=800 net.inet.icmp.icmplim=2000 Cheers, > > dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - > 2.5.25 > dev.ix.1.%driver: ix > dev.ix.1.%location: slot=0 function=1 > dev.ix.1.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 > subdevice=0x7a11 class=0x020000 > dev.ix.1.%parent: pci1 > dev.ix.1.fc: 3 > dev.ix.1.enable_aim: 1 > dev.ix.1.advertise_speed: 0 > dev.ix.1.dropped: 0 > dev.ix.1.mbuf_defrag_failed: 0 > dev.ix.1.watchdog_events: 0 > dev.ix.1.link_irq: 2 > dev.ix.1.queue0.interrupt_rate: 31250 > dev.ix.1.queue0.irqs: 152182132 > dev.ix.1.queue0.txd_head: 526 > dev.ix.1.queue0.txd_tail: 526 > dev.ix.1.queue0.tso_tx: 2 > dev.ix.1.queue0.no_tx_dma_setup: 0 > dev.ix.1.queue0.no_desc_avail: 119373 > dev.ix.1.queue0.tx_packets: 274418209 > dev.ix.1.queue0.rxd_head: 441 > dev.ix.1.queue0.rxd_tail: 440 > dev.ix.1.queue0.rx_packets: 103852222 > dev.ix.1.queue0.rx_bytes: 25483398737 > dev.ix.1.queue0.rx_copies: 87237192 > dev.ix.1.queue0.lro_queued: 0 > dev.ix.1.queue0.lro_flushed: 0 > dev.ix.1.queue1.interrupt_rate: 31250 > dev.ix.1.queue1.irqs: 152973172 > dev.ix.1.queue1.txd_head: 1282 > dev.ix.1.queue1.txd_tail: 1282 > dev.ix.1.queue1.tso_tx: 50 > dev.ix.1.queue1.no_tx_dma_setup: 0 > dev.ix.1.queue1.no_desc_avail: 114840 > dev.ix.1.queue1.tx_packets: 271827311 > dev.ix.1.queue1.rxd_head: 495 > dev.ix.1.queue1.rxd_tail: 494 > dev.ix.1.queue1.rx_packets: 103292003 > dev.ix.1.queue1.rx_bytes: 20681070454 > dev.ix.1.queue1.rx_copies: 89490795 > dev.ix.1.queue1.lro_queued: 0 > dev.ix.1.queue1.lro_flushed: 0 > dev.ix.1.queue2.interrupt_rate: 21739 > dev.ix.1.queue2.irqs: 81502157 > dev.ix.1.queue2.txd_head: 313 > dev.ix.1.queue2.txd_tail: 313 > dev.ix.1.queue2.tso_tx: 50 > dev.ix.1.queue2.no_tx_dma_setup: 0 > dev.ix.1.queue2.no_desc_avail: 111923 > dev.ix.1.queue2.tx_packets: 23189156 > dev.ix.1.queue2.rxd_head: 349 > dev.ix.1.queue2.rxd_tail: 348 > dev.ix.1.queue2.rx_packets: 94457797 > dev.ix.1.queue2.rx_bytes: 18053240855 > dev.ix.1.queue2.rx_copies: 82430878 > dev.ix.1.queue2.lro_queued: 0 > dev.ix.1.queue2.lro_flushed: 0 > dev.ix.1.queue3.interrupt_rate: 83333 > dev.ix.1.queue3.irqs: 81678068 > dev.ix.1.queue3.txd_head: 1990 > dev.ix.1.queue3.txd_tail: 1990 > dev.ix.1.queue3.tso_tx: 3 > dev.ix.1.queue3.no_tx_dma_setup: 0 > dev.ix.1.queue3.no_desc_avail: 110545 > dev.ix.1.queue3.tx_packets: 21258249 > dev.ix.1.queue3.rxd_head: 1544 > dev.ix.1.queue3.rxd_tail: 1543 > dev.ix.1.queue3.rx_packets: 99140619 > dev.ix.1.queue3.rx_bytes: 19476001717 > dev.ix.1.queue3.rx_copies: 86228489 > dev.ix.1.queue3.lro_queued: 0 > dev.ix.1.queue3.lro_flushed: 0 > dev.ix.1.mac_stats.crc_errs: 0 > dev.ix.1.mac_stats.ill_errs: 0 > dev.ix.1.mac_stats.byte_errs: 0 > dev.ix.1.mac_stats.short_discards: 0 > dev.ix.1.mac_stats.local_faults: 0 > dev.ix.1.mac_stats.remote_faults: 2 > dev.ix.1.mac_stats.rec_len_errs: 0 > dev.ix.1.mac_stats.xon_txd: 0 > dev.ix.1.mac_stats.xon_recvd: 0 > dev.ix.1.mac_stats.xoff_txd: 0 > dev.ix.1.mac_stats.xoff_recvd: 0 > dev.ix.1.mac_stats.total_octets_rcvd: 85345731961 > dev.ix.1.mac_stats.good_octets_rcvd: 85345731961 > dev.ix.1.mac_stats.total_pkts_rcvd: 400734734 > dev.ix.1.mac_stats.good_pkts_rcvd: 400734734 > dev.ix.1.mac_stats.mcast_pkts_rcvd: 0 > dev.ix.1.mac_stats.bcast_pkts_rcvd: 0 > dev.ix.1.mac_stats.rx_frames_64: 190985269 > dev.ix.1.mac_stats.rx_frames_65_127: 142879679 > dev.ix.1.mac_stats.rx_frames_128_255: 18114498 > dev.ix.1.mac_stats.rx_frames_256_511: 5882972 > dev.ix.1.mac_stats.rx_frames_512_1023: 7407080 > dev.ix.1.mac_stats.rx_frames_1024_1522: 35465236 > dev.ix.1.mac_stats.recv_undersized: 0 > dev.ix.1.mac_stats.recv_fragmented: 0 > dev.ix.1.mac_stats.recv_oversized: 0 > dev.ix.1.mac_stats.recv_jabberd: 0 > dev.ix.1.mac_stats.management_pkts_rcvd: 0 > dev.ix.1.mac_stats.management_pkts_drpd: 0 > dev.ix.1.mac_stats.checksum_errs: 462621 > dev.ix.1.mac_stats.good_octets_txd: 705061464268 > dev.ix.1.mac_stats.total_pkts_txd: 590658400 > dev.ix.1.mac_stats.good_pkts_txd: 590658400 > dev.ix.1.mac_stats.bcast_pkts_txd: 2907 > dev.ix.1.mac_stats.mcast_pkts_txd: 0 > dev.ix.1.mac_stats.management_pkts_txd: 0 > dev.ix.1.mac_stats.tx_frames_64: 24533923 > dev.ix.1.mac_stats.tx_frames_65_127: 65221733 > dev.ix.1.mac_stats.tx_frames_128_255: 24274012 > dev.ix.1.mac_stats.tx_frames_256_511: 12092320 > dev.ix.1.mac_stats.tx_frames_512_1023: 10358523 > dev.ix.1.mac_stats.tx_frames_1024_1522: 454177889 > >> -----Original Message----- >> From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd- >> net@freebsd.org] On Behalf Of Marcelo Gondim >> Sent: Monday, September 08, 2014 8:52 PM >> To: Eric Joyner >> Cc: Jack F Vogel; FreeBSD Net; Adrian Chadd >> Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! >> >> On 08/09/2014 15:38, Eric Joyner wrote: >>> Getting local / remote faults is strange; are these stats from after you >>> rebooted? >> When the crash happens I just do: >> >> # ifconfig ix0 down; ifconfig ix0 up; ifconfig ix1 down; ifconfig up ixi1 >> >> After that the two interface ports return to work. >> >> In no time I had to restart the server. >> >> # uptime >> 3:51PM up 11 days, 15:44, 2 users, load averages: 5.36, 5.68, 5.69 >> >> Tomorrow will be going to the Datacenter to check on the temperature of >> the network interface. As the crashes occur when traffic is high, I >> believe that the temperature increases because processing is high. >> >> >>> --- >>> - Eric Joyner >>> >>> On Fri, Sep 5, 2014 at 8:39 PM, Marcelo Gondim >>> wrote: >>> >>>> On 05/09/2014 17:17, Marcelo Gondim wrote: >>>> >>>>> On 05/09/2014 16:49, Adrian Chadd wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> But is the airflow in the unit sufficient? >>>>>> >>>>>> I had this problem at a previous job - the box was running fine, the >>>>>> room was very cold, but the internal fans in the server were set to >>>>>> "be very quiet". It wasn't enough to keep the ixgbe NICs happy. I had >>>>>> to change the fan settings to "just always run full speed". >>>>>> >>>>>> The fan temperature feedback loop was based on sensors on the CPU, >>>>>> _not_ on the peripherals. >>>>>> >>>>> Hi Adrian, >>>>> >>>>> Ummm. I'll check it and improve internal cooling. :) >>>>> She is not happy and I'm also not. rsrsrsr >>>>> >>>>> Cheers, >>>>> >>>> Besides the problem of heating of the network interface, I am putting >> some >>>> information here. Could you tell me if there is something strange or is > it >>>> normal? >>>> >>>> dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version >> - >>>> 2.5.15 >>>> dev.ix.0.%driver: ix >>>> dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0 >>>> dev.ix.0.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 >>>> subdevice=0x7b11 class=0x020000 >>>> dev.ix.0.%parent: pci131 >>>> dev.ix.0.fc: 3 >>>> dev.ix.0.enable_aim: 1 >>>> dev.ix.0.advertise_speed: 0 >>>> dev.ix.0.dropped: 0 >>>> dev.ix.0.mbuf_defrag_failed: 0 >>>> dev.ix.0.watchdog_events: 0 >>>> dev.ix.0.link_irq: 121769 >>>> dev.ix.0.queue0.interrupt_rate: 5319 >>>> dev.ix.0.queue0.irqs: 7900830877 >>>> dev.ix.0.queue0.txd_head: 1037 >>>> dev.ix.0.queue0.txd_tail: 1037 >>>> dev.ix.0.queue0.tso_tx: 142 >>>> dev.ix.0.queue0.no_tx_dma_setup: 0 >>>> dev.ix.0.queue0.no_desc_avail: 0 >>>> dev.ix.0.queue0.tx_packets: 9725701450 >>>> dev.ix.0.queue0.rxd_head: 1175 >>>> dev.ix.0.queue0.rxd_tail: 1174 >>>> dev.ix.0.queue0.rx_packets: 13069276955 >>>> dev.ix.0.queue0.rx_bytes: 3391061018 >>>> dev.ix.0.queue0.rx_copies: 8574407 >>>> dev.ix.0.queue0.lro_queued: 0 >>>> dev.ix.0.queue0.lro_flushed: 0 >>>> dev.ix.0.queue1.interrupt_rate: 41666 >>>> dev.ix.0.queue1.irqs: 7681141208 >>>> dev.ix.0.queue1.txd_head: 219 >>>> dev.ix.0.queue1.txd_tail: 221 >>>> dev.ix.0.queue1.tso_tx: 57 >>>> dev.ix.0.queue1.no_tx_dma_setup: 0 >>>> dev.ix.0.queue1.no_desc_avail: 44334 >>>> dev.ix.0.queue1.tx_packets: 10196891433 >>>> dev.ix.0.queue1.rxd_head: 1988 >>>> dev.ix.0.queue1.rxd_tail: 1987 >>>> dev.ix.0.queue1.rx_packets: 13210132242 >>>> dev.ix.0.queue1.rx_bytes: 4317357059 >>>> dev.ix.0.queue1.rx_copies: 8131936 >>>> dev.ix.0.queue1.lro_queued: 0 >>>> dev.ix.0.queue1.lro_flushed: 0 >>>> dev.ix.0.queue2.interrupt_rate: 5319 >>>> dev.ix.0.queue2.irqs: 7647486080 >>>> dev.ix.0.queue2.txd_head: 761 >>>> dev.ix.0.queue2.txd_tail: 761 >>>> dev.ix.0.queue2.tso_tx: 409 >>>> dev.ix.0.queue2.no_tx_dma_setup: 0 >>>> dev.ix.0.queue2.no_desc_avail: 54207 >>>> dev.ix.0.queue2.tx_packets: 10161246425 >>>> dev.ix.0.queue2.rxd_head: 1874 >>>> dev.ix.0.queue2.rxd_tail: 1872 >>>> dev.ix.0.queue2.rx_packets: 13175551880 >>>> dev.ix.0.queue2.rx_bytes: 4472798418 >>>> dev.ix.0.queue2.rx_copies: 7488876 >>>> dev.ix.0.queue2.lro_queued: 0 >>>> dev.ix.0.queue2.lro_flushed: 0 >>>> dev.ix.0.queue3.interrupt_rate: 500000 >>>> dev.ix.0.queue3.irqs: 7641129521 >>>> dev.ix.0.queue3.txd_head: 2039 >>>> dev.ix.0.queue3.txd_tail: 2039 >>>> dev.ix.0.queue3.tso_tx: 9 >>>> dev.ix.0.queue3.no_tx_dma_setup: 0 >>>> dev.ix.0.queue3.no_desc_avail: 150346 >>>> dev.ix.0.queue3.tx_packets: 10619971896 >>>> dev.ix.0.queue3.rxd_head: 1055 >>>> dev.ix.0.queue3.rxd_tail: 1054 >>>> dev.ix.0.queue3.rx_packets: 13137835529 >>>> dev.ix.0.queue3.rx_bytes: 4063197306 >>>> dev.ix.0.queue3.rx_copies: 8188713 >>>> dev.ix.0.queue3.lro_queued: 0 >>>> dev.ix.0.queue3.lro_flushed: 0 >>>> dev.ix.0.queue4.interrupt_rate: 5319 >>>> dev.ix.0.queue4.irqs: 7439824996 >>>> dev.ix.0.queue4.txd_head: 26 >>>> dev.ix.0.queue4.txd_tail: 26 >>>> dev.ix.0.queue4.tso_tx: 553912 >>>> dev.ix.0.queue4.no_tx_dma_setup: 0 >>>> dev.ix.0.queue4.no_desc_avail: 0 >>>> dev.ix.0.queue4.tx_packets: 10658683718 >>>> dev.ix.0.queue4.rxd_head: 684 >>>> dev.ix.0.queue4.rxd_tail: 681 >>>> dev.ix.0.queue4.rx_packets: 13204786830 >>>> dev.ix.0.queue4.rx_bytes: 3700845239 >>>> dev.ix.0.queue4.rx_copies: 8193379 >>>> dev.ix.0.queue4.lro_queued: 0 >>>> dev.ix.0.queue4.lro_flushed: 0 >>>> dev.ix.0.queue5.interrupt_rate: 15151 >>>> dev.ix.0.queue5.irqs: 7456613396 >>>> dev.ix.0.queue5.txd_head: 603 >>>> dev.ix.0.queue5.txd_tail: 603 >>>> dev.ix.0.queue5.tso_tx: 17 >>>> dev.ix.0.queue5.no_tx_dma_setup: 0 >>>> dev.ix.0.queue5.no_desc_avail: 0 >>>> dev.ix.0.queue5.tx_packets: 10639139790 >>>> dev.ix.0.queue5.rxd_head: 404 >>>> dev.ix.0.queue5.rxd_tail: 403 >>>> dev.ix.0.queue5.rx_packets: 13144301293 >>>> dev.ix.0.queue5.rx_bytes: 3986784766 >>>> dev.ix.0.queue5.rx_copies: 8256195 >>>> dev.ix.0.queue5.lro_queued: 0 >>>> dev.ix.0.queue5.lro_flushed: 0 >>>> dev.ix.0.queue6.interrupt_rate: 125000 >>>> dev.ix.0.queue6.irqs: 7466940576 >>>> dev.ix.0.queue6.txd_head: 1784 >>>> dev.ix.0.queue6.txd_tail: 1784 >>>> dev.ix.0.queue6.tso_tx: 2001 >>>> dev.ix.0.queue6.no_tx_dma_setup: 0 >>>> dev.ix.0.queue6.no_desc_avail: 0 >>>> dev.ix.0.queue6.tx_packets: 9784312967 >>>> dev.ix.0.queue6.rxd_head: 395 >>>> dev.ix.0.queue6.rxd_tail: 394 >>>> dev.ix.0.queue6.rx_packets: 13103079970 >>>> dev.ix.0.queue6.rx_bytes: 3581485264 >>>> dev.ix.0.queue6.rx_copies: 7336569 >>>> dev.ix.0.queue6.lro_queued: 0 >>>> dev.ix.0.queue6.lro_flushed: 0 >>>> dev.ix.0.queue7.interrupt_rate: 5319 >>>> dev.ix.0.queue7.irqs: 7486391989 >>>> dev.ix.0.queue7.txd_head: 1549 >>>> dev.ix.0.queue7.txd_tail: 1549 >>>> dev.ix.0.queue7.tso_tx: 2052 >>>> dev.ix.0.queue7.no_tx_dma_setup: 0 >>>> dev.ix.0.queue7.no_desc_avail: 0 >>>> dev.ix.0.queue7.tx_packets: 9758335509 >>>> dev.ix.0.queue7.rxd_head: 1990 >>>> dev.ix.0.queue7.rxd_tail: 1986 >>>> dev.ix.0.queue7.rx_packets: 13141436559 >>>> dev.ix.0.queue7.rx_bytes: 3877881546 >>>> dev.ix.0.queue7.rx_copies: 8505244 >>>> dev.ix.0.queue7.lro_queued: 0 >>>> dev.ix.0.queue7.lro_flushed: 0 >>>> dev.ix.0.mac_stats.crc_errs: 159 >>>> dev.ix.0.mac_stats.ill_errs: 5 >>>> dev.ix.0.mac_stats.byte_errs: 10 >>>> dev.ix.0.mac_stats.short_discards: 0 >>>> dev.ix.0.mac_stats.local_faults: 213 >>>> dev.ix.0.mac_stats.remote_faults: 37 >>>> dev.ix.0.mac_stats.rec_len_errs: 0 >>>> dev.ix.0.mac_stats.xon_txd: 33623392288 >>>> dev.ix.0.mac_stats.xon_recvd: 0 >>>> dev.ix.0.mac_stats.xoff_txd: 110161 >>>> dev.ix.0.mac_stats.xoff_recvd: 0 >>>> dev.ix.0.mac_stats.total_octets_rcvd: 37967032023930 >>>> dev.ix.0.mac_stats.good_octets_rcvd: 37965112076546 >>>> dev.ix.0.mac_stats.total_pkts_rcvd: 105201314224 >>>> dev.ix.0.mac_stats.good_pkts_rcvd: 18446743122750685300 >>>> dev.ix.0.mac_stats.mcast_pkts_rcvd: 43835 >>>> dev.ix.0.mac_stats.bcast_pkts_rcvd: 960566 >>>> dev.ix.0.mac_stats.rx_frames_64: 454067 >>>> dev.ix.0.mac_stats.rx_frames_65_127: 75350664570 >>>> dev.ix.0.mac_stats.rx_frames_128_255: 4726321485 >>>> dev.ix.0.mac_stats.rx_frames_256_511: 2282382994 >>>> dev.ix.0.mac_stats.rx_frames_512_1023: 3293330145 >>>> dev.ix.0.mac_stats.rx_frames_1024_1522: 19541190234 >>>> dev.ix.0.mac_stats.recv_undersized: 0 >>>> dev.ix.0.mac_stats.recv_fragmented: 0 >>>> dev.ix.0.mac_stats.recv_oversized: 0 >>>> dev.ix.0.mac_stats.recv_jabberd: 4 >>>> dev.ix.0.mac_stats.management_pkts_rcvd: 0 >>>> dev.ix.0.mac_stats.management_pkts_drpd: 0 >>>> dev.ix.0.mac_stats.checksum_errs: 527716650 >>>> dev.ix.0.mac_stats.good_octets_txd: 81355567084088 >>>> dev.ix.0.mac_stats.total_pkts_txd: 81545370171 >>>> dev.ix.0.mac_stats.good_pkts_txd: 47921867720 >>>> dev.ix.0.mac_stats.bcast_pkts_txd: 298646 >>>> dev.ix.0.mac_stats.mcast_pkts_txd: 18446744040086169062 >>>> dev.ix.0.mac_stats.management_pkts_txd: 0 >>>> dev.ix.0.mac_stats.tx_frames_64: 18446744045326220993 >>>> dev.ix.0.mac_stats.tx_frames_65_127: 15629697019 >>>> dev.ix.0.mac_stats.tx_frames_128_255: 4141756940 >>>> dev.ix.0.mac_stats.tx_frames_256_511: 2195917491 >>>> dev.ix.0.mac_stats.tx_frames_512_1023: 2717769904 >>>> dev.ix.0.mac_stats.tx_frames_1024_1522: 51620056990 >>>> >>>> dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version >> - >>>> 2.5.15 >>>> dev.ix.1.%driver: ix >>>> dev.ix.1.%location: slot=0 function=1 handle=\_SB_.PCI1.BR48.S3F1 >>>> dev.ix.1.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086 >>>> subdevice=0x7b11 class=0x020000 >>>> dev.ix.1.%parent: pci131 >>>> dev.ix.1.fc: 3 >>>> dev.ix.1.enable_aim: 1 >>>> dev.ix.1.advertise_speed: 0 >>>> dev.ix.1.dropped: 0 >>>> dev.ix.1.mbuf_defrag_failed: 0 >>>> dev.ix.1.watchdog_events: 0 >>>> dev.ix.1.link_irq: 46692 >>>> dev.ix.1.queue0.interrupt_rate: 15151 >>>> dev.ix.1.queue0.irqs: 2634341620 >>>> dev.ix.1.queue0.txd_head: 1101 >>>> dev.ix.1.queue0.txd_tail: 1101 >>>> dev.ix.1.queue0.tso_tx: 0 >>>> dev.ix.1.queue0.no_tx_dma_setup: 0 >>>> dev.ix.1.queue0.no_desc_avail: 4142 >>>> dev.ix.1.queue0.tx_packets: 4749431326 >>>> dev.ix.1.queue0.rxd_head: 812 >>>> dev.ix.1.queue0.rxd_tail: 811 >>>> dev.ix.1.queue0.rx_packets: 21135516 >>>> dev.ix.1.queue0.rx_bytes: 33229682 >>>> dev.ix.1.queue0.rx_copies: 119434 >>>> dev.ix.1.queue0.lro_queued: 0 >>>> dev.ix.1.queue0.lro_flushed: 0 >>>> dev.ix.1.queue1.interrupt_rate: 15151 >>>> dev.ix.1.queue1.irqs: 2616347302 >>>> dev.ix.1.queue1.txd_head: 797 >>>> dev.ix.1.queue1.txd_tail: 797 >>>> dev.ix.1.queue1.tso_tx: 0 >>>> dev.ix.1.queue1.no_tx_dma_setup: 0 >>>> dev.ix.1.queue1.no_desc_avail: 4129 >>>> dev.ix.1.queue1.tx_packets: 4730356550 >>>> dev.ix.1.queue1.rxd_head: 1345 >>>> dev.ix.1.queue1.rxd_tail: 1344 >>>> dev.ix.1.queue1.rx_packets: 20844457 >>>> dev.ix.1.queue1.rx_bytes: 35014827 >>>> dev.ix.1.queue1.rx_copies: 125167 >>>> dev.ix.1.queue1.lro_queued: 0 >>>> dev.ix.1.queue1.lro_flushed: 0 >>>> dev.ix.1.queue2.interrupt_rate: 5555 >>>> dev.ix.1.queue2.irqs: 2603166062 >>>> dev.ix.1.queue2.txd_head: 1428 >>>> dev.ix.1.queue2.txd_tail: 1432 >>>> dev.ix.1.queue2.tso_tx: 0 >>>> dev.ix.1.queue2.no_tx_dma_setup: 0 >>>> dev.ix.1.queue2.no_desc_avail: 34368 >>>> dev.ix.1.queue2.tx_packets: 4681339909 >>>> dev.ix.1.queue2.rxd_head: 1153 >>>> dev.ix.1.queue2.rxd_tail: 1152 >>>> dev.ix.1.queue2.rx_packets: 21263862 >>>> dev.ix.1.queue2.rx_bytes: 46251419 >>>> dev.ix.1.queue2.rx_copies: 125156 >>>> dev.ix.1.queue2.lro_queued: 0 >>>> dev.ix.1.queue2.lro_flushed: 0 >>>> dev.ix.1.queue3.interrupt_rate: 5319 >>>> dev.ix.1.queue3.irqs: 2658483423 >>>> dev.ix.1.queue3.txd_head: 179 >>>> dev.ix.1.queue3.txd_tail: 179 >>>> dev.ix.1.queue3.tso_tx: 0 >>>> dev.ix.1.queue3.no_tx_dma_setup: 0 >>>> dev.ix.1.queue3.no_desc_avail: 4189 >>>> dev.ix.1.queue3.tx_packets: 4811705668 >>>> dev.ix.1.queue3.rxd_head: 583 >>>> dev.ix.1.queue3.rxd_tail: 582 >>>> dev.ix.1.queue3.rx_packets: 20472051 >>>> dev.ix.1.queue3.rx_bytes: 29823943 >>>> dev.ix.1.queue3.rx_copies: 129450 >>>> dev.ix.1.queue3.lro_queued: 0 >>>> dev.ix.1.queue3.lro_flushed: 0 >>>> dev.ix.1.queue4.interrupt_rate: 5319 >>>> dev.ix.1.queue4.irqs: 2629295642 >>>> dev.ix.1.queue4.txd_head: 592 >>>> dev.ix.1.queue4.txd_tail: 594 >>>> dev.ix.1.queue4.tso_tx: 0 >>>> dev.ix.1.queue4.no_tx_dma_setup: 0 >>>> dev.ix.1.queue4.no_desc_avail: 4106 >>>> dev.ix.1.queue4.tx_packets: 4774679355 >>>> dev.ix.1.queue4.rxd_head: 259 >>>> dev.ix.1.queue4.rxd_tail: 258 >>>> dev.ix.1.queue4.rx_packets: 22561633 >>>> dev.ix.1.queue4.rx_bytes: 40196457 >>>> dev.ix.1.queue4.rx_copies: 135966 >>>> dev.ix.1.queue4.lro_queued: 0 >>>> dev.ix.1.queue4.lro_flushed: 0 >>>> dev.ix.1.queue5.interrupt_rate: 18518 >>>> dev.ix.1.queue5.irqs: 2628129098 >>>> dev.ix.1.queue5.txd_head: 1070 >>>> dev.ix.1.queue5.txd_tail: 1072 >>>> dev.ix.1.queue5.tso_tx: 0 >>>> dev.ix.1.queue5.no_tx_dma_setup: 0 >>>> dev.ix.1.queue5.no_desc_avail: 4127 >>>> dev.ix.1.queue5.tx_packets: 4750613588 >>>> dev.ix.1.queue5.rxd_head: 595 >>>> dev.ix.1.queue5.rxd_tail: 594 >>>> dev.ix.1.queue5.rx_packets: 21309940 >>>> dev.ix.1.queue5.rx_bytes: 29264004 >>>> dev.ix.1.queue5.rx_copies: 134128 >>>> dev.ix.1.queue5.lro_queued: 0 >>>> dev.ix.1.queue5.lro_flushed: 0 >>>> dev.ix.1.queue6.interrupt_rate: 125000 >>>> dev.ix.1.queue6.irqs: 2615444123 >>>> dev.ix.1.queue6.txd_head: 1008 >>>> dev.ix.1.queue6.txd_tail: 1008 >>>> dev.ix.1.queue6.tso_tx: 0 >>>> dev.ix.1.queue6.no_tx_dma_setup: 0 >>>> dev.ix.1.queue6.no_desc_avail: 4106 >>>> dev.ix.1.queue6.tx_packets: 4703519800 >>>> dev.ix.1.queue6.rxd_head: 1209 >>>> dev.ix.1.queue6.rxd_tail: 1208 >>>> dev.ix.1.queue6.rx_packets: 22783319 >>>> dev.ix.1.queue6.rx_bytes: 28091721 >>>> dev.ix.1.queue6.rx_copies: 131653 >>>> dev.ix.1.queue6.lro_queued: 0 >>>> dev.ix.1.queue6.lro_flushed: 0 >>>> dev.ix.1.queue7.interrupt_rate: 6024 >>>> dev.ix.1.queue7.irqs: 2644640254 >>>> dev.ix.1.queue7.txd_head: 1313 >>>> dev.ix.1.queue7.txd_tail: 1313 >>>> dev.ix.1.queue7.tso_tx: 0 >>>> dev.ix.1.queue7.no_tx_dma_setup: 0 >>>> dev.ix.1.queue7.no_desc_avail: 4106 >>>> dev.ix.1.queue7.tx_packets: 4824584039 >>>> dev.ix.1.queue7.rxd_head: 1463 >>>> dev.ix.1.queue7.rxd_tail: 1462 >>>> dev.ix.1.queue7.rx_packets: 21464418 >>>> dev.ix.1.queue7.rx_bytes: 38133966 >>>> dev.ix.1.queue7.rx_copies: 163180 >>>> dev.ix.1.queue7.lro_queued: 0 >>>> dev.ix.1.queue7.lro_flushed: 0 >>>> dev.ix.1.mac_stats.crc_errs: 0 >>>> dev.ix.1.mac_stats.ill_errs: 0 >>>> dev.ix.1.mac_stats.byte_errs: 0 >>>> dev.ix.1.mac_stats.short_discards: 0 >>>> dev.ix.1.mac_stats.local_faults: 0 >>>> dev.ix.1.mac_stats.remote_faults: 2 >>>> dev.ix.1.mac_stats.rec_len_errs: 0 >>>> dev.ix.1.mac_stats.xon_txd: 11207820571 >>>> dev.ix.1.mac_stats.xon_recvd: 0 >>>> dev.ix.1.mac_stats.xoff_txd: 3735958948 >>>> dev.ix.1.mac_stats.xoff_recvd: 0 >>>> dev.ix.1.mac_stats.total_octets_rcvd: 47828274751 >>>> dev.ix.1.mac_stats.good_octets_rcvd: 47225376551 >>>> dev.ix.1.mac_stats.total_pkts_rcvd: 177931572 >>>> dev.ix.1.mac_stats.good_pkts_rcvd: 18446743954331576567 >>>> dev.ix.1.mac_stats.mcast_pkts_rcvd: 7410 >>>> dev.ix.1.mac_stats.bcast_pkts_rcvd: 8907 >>>> dev.ix.1.mac_stats.rx_frames_64: 0 >>>> dev.ix.1.mac_stats.rx_frames_65_127: 134619798 >>>> dev.ix.1.mac_stats.rx_frames_128_255: 7996096 >>>> dev.ix.1.mac_stats.rx_frames_256_511: 3996945 >>>> dev.ix.1.mac_stats.rx_frames_512_1023: 5145938 >>>> dev.ix.1.mac_stats.rx_frames_1024_1522: 21331154 >>>> dev.ix.1.mac_stats.recv_undersized: 0 >>>> dev.ix.1.mac_stats.recv_fragmented: 0 >>>> dev.ix.1.mac_stats.recv_oversized: 0 >>>> dev.ix.1.mac_stats.recv_jabberd: 0 >>>> dev.ix.1.mac_stats.management_pkts_rcvd: 0 >>>> dev.ix.1.mac_stats.management_pkts_drpd: 0 >>>> dev.ix.1.mac_stats.checksum_errs: 219681 >>>> dev.ix.1.mac_stats.good_octets_txd: 38874487735713 >>>> dev.ix.1.mac_stats.total_pkts_txd: 38026147390 >>>> dev.ix.1.mac_stats.good_pkts_txd: 27377335167 >>>> dev.ix.1.mac_stats.bcast_pkts_txd: 67411 >>>> dev.ix.1.mac_stats.mcast_pkts_txd: 18446744063060778699 >>>> dev.ix.1.mac_stats.management_pkts_txd: 0 >>>> dev.ix.1.mac_stats.tx_frames_64: 18446744065500352065 >>>> dev.ix.1.mac_stats.tx_frames_65_127: 6755049420 >>>> dev.ix.1.mac_stats.tx_frames_128_255: 1914523761 >>>> dev.ix.1.mac_stats.tx_frames_256_511: 1038553757 >>>> dev.ix.1.mac_stats.tx_frames_512_1023: 1240890942 >>>> dev.ix.1.mac_stats.tx_frames_1024_1522: 24637516838 >>>> >>>> >>>> Cheers, >>>> >>>> >>>>>> -a >>>>>> >>>>>> >>>>>> On 5 September 2014 07:35, Marcelo Gondim >> wrote: >>>>>>> Hi Adrian, >>>>>>> >>>>>>> I confirmed with the support staff of the room where the server is, >>>>>>> that the >>>>>>> ambient temperature was normal. >>>>>>> >>>>>>> >>>>>>> On 04/09/2014 22:46, Marcelo Gondim wrote: >>>>>>> >>>>>>>> On 04/09/2014 20:48, Adrian Chadd wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> The only time this has happened to me is because the card >> overheated. >>>>>>>>> Can you check that? >>>>>>>>> >>>>>>>> Hi Adrian, >>>>>>>> >>>>>>>> The room where the equipment is located is very cold but I'll check > it >>>>>>>> out. >>>>>>>> Also seen at the time of the problem, a lot of dropped packets. >>>>>>>> >>>>>>>> # netstat -idn >>>>>>>> ... >>>>>>>> ix0 1500 a0:36:9f:2a:6d:ac 18446743423829095869 > 159 >>>>>>>> 750924631703 53285910688 0 0 0 >>>>>>>> ix0 - fe80::a236:9f fe80::a236:9fff:f 0 - - > 2 >>>>>>>> - - - >>>>>>>> ix1 1500 a0:36:9f:2a:6d:ae 18446743954328745465 > 0 >>>>>>>> 119550050209 20178077451 0 0 0 >>>>>>>> ix1 - fe80::a236:9f fe80::a236:9fff:f 0 - - > 1 >>>>>>>> - - - >>>>>>>> ... >>>>>>>> >>>>>>>> 119550050209 droped packets on ix1 and 750924631703 droped on ix0 >>>>>>>> >>>>>>>> Could be interesting I upgrade to10.1-PRERELEASE? >>>>>>>> Could there be a problem with the driver? >>>>>>>> >>>>>>>> Traffic on ix0: 1.4Gbps output / 600Mbps input >>>>>>>> Traffic on ix1: 1.2Gbps output >>>>>>>> >>>>>>>> PPS on ix0: 163Kpps output / 215Kpps input >>>>>>>> PPS on ix1: 131Kpps output >>>>>>>> >>>>>>>> Thanks for your help. >>>>>>>> >>>>>>>>> -a >>>>>>>>> >>>>>>>>> >>>>>>>>> On 4 September 2014 16:14, Marcelo Gondim >> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> I have an Intel X520-SR2and today was working when all traffic >>>>>>>>>> stopped. >>>>>>>>>> I looked in the logs and found this message: >>>>>>>>>> >>>>>>>>>> Sep 4 18:29:53 rt01 kernel: ix1: >>>>>>>>>> Sep 4 18:29:53 rt01 kernel: CRITICAL: ECC ERROR!! Please > Reboot!! >>>>>>>>>> # uname -a >>>>>>>>>> FreeBSD rt01.xxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #10 >>>>>>>>>> r267839: >>>>>>>>>> Thu >>>>>>>>>> Jul 10 15:35:04 BRT 2014 >>>>>>>>>> root@rt01.xxxxx.com.br:/usr/obj/usr/src/sys/GONDIM10 amd64 >>>>>>>>>> >>>>>>>>>> # netstat -m >>>>>>>>>> 98324/53476/151800 mbufs in use (current/cache/total) >>>>>>>>>> 98301/44951/143252/1014370 mbuf clusters in use >>>>>>>>>> (current/cache/total/max) >>>>>>>>>> 98301/44897 mbuf+clusters out of packet secondary zone in use >>>>>>>>>> (current/cache) >>>>>>>>>> 0/421/421/507184 4k (page size) jumbo clusters in use >>>>>>>>>> (current/cache/total/max) >>>>>>>>>> 0/0/0/150276 9k jumbo clusters in use (current/cache/total/max) >>>>>>>>>> 0/0/0/84530 16k jumbo clusters in use (current/cache/total/max) >>>>>>>>>> 221183K/104955K/326138K bytes allocated to network >>>>>>>>>> (current/cache/total) >>>>>>>>>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >>>>>>>>>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >>>>>>>>>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >>>>>>>>>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >>>>>>>>>> 0 requests for sfbufs denied >>>>>>>>>> 0 requests for sfbufs delayed >>>>>>>>>> 0 requests for I/O initiated by sendfile >>>>>>>>>> >>>>>>>>>> Best regards, From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 04:06:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93511B16; Tue, 9 Sep 2014 04:06:33 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DE3490F; Tue, 9 Sep 2014 04:06:33 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id i8so16262348qcq.8 for ; Mon, 08 Sep 2014 21:06:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=JLgRqluX5dUnx588M92L3RghB2jonn8n9xBRhy8Xd6g=; b=xcPpKdX93laQ+QJh/gjHPAfC1KUBHkWH4Nf51ek44wGtwBhYSfVXbzfVLsNYeJFDlW nS+4UrDDRniZiqUUfnn6vi8kWg/nMFsrGFfMsla/fTUrra8PqwtIPeHCPcFejF8QKIcu e3qhENXr4AJ4MGlB1Yqvqc9HxPJK5H1fD9o/3QX3Q/udI/gfpczCY5bcYWLxLI0i4IxP a8Eni/x3+3njpRFcSsz9oil6l19H53y2eqN40XY5nOdOgMPi6R03CVx/1wQ4LU/d7lJf W+sSw4OPImFLwowRZO3YoLgpx6m8cAyF/+pziMxB+I1jYw3Tt7SAfIFxWaYEsXiviS+k ty4Q== MIME-Version: 1.0 X-Received: by 10.229.38.3 with SMTP id z3mr47385582qcd.17.1410235592464; Mon, 08 Sep 2014 21:06:32 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.39.139 with HTTP; Mon, 8 Sep 2014 21:06:32 -0700 (PDT) In-Reply-To: <540E547C.9030703@bsdinfo.com.br> References: <5408F23C.2030309@bsdinfo.com.br> <54091607.9010100@bsdinfo.com.br> <5409CA44.8070203@bsdinfo.com.br> <540A1A3E.2040306@bsdinfo.com.br> <540A8200.5010404@bsdinfo.com.br> <540DFAE3.8070001@bsdinfo.com.br> <002001cfcba0$4812c7f0$d83857d0$@videon-znojmo.cz> <540E547C.9030703@bsdinfo.com.br> Date: Mon, 8 Sep 2014 21:06:32 -0700 X-Google-Sender-Auth: WMylW-SNozcy5nPVSWwCjDhRxdU Message-ID: Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! From: Adrian Chadd To: Marcelo Gondim Content-Type: text/plain; charset=UTF-8 Cc: Jack F Vogel , FreeBSD Net , Eric Joyner , =?UTF-8?B?SW5nLiBBbGXFoSBOZWNodsOhdGFs?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 04:06:33 -0000 Hi, A bunch of us spent a whole bunch of time on the driver before and after 10.0-REL happened to squish a number of ixgbe hanging and out of order bugs. Please update. :) -a From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 05:12:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B716392C; Tue, 9 Sep 2014 05:12:15 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69BBBE09; Tue, 9 Sep 2014 05:12:15 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so15298657qaj.35 for ; Mon, 08 Sep 2014 22:12:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=uFuIApWz9uO/GJmzNmxVWOmRzQG7Z5srD9iTjAY9ka8=; b=svfMywDzIOYQkw/gfyi2W+tMF0mvI+b9BwbMxPH9Gec1YoiC72vBj3kY4bR3jI2HMc 0KK8l5Vl4ZRGxKDqY7H0WmS8BuFw2skB2F4X+AeVwz9nHvsOz6l6sYMza4FizAvUKsaT wPdUF3jeNSI2s7R3758qWlKTvan+Tifit6xgTb2Q7kZel5ahp/xll1HN6PrBE6Lc5QKo zGnkbJTLj6b31q8suEIIt5Fcf9mj5blxLNdN4fKTfO3M+BjXcoLRKpyWQiEQWdG+JING R0SVkV6fKdcd91lAfiRAy7JVHH1U83+gxXVaQldgN6dcR54ytJZeyujnQ2mBi0M76RY7 f5GQ== MIME-Version: 1.0 X-Received: by 10.140.31.75 with SMTP id e69mr46157243qge.2.1410239534540; Mon, 08 Sep 2014 22:12:14 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.39.139 with HTTP; Mon, 8 Sep 2014 22:12:14 -0700 (PDT) Date: Mon, 8 Sep 2014 22:12:14 -0700 X-Google-Sender-Auth: xJqxmak3LWd_lbucGZMG20mnqi0 Message-ID: Subject: [igb] add DROP_EN to each RX queue config if TX flow control is disabled From: Adrian Chadd To: FreeBSD Net , Jack F Vogel , "Joyner, Eric" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 05:12:15 -0000 Hi, This patch enables the DROP_EN flag to each RX queue if TX flow control is disabled. It (mostly) mirrors what the ixgbe driver does. This prevents a single full RX ring from stalling the rest of the RX rings. How's it look? (indenting and such aside, thanks google.) -a Index: sys/dev/e1000/if_igb.c =================================================================== --- sys/dev/e1000/if_igb.c (revision 271290) +++ sys/dev/e1000/if_igb.c (working copy) @@ -4712,6 +4712,18 @@ rctl |= E1000_RCTL_SZ_2048; } + /* + * If TX flow control is disabled and there's >1 queue defined, + * enable DROP. + * + * This drops frames rather than hanging the RX MAC for all queues. + */ + if ((adapter->num_queues > 1) && + (adapter->fc == e1000_fc_none || + adapter->fc == e1000_fc_rx_pause)) { + srrctl |= E1000_SRRCTL_DROP_EN; + } + /* Setup the Base and Length of the Rx Descriptor Rings */ for (int i = 0; i < adapter->num_queues; i++, rxr++) { u64 bus_addr = rxr->rxdma.dma_paddr; @@ -6255,6 +6267,7 @@ adapter->hw.fc.current_mode = adapter->hw.fc.requested_mode; e1000_force_mac_fc(&adapter->hw); + /* XXX TODO: update DROP_EN on each RX queue if appropriate */ return (error); } From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 09:33:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2AD4D446 for ; Tue, 9 Sep 2014 09:33:12 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F7D5B06 for ; Tue, 9 Sep 2014 09:33:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s899X56X014701; Tue, 9 Sep 2014 19:33:06 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Tue, 9 Sep 2014 19:33:05 +1000 (EST) From: Ian Smith To: John Case Subject: Re: How can sshuttle be used properly with FreeBSD (and with DNS) ? In-Reply-To: Message-ID: <20140909183722.R58647@sola.nimnet.asn.au> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-net@freebsd.org, John Nielsen X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 09:33:12 -0000 On Sat, 6 Sep 2014 02:52:22 +0000, John Case wrote: > I would like to use sshuttle (http://github.com/apenwarr/sshuttle) on > FreeBSD. > > I have it working for TCP connections, but it does not properly tunnel DNS > requests. The documentation for sshuttle says that ipfw forward rules will > not properly forward UDP packets, and so when it runs on FreeBSD, sshuttle > inserts divert rules instead. The project author believes that this will > work properly (inserting divert rules to tunnel UDP) but I am not having any > success. Forward doesn't update headers, likely why it's of no use to sshuttle. > BUT, I already have a divert rule (and natd running) on this system even > before I run sshuttle at all - because the system won't function as a normal > gateway unless I use divert/natd. I prefer to run a gateway without > divert/natd, but since both sides of this gateway are non-routable IPs, I > cannot do that - in order to function as a gateway with 10.x.x.x networks on > both sides, I need to run natd/divert. Or as John pointed out in your other thread, you may find it easier to use ipfw nat (in-kernel nat) instead, though sshuttle may be hard-wired to use natd. Either way, you need separate NAT instances for each task. As sshuttle needs to use divert sockets in particular, is the divert port used able to be specified? Or does it insist on using the default port natd (8668/divert)? If so, you need to use another port for your regular NAT and 8668 for sshuttle; anyway, two different ports, or you could even use natd for sshuttle and ipfw nat for regular translation. > So that means that when sshuttle inserts its own divert rules, they conflict > with the existing ones, and I am not running a second natd daemon, so I think > it all just falls apart. See natd(8) section 'MULTIPLE INSTANCES' for a well-doicumented example. And/or check ipfw(8) section 'NETWORK ADDRESS TRANSLATION (NAT)' and in the examples, 'NAT, REDIRECT AND LSNAT'. > How can this be fixed ? > > Is anyone out there using sshuttle on FreeBSD with the --dns switch ? Nope. > Here is what my ipfw.conf looks like BEFORE I run sshuttle: > > > add 1000 divert natd ip from any to any in via xl0 > add 2000 divert natd ip from any to any out via xl0 Another potential issue here; 'out via xl0' may be ambiguous, in that traffic going 'out' to anywhere, on any interface, that originally came 'in' on xl0 also matches 'out via xl0'. Unambiguous would be: add 1000 divert natd ip from any to any in recv xl0 add 2000 divert natd ip from any to any out xmit xl0 though you can save natd (or ipfw nat) some work/time if you restrict outbound nat to packets that may qualify, perhaps 'from any to any out xmit xl0 recv xl1' in the case of your regular NATting of inside net traffic, assuming xl1 to be your inside interface. 'natd' in the rules above is just the port name from /etc/services; you can use any unused port number for each (or another) instance, as per natd(8)'s multiple instances example. > and in rc.conf: > > > gateway_enable="yes" > natd_enable="yes" > natd_interface="xl0" > > > Again, this works fine - I have a functioning internet gateway and both of > the interfaces on it have non-routable IP address. > > Then I run sshuttle and it *also* works fine - but only for TCP. It does not > tunnel UDP (dns) properly like it is supposed to, and I think the reason is > that I already have diverting/natd going on and then I run sshuttle and it > inserts another two divert rules into ipfw. > > But I am not sure wha the fix would be ... Just make sure that both NAT instances use separate ports and rules. To see it in action, to confirm both regular NAT and the sshuttle UDP diversion are working correctly, run tcpdump in a couple of consoles. cheers, Ian From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 10:29:30 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A4B27349 for ; Tue, 9 Sep 2014 10:29:30 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0C904FEC for ; Tue, 9 Sep 2014 10:29:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s89ATQ7p016633; Tue, 9 Sep 2014 20:29:26 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Tue, 9 Sep 2014 20:29:26 +1000 (EST) From: Ian Smith To: John Case Subject: Re: How can sshuttle be used properly with FreeBSD (and with DNS) ? In-Reply-To: <20140909183722.R58647@sola.nimnet.asn.au> Message-ID: <20140909195046.L58647@sola.nimnet.asn.au> References: <20140909183722.R58647@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-net@freebsd.org, John Nielsen X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 10:29:30 -0000 On Tue, 9 Sep 2014 19:33:05, Ian Smith wrote: > add 1000 divert natd ip from any to any in recv xl0 > add 2000 divert natd ip from any to any out xmit xl0 Oops, 'ip' should nowadays be 'ip4|ipv4' for divert rules, if ip6 is configured on that interface. Last I heard, ip6 packets break divert. cheers, Ian From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 20:41:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8F86C3E for ; Tue, 9 Sep 2014 20:41:26 +0000 (UTC) Received: from mail-qc0-x235.google.com (mail-qc0-x235.google.com [IPv6:2607:f8b0:400d:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A17A636 for ; Tue, 9 Sep 2014 20:41:26 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id i17so17896751qcy.40 for ; Tue, 09 Sep 2014 13:41:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=BLsiPm47MK6joEJAYWS0jGurGoBKIw+YygchNxRQCcc=; b=kKhYqN0SVvauJ2P2apqvcy88RLHPS8V1ZGny7c2atnm4grQwPBYGj55Oik+HkI+EJR btEk2wx65NB/WhbunFyNbWeFV5Z1iUqvFBbHjRsFHqSjDMJSTXKDoNWj4lh5QUh5Egbu RnNCgyCdRR1xftckL7md/pVTJfxTVnoScQelVnq4Py1S2V0rTSKA6+l6e8QENs8xce6T ca/DUJ6dn/FpjYGxkliNHtf7jdRr68X0BhGMn7uqJ5AXXYesX9EzN2TC0tQL7Ix2F+7D 0OrFNtNCFXCGgJA9Ou27zbmyLdceq8v5GmTR1sIWvvxkhrAZAHo94tTUwQvK9JKiKTW7 GEzw== MIME-Version: 1.0 X-Received: by 10.224.151.69 with SMTP id b5mr55316755qaw.37.1410295285081; Tue, 09 Sep 2014 13:41:25 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.39.139 with HTTP; Tue, 9 Sep 2014 13:41:25 -0700 (PDT) Date: Tue, 9 Sep 2014 13:41:25 -0700 X-Google-Sender-Auth: hu2P0zrKUarud81T5hQYE19l-oY Message-ID: Subject: inpcb lookups in udp_send -> inpcb laddr and flowtable From: Adrian Chadd To: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 20:41:26 -0000 Hi! I found this whilst doing UDP RSS testing. root@adrian-hackbox:/home/adrian/work/freebsd/head/src/sys/net # sysctl debug.lock.prof.stats | head -2 ; sysctl debug.lock.prof.stats | sort -nk4 | tail -5 debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 40 11 114900 55276 359106 0 0 0 20962 /usr/home/adrian/work/freebsd/head/src/sys/dev/ixgbe/ixgbe.c:1550 (sleep mutex:ix0:tx(6)) 21 11 144253 71959 454169 0 0 0 27506 /usr/home/adrian/work/freebsd/head/src/sys/dev/ixgbe/ixgbe.c:1550 (sleep mutex:ix0:tx(1)) 9 10 3132066 85999 42942422 0 0 0 35943 /usr/home/adrian/work/freebsd/head/src/sys/netinet/udp_usrreq.c:369 (sleep mutex:so_rcv) 676 1713 805307 169654 7076283 0 0 0 69741 /usr/home/adrian/work/freebsd/head/src/sys/netinet/in_pcb.c:963 (sleep mutex:rtentry) 674 4426 2111324 6415012 7076284 0 0 0 489745 /usr/home/adrian/work/freebsd/head/src/sys/net/route.c:420 (sleep mutex:rtentry) then... root@adrian-hackbox:/home/adrian/git/github/erikarn/freebsd-rss/rss-udp-srv # dtrace -n 'fbt::rtalloc1_fib:entry { @[stack()] = count(); }' dtrace: description 'fbt::rtalloc1_fib:entry ' matched 1 probe ^[[A^C kernel`rtalloc_ign_fib+0x98 kernel`in_pcbladdr+0x7d kernel`in_pcbconnect_setup+0x1af kernel`udp_send+0x4af kernel`sosend_dgram+0x30b kernel`kern_sendit+0x20b kernel`sendit+0x129 kernel`sys_sendto+0x4d kernel`amd64_syscall+0x25a kernel`0xffffffff80cd178b 1848628 It turns out that if a local address isn't bound (ie, it's INADDR_ANY) then in_pcbladdr() will do a route table lookup to find the interface to transmit from. The v4/v6 laddr paths don't know about flowtable; they just do a routing table lookup. I'm currently experimenting with a kernel fix to use flowtable here for IPv4 UDP sockets but I'm not sure about the IPv6 path (the source interface/address selection is much crazier with all the various scope bits going on.) I'd like some help with the IPv6 source selection please! Thanks! -a From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 21:40:06 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4DAC98DA for ; Tue, 9 Sep 2014 21:40:06 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 347DAC45 for ; Tue, 9 Sep 2014 21:40:06 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s89Le6jm091328 for ; Tue, 9 Sep 2014 21:40:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 157287] [re] [panic] INVARIANTS panic (Memory modified after free) Date: Tue, 09 Sep 2014 21:40:06 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: joerg@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 21:40:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=157287 Joerg Wunsch changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Issue Resolved Resolution|--- |Overcome By Events --- Comment #7 from Joerg Wunsch --- The respective system has been decommisioned recently. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 21:43:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CAF14987 for ; Tue, 9 Sep 2014 21:43:01 +0000 (UTC) Received: from smtpout100.ehv.onlinespamfilter.nl (smtpout100.ehv.onlinespamfilter.nl [IPv6:2001:4cb8:1:1620:217:21:240:168]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 764C2CE0 for ; Tue, 9 Sep 2014 21:43:01 +0000 (UTC) Received: from smtp.onlinespamfilter.nl (localhost [127.0.0.1]) by smtp.onlinespamfilter.nl (Postfix) with ESMTP id 3ht0Fz49hLzQx for ; Tue, 9 Sep 2014 23:42:47 +0200 (CEST) Received: from smtp.debank.tv (145-158-ftth.on.nl [88.159.158.145]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.onlinespamfilter.nl (Postfix) with ESMTPS for ; Tue, 9 Sep 2014 23:42:47 +0200 (CEST) Received: from smtp.debank.tv (smtp.debank.tv [172.16.143.25]) by smtp.debank.tv (Postfix) with ESMTP id 255242A2E16 for ; Tue, 9 Sep 2014 23:42:47 +0200 (CEST) Received: from roundcube.debank.tv (roundcube.debank.tv [IPv6:2001:1af8:fe30::41]) by smtp.debank.tv (Postfix) with ESMTP id C45922A2E13 for ; Tue, 9 Sep 2014 23:42:46 +0200 (CEST) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 10 Sep 2014 09:42:46 +1200 From: mailinglists@debank.tv To: freebsd-net@freebsd.org Subject: Performance problem with slow link behind fast gateway Message-ID: X-Sender: mailinglists@debank.tv User-Agent: Roundcube Webmail/1.0.2 X-Virus-Scanned: ClamAV using ClamSMTP @ debank.tv X-OSF-Virus: CLEAN X-OSF-Outgoing: Innocent X-OSF-Account: 1327 X-OSF-SUM: 63c0bbcd45dcebce75c580685ecd5b7b X-OSF-Info: Checked for spam and viruses X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 21:43:01 -0000 All, I'm seeing some performance problems with a slowish VPN connection behind a fast gateway, the setup looks like this: |----------------------------------| |-----------------------------| |client (zandbak) (DSL connection)| ---- 'VPN tunnel' ----- |Gateway (vps) using NAT on 1G|------ 'Internet' |----------------------------------| |-----------------------------| Transfers from the gateway to the client are reasonably fast (easily within usable range for me): root@zandbak:/usr/home/rob # scp rob@gateway:test_file ./ test_file 100% 10MB 445.2KB/s 00:23 Transfers from the internet to the gateway are fast: root@vps:/usr/home/rob # fetch -4 "http://149.20.53.23/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/10.0/FreeBSD-10.0-RELEASE-amd64-bootonly.iso" FreeBSD-10.0-RELEASE-amd64-bootonly.iso 100% of 209 MB 10 MBps 00m20s But transfers from the client to the internet through the tunnel are showing a very degraded connection speed, the speed jumps up and down but averages at around 20kBps: root@zandbak:/usr/home/rob # fetch "http://149.20.53.23/pub/FreeBSD/ISO-IMAGES-amd64/10.0/FreeBSD-10.0-RELEASE-amd64-bootonly.iso" FreeBSD-10.0-RELEASE-amd64-bootonly.iso 0% of 209 MB 8275 Bps 07h27m I've tried to eliminate some variables: -VPN: tinc as a L2 VPN and openVPN as a L3 VPN, results are the same -NAT: pf and ipfw, results are the same I suspect that there's a problem with the fast link receiving too much data and once the buffers are full dropping packets although I'm not sure if this is actually the problem. My question is: how can I debug this issue? Below some system information, I can supply more info if needed Thanks! Rob Evers System info: Gateway: This is a VPS on KVM root@vps:/usr/home/rob # uname -a FreeBSD vps.debank.tv 10.0-STABLE FreeBSD 10.0-STABLE #5 r268727M: Wed Jul 16 13:17:24 NZST 2014 root@vps.debank.tv:/usr/obj/usr/src/sys/GENERIC amd64 root@vps:/usr/home/rob # ifconfig vtnet0 vtnet0: flags=8843 metric 0 mtu 1500 options=6c00ab ether 00:16:3c:55:17:b9 inet 192.227.xxx.xxx netmask 0xffffff00 broadcast 192.227.xxx.xxx inet6 fe80::216:3cff:fe55:17b9%vtnet0 prefixlen 64 scopeid 0x1 nd6 options=21 media: Ethernet 10Gbase-T status: active root@vps:/usr/home/rob # ifconfig tap0 tap0: flags=8843 metric 0 mtu 1500 options=80000 ether 00:bd:61:01:00:00 inet6 fd7c:3e16:580b:4ccf::50 prefixlen 64 inet6 fe80::2bd:61ff:fe01:0%tap0 prefixlen 64 scopeid 0x4 inet 172.16.143.50 netmask 0xffffff00 broadcast 172.16.143.255 nd6 options=61 media: Ethernet autoselect status: active Opened by PID 61485 Client: This is a VM on bhyve root@zandbak:/usr/home/rob # uname -a FreeBSD zandbak 10.0-RELEASE-p7 FreeBSD 10.0-RELEASE-p7 #0: Tue Jul 8 06:37:44 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 root@zandbak:/usr/home/rob # ifconfig vtnet0 vtnet0: flags=8943 metric 0 mtu 1500 options=80028 ether 52:54:00:13:fd:78 inet 192.168.1.129 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::5054:ff:fe13:fd78%vtnet0 prefixlen 64 scopeid 0x1 nd6 options=29 media: Ethernet 10Gbase-T status: active root@zandbak:/usr/home/rob # ifconfig tap0 tap0: flags=8843 metric 0 mtu 1500 options=80000 ether 00:bd:3d:94:05:00 inet 172.16.143.55 netmask 0xffffff00 broadcast 172.16.143.255 nd6 options=29 media: Ethernet autoselect status: active Opened by PID 1411 From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 23:32:15 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7BB736CD for ; Tue, 9 Sep 2014 23:32:15 +0000 (UTC) Received: from sdf.lonestar.org (mx.sdf.org [192.94.73.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx.sdf.org", Issuer "SDF.ORG" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 40A0D9DB for ; Tue, 9 Sep 2014 23:32:14 +0000 (UTC) Received: from otaku.freeshell.org (IDENT:case@otaku.freeshell.org [192.94.73.9]) by sdf.lonestar.org (8.14.8/8.14.5) with ESMTP id s89NVwF7010757 (using TLSv1/SSLv3 with cipher DHE-RSA-AES256-SHA (256 bits) verified NO); Tue, 9 Sep 2014 23:32:01 GMT Date: Tue, 9 Sep 2014 23:31:58 +0000 (UTC) From: John Case X-X-Sender: case@faeroes.freeshell.org To: Ian Smith Subject: Re: How can sshuttle be used properly with FreeBSD (and with DNS) ? In-Reply-To: <20140909195046.L58647@sola.nimnet.asn.au> Message-ID: References: <20140909183722.R58647@sola.nimnet.asn.au> <20140909195046.L58647@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org, John Nielsen X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 23:32:15 -0000 On Tue, 9 Sep 2014, Ian Smith wrote: > On Tue, 9 Sep 2014 19:33:05, Ian Smith wrote: > > > add 1000 divert natd ip from any to any in recv xl0 > > add 2000 divert natd ip from any to any out xmit xl0 > > Oops, 'ip' should nowadays be 'ip4|ipv4' for divert rules, if ip6 is > configured on that interface. Last I heard, ip6 packets break divert. Thanks for your help. I will try it with two sets of nat and see how it goes... From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 23:37:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 378787DB for ; Tue, 9 Sep 2014 23:37:28 +0000 (UTC) Received: from sdf.lonestar.org (mx.sdf.org [192.94.73.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx.sdf.org", Issuer "SDF.ORG" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 19A64A15 for ; Tue, 9 Sep 2014 23:37:27 +0000 (UTC) Received: from otaku.freeshell.org (IDENT:case@otaku.freeshell.org [192.94.73.9]) by sdf.lonestar.org (8.14.8/8.14.5) with ESMTP id s89NbJ36028629 (using TLSv1/SSLv3 with cipher DHE-RSA-AES256-SHA (256 bits) verified NO) for ; Tue, 9 Sep 2014 23:37:19 GMT Date: Tue, 9 Sep 2014 23:37:19 +0000 (UTC) From: John Case X-X-Sender: case@faeroes.freeshell.org To: freebsd-net@freebsd.org Subject: Can I make this simple ipfw ruleset any more restrictive ? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 23:37:28 -0000 I have a very simple firewall - it*blocks everything*, and the only traffic that is allowed is for internal clients to make outbound connections to tcp port 40. Also, internal clients can ping/traceroute. But that's it - no other connections in or out are allowed. I have this ruleset and it is working perfectly: ipfw add 10 allow tcp from any to any established ipfw add 20 allow icmp from any to any icmptypes 0,3,8,11 ipfw add 30 allow udp from any to any 33433-33499 in via fxp1 ipfw add 40 allow tcp from any to any 40 in via fxp1 (fxp1 is the internal interface, and so I allow the port 40 connections and the udp for traceroute only for requests that come in from the internal network) Is there anything I have screwed up here ? Any unintentional traffic that I am letting through ? Is there any way to lock this down further, and make it even more strict ? Thank yo. From owner-freebsd-net@FreeBSD.ORG Tue Sep 9 23:50:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A451EA69 for ; Tue, 9 Sep 2014 23:50:44 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 65A3CB1C for ; Tue, 9 Sep 2014 23:50:44 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s89NogqP024537 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 9 Sep 2014 16:50:42 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s89Nog17024536; Tue, 9 Sep 2014 16:50:42 -0700 (PDT) (envelope-from jmg) Date: Tue, 9 Sep 2014 16:50:42 -0700 From: John-Mark Gurney To: John Case Subject: Re: Can I make this simple ipfw ruleset any more restrictive ? Message-ID: <20140909235042.GP82175@funkthat.com> Mail-Followup-To: John Case , freebsd-net@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 09 Sep 2014 16:50:42 -0700 (PDT) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 23:50:44 -0000 John Case wrote this message on Tue, Sep 09, 2014 at 23:37 +0000: > > I have a very simple firewall - it*blocks everything*, and the only > traffic that is allowed is for internal clients to make outbound > connections to tcp port 40. > > Also, internal clients can ping/traceroute. > > But that's it - no other connections in or out are allowed. I have this > ruleset and it is working perfectly: > > ipfw add 10 allow tcp from any to any established > ipfw add 20 allow icmp from any to any icmptypes 0,3,8,11 > ipfw add 30 allow udp from any to any 33433-33499 in via fxp1 > ipfw add 40 allow tcp from any to any 40 in via fxp1 > > (fxp1 is the internal interface, and so I allow the port 40 connections > and the udp for traceroute only for requests that come in from the > internal network) > > Is there anything I have screwed up here ? Any unintentional traffic that > I am letting through ? > > Is there any way to lock this down further, and make it even more strict ? You could lock down the UDP ports to a single one and remember to use -e with traceroute: -e Firewall evasion mode. Use fixed destination ports for UDP and TCP probes. The destination port does NOT increment with each packet sent. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Sep 10 00:48:30 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 061A8D16 for ; Wed, 10 Sep 2014 00:48:30 +0000 (UTC) Received: from gw.catspoiler.org (cl-1657.chi-02.us.sixxs.net [IPv6:2001:4978:f:678::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 89063149 for ; Wed, 10 Sep 2014 00:48:29 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id s8A0mJL9011252; Tue, 9 Sep 2014 17:48:23 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201409100048.s8A0mJL9011252@gw.catspoiler.org> Date: Tue, 9 Sep 2014 17:48:19 -0700 (PDT) From: Don Lewis Subject: Re: Can I make this simple ipfw ruleset any more restrictive ? To: case@SDF.ORG In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2014 00:48:30 -0000 On 9 Sep, John Case wrote: > > I have a very simple firewall - it*blocks everything*, and the only > traffic that is allowed is for internal clients to make outbound > connections to tcp port 40. > > Also, internal clients can ping/traceroute. > > But that's it - no other connections in or out are allowed. I have this > ruleset and it is working perfectly: > > ipfw add 10 allow tcp from any to any established > ipfw add 20 allow icmp from any to any icmptypes 0,3,8,11 > ipfw add 30 allow udp from any to any 33433-33499 in via fxp1 > ipfw add 40 allow tcp from any to any 40 in via fxp1 > > (fxp1 is the internal interface, and so I allow the port 40 connections > and the udp for traceroute only for requests that come in from the > internal network) > > Is there anything I have screwed up here ? Any unintentional traffic that > I am letting through ? > > Is there any way to lock this down further, and make it even more strict ? I generally do something like that, but I also add anti-spoofing rules as well. Basically block any packets that have an inside source IP address that are received via the outside interface, and block any packets that don't have an inside IP source address that are received via the inside interface. The established keyword will let any TCP packets through that have either the ACK or RST flags set. That only blocks incoming connection attempts which have SYN without ACK. It would allow someone to scan your internal network by sending packets with ACK set and SYN not set and watching for RST packets being returned. I think you could avoid this with these two rules: ipfw add 10 check-state [...] ipfw add 40 allow tcp from any to any 40 in via fxp1 keep-state but I haven't actually done this. From owner-freebsd-net@FreeBSD.ORG Wed Sep 10 14:22:58 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC168D81 for ; Wed, 10 Sep 2014 14:22:58 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [67.212.89.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B19B71A2C for ; Wed, 10 Sep 2014 14:22:58 +0000 (UTC) Received: from mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) by mail.bsdinfo.com.br (Postfix) with ESMTP id A9523139CE for ; Wed, 10 Sep 2014 11:22:52 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bsdinfo.com.br; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:to:mime-version :user-agent:from:from:date:date:message-id; s=dkim; t= 1410358968; x=1411222969; bh=EjFw5N632npNg6QdQdJ7KhzeWGMNHvU4xSw mlhRcxP0=; b=dlrKRLx/o2S2ougf9RG94uqRnvey/38i90IiArKD2r3PJ+XNQUM Zg+72zmcuR4VF/G9uTX/SX1JPB+cdFeH/pLzV7qm3IQVoq+tyCLXmKD9mBfFfCxX bK54ud6uvdZpGKun0KzKYZLEOZZlvmPj/Xs2vFKLgwZZwdvXu+heQ32w= X-Virus-Scanned: amavisd-new at mail.bsdinfo.com.br Received: from mail.bsdinfo.com.br ([127.0.0.1]) by mail.bsdinfo.com.br (mail.bsdinfo.com.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VwnvY3Nb2V2p for ; Wed, 10 Sep 2014 11:22:48 -0300 (BRT) Received: from [192.168.10.208] (unknown [186.193.54.69]) by mail.bsdinfo.com.br (Postfix) with ESMTPSA id 16C96139C4; Wed, 10 Sep 2014 11:22:45 -0300 (BRT) Message-ID: <54105EB1.4050607@bsdinfo.com.br> Date: Wed, 10 Sep 2014 11:22:41 -0300 From: Marcelo Gondim User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!! References: <5408F23C.2030309@bsdinfo.com.br> <54091607.9010100@bsdinfo.com.br> <5409CA44.8070203@bsdinfo.com.br> <540A1A3E.2040306@bsdinfo.com.br> <540A8200.5010404@bsdinfo.com.br> <540DFAE3.8070001@bsdinfo.com.br> <002001cfcba0$4812c7f0$d83857d0$@videon-znojmo.cz> <540E547C.9030703@bsdinfo.com.br> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Jack F Vogel , FreeBSD Net , Eric Joyner , =?UTF-8?B?IkluZy4gQWxlxaEgTmVjaHbDoXRhbCI=?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2014 14:22:59 -0000 On 09/09/2014 01:06, Adrian Chadd wrote: > Hi, > > A bunch of us spent a whole bunch of time on the driver before and > after 10.0-REL happened to squish a number of ixgbe hanging and out of > order bugs. > > Please update. :) > > > -a > Dear Yesterday performed the following tasks: - I removed the front door of the rack in the datacenter. - I increased by 100% the speed of the fans. - I removed the dust filter that was in front of the server, letting air circulate more easily. I'll wait a few days to see if the crashes will stop. Thanks to everyone and if the problem happens again, I'll let you know. :) Cheers, From owner-freebsd-net@FreeBSD.ORG Wed Sep 10 15:59:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B6F9898; Wed, 10 Sep 2014 15:59:01 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F8BA170B; Wed, 10 Sep 2014 15:59:01 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7B6E7B987; Wed, 10 Sep 2014 11:59:00 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: ixgbe(4) spin lock held too long Date: Wed, 10 Sep 2014 11:24:41 -0400 Message-ID: <1577813.IPE4JfnhZd@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <540E04AA.80806@vangyzen.net> References: <1410203348.1343.1.camel@bruno> <1410203965.1343.3.camel@bruno> <540E04AA.80806@vangyzen.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 10 Sep 2014 11:59:00 -0400 (EDT) Cc: Eric van Gyzen X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2014 15:59:01 -0000 On Monday, September 08, 2014 03:34:02 PM Eric van Gyzen wrote: > On 09/08/2014 15:19, Sean Bruno wrote: > > On Mon, 2014-09-08 at 12:09 -0700, Sean Bruno wrote: > >> This sort of looks like the hardware failed to respond to us in time? > >> Too busy? > >> > >> sean > > > > This seems to be affecting my 10/stable machines from 15Aug2014. > > > > Not a lot of churn in the code so I don't think this is new. The > > afflicted machines, quite a few by my count, appear to have not been > > super busy (pushing about 200 Mb/s). > > > > sean > > > >> panic: spin lock held too long > >> > >> GNU gdb 6.1.1 [FreeBSD] > >> Copyright 2004 Free Software Foundation, Inc. > >> GDB is free software, covered by the GNU General Public License, and you > >> are > >> welcome to change it and/or distribute copies of it under certain > >> conditions. > >> Type "show copying" to see the conditions. > >> There is absolutely no warranty for GDB. Type "show warranty" for > >> details. > >> This GDB was configured as "amd64-marcel-freebsd"... > >> > >> Unread portion of the kernel message buffer: > >> spin lock 0xffffffff812a0400 (callout) held by 0xfffff800151fe000 (tid > >> 100003) too long > > TID 100003 is usually a kernel idle thread, which would seem to indicate > a dangling lock. Can you enable WITNESS (without WITNESS_SKIPSPIN) on > this box? Also, do 'tid 100003' and 'bt' in kgdb to see what the thread holding the lock was doing. -- John Baldwin From owner-freebsd-net@FreeBSD.ORG Wed Sep 10 17:27:00 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF04514C for ; Wed, 10 Sep 2014 17:27:00 +0000 (UTC) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "nyi.unixathome.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AECA01FC for ; Wed, 10 Sep 2014 17:27:00 +0000 (UTC) Received: from nyi.unixathome.org (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id 019445082E for ; Wed, 10 Sep 2014 17:26:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by nyi.unixathome.org (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2fh-uTV6GA_C for ; Wed, 10 Sep 2014 17:26:44 +0000 (UTC) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id 6201950823 for ; Wed, 10 Sep 2014 17:26:44 +0000 (UTC) From: Dan Langille Content-Type: multipart/signed; boundary="Apple-Mail=_050A1378-1121-459A-9A34-0E76F2C1640F"; protocol="application/pgp-signature"; micalg=pgp-sha1 Subject: Configuration for IPv6 over tunnel Message-Id: <14E3A97C-4FCB-4A2C-B22F-3D0849CECA2D@langille.org> Date: Wed, 10 Sep 2014 13:26:42 -0400 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2014 17:27:01 -0000 --Apple-Mail=_050A1378-1121-459A-9A34-0E76F2C1640F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 The existing documentation for setting up IPv6 over a tunnel is = incorrect. Specifically: gifconfig is deprecated. Use cloned_interfaces=3D"gif0" = instead. https://www.freebsd.org/doc/handbook/network-ipv6.html I have filed a PR: = https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D193523 I have the following configuration for my IPv6 tunnel on FreeBSD 9.3 = from Hurricane Electric. Given: IPv6 Tunnel Endpoints Server IPv4 Address: 209.51.x.y Server IPv6 Address: 2001:470:xx06:9ea::1/64 Client IPv4 Address: 96.245.100.201 Client IPv6 Address: 2001:470:xx06:9ea::2/64 Routed /64: 2001:470:xx07:9ea::/64 My /etc/rc.conf includes cloned_interfaces=3D"gif0=94 ifconfig_gif0=3D"tunnel 96.245.100.201 209.51.x.y mtu 1480=94 ifconfig_gif0_ipv6=3D"inet6 2001:470:xx06:9ea::2 2001:470:xx06:9ea::1 = prefixlen 128" ifconfig_em0_ipv6=3D"inet6 2001:470:xx07:9ea:1::1=94 ipv6_defaultrouter=3D"2001:470:xx06:9ea::1" ipv6_gateway_enable=3D=93YES" rtadvd_enable=3D=93YES=94 Comments? Suggestions? --Apple-Mail=_050A1378-1121-459A-9A34-0E76F2C1640F Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iKYEARECAGYFAlQQidJfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDA3REZBQjJGRUQ3NEE5QkE0NTNGOUJCNzBB MEIxNzE0Q0ZGQjlEM0MACgkQCgsXFM/7nTzTUACeITynB0RWzR2+GBl1IkH+JbUE xjwAn0L5j1btaGV7hrK7tVTx6xPAIZUn =Yin9 -----END PGP SIGNATURE----- --Apple-Mail=_050A1378-1121-459A-9A34-0E76F2C1640F-- From owner-freebsd-net@FreeBSD.ORG Wed Sep 10 20:52:27 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 416B6D79 for ; Wed, 10 Sep 2014 20:52:27 +0000 (UTC) Received: from mail.as41113.net (mail.as41113.net [91.208.177.22]) by mx1.freebsd.org (Postfix) with ESMTP id 05D3DB68 for ; Wed, 10 Sep 2014 20:52:25 +0000 (UTC) Received: from [172.21.87.41] (193.98.9.212.in-addr.arpa [212.9.98.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: lists@rewt.org.uk) by mail.as41113.net (Postfix) with ESMTPSA id 3htZtJ5DCHz1N1yG for ; Wed, 10 Sep 2014 21:42:48 +0100 (BST) Message-ID: <5410B7BE.9060005@rewt.org.uk> Date: Wed, 10 Sep 2014 21:42:38 +0100 From: Joe Holden User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Configuration for IPv6 over tunnel References: <14E3A97C-4FCB-4A2C-B22F-3D0849CECA2D@langille.org> In-Reply-To: <14E3A97C-4FCB-4A2C-B22F-3D0849CECA2D@langille.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2014 20:52:27 -0000 On 10/09/2014 18:26, Dan Langille wrote: > The existing documentation for setting up IPv6 over a tunnel is incorrect. > > Specifically: gifconfig is deprecated. Use cloned_interfaces="gif0" instead. > > https://www.freebsd.org/doc/handbook/network-ipv6.html > > I have filed a PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193523 > > I have the following configuration for my IPv6 tunnel on FreeBSD 9.3 from Hurricane Electric. > > Given: > > IPv6 Tunnel Endpoints > Server IPv4 Address: 209.51.x.y > Server IPv6 Address: 2001:470:xx06:9ea::1/64 > Client IPv4 Address: 96.245.100.201 > Client IPv6 Address: 2001:470:xx06:9ea::2/64 > > Routed /64: 2001:470:xx07:9ea::/64 > > > My /etc/rc.conf includes > > cloned_interfaces="gif0” > > ifconfig_gif0="tunnel 96.245.100.201 209.51.x.y mtu 1480” > > ifconfig_gif0_ipv6="inet6 2001:470:xx06:9ea::2 2001:470:xx06:9ea::1 prefixlen 128" > > ifconfig_em0_ipv6="inet6 2001:470:xx07:9ea:1::1” > > ipv6_defaultrouter="2001:470:xx06:9ea::1" > > ipv6_gateway_enable=“YES" > > rtadvd_enable=“YES” > > > Comments? Suggestions? > While we're at it, "Routers only store network aggregation addresses in their routing tables, thus reducing the average space of a routing table to 8192 entries. This addresses the scalability issues associated with IPv4, which required every allocated block of IPv4 addresses to be exchanged between Internet routers, causing their routing tables to become too large to allow efficient routing." isn't true and neither is "These addresses are syntactically indistinguishable from unicast addresses but they address a group of interfaces. The packet destined for an anycast address will arrive at the nearest router interface. Anycast addresses are only used by routers." No idea what you'd put in place, though :) From owner-freebsd-net@FreeBSD.ORG Thu Sep 11 03:22:08 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE79B614 for ; Thu, 11 Sep 2014 03:22:07 +0000 (UTC) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.allbsd.org", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E6A9F272 for ; Thu, 11 Sep 2014 03:22:06 +0000 (UTC) Received: from alph.d.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=56) by mail.allbsd.org (8.14.9/8.14.8) with ESMTP id s8B3LeR9033292 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 11 Sep 2014 12:21:52 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.d.allbsd.org (8.14.8/8.14.8) with ESMTP id s8B3LcbC016579; Thu, 11 Sep 2014 12:21:40 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Thu, 11 Sep 2014 12:21:05 +0900 (JST) Message-Id: <20140911.122105.2066013438047221946.hrs@allbsd.org> To: dan@langille.org Subject: Re: Configuration for IPv6 over tunnel From: Hiroki Sato In-Reply-To: <14E3A97C-4FCB-4A2C-B22F-3D0849CECA2D@langille.org> References: <14E3A97C-4FCB-4A2C-B22F-3D0849CECA2D@langille.org> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.6 on Emacs 24.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Thu_Sep_11_12_21_05_2014_745)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.4 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Thu, 11 Sep 2014 12:21:56 +0900 (JST) X-Spam-Status: No, score=-98.0 required=13.0 tests=CONTENT_TYPE_PRESENT, ISO2022JP_BODY,RDNS_NONE,SPF_SOFTFAIL,USER_IN_WHITELIST autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on gatekeeper.allbsd.org Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2014 03:22:08 -0000 ----Security_Multipart(Thu_Sep_11_12_21_05_2014_745)-- Content-Type: Text/Plain; charset=iso-2022-jp Content-Transfer-Encoding: 7bit Dan Langille wrote in <14E3A97C-4FCB-4A2C-B22F-3D0849CECA2D@langille.org>: da> IPv6 Tunnel Endpoints da> Server IPv4 Address: 209.51.x.y da> Server IPv6 Address: 2001:470:xx06:9ea::1/64 da> Client IPv4 Address: 96.245.100.201 da> Client IPv6 Address: 2001:470:xx06:9ea::2/64 da> da> Routed /64: 2001:470:xx07:9ea::/64 da> da> My /etc/rc.conf includes da> da> cloned_interfaces="gif0$B!I(B da> ifconfig_gif0="tunnel 96.245.100.201 209.51.x.y mtu 1480$B!I(B da> ifconfig_gif0_ipv6="inet6 2001:470:xx06:9ea::2 2001:470:xx06:9ea::1 prefixlen 128" da> ifconfig_em0_ipv6="inet6 2001:470:xx07:9ea:1::1$B!I(B da> ipv6_defaultrouter="2001:470:xx06:9ea::1" da> ipv6_gateway_enable=$B!H(BYES" da> rtadvd_enable=$B!H(BYES$B!I(B The following line is enough for ifconfig_gif0_ipv6. A /128 configuration works but ugly: -ifconfig_gif0_ipv6="inet6 2001:470:xx06:9ea::2 2001:470:xx06:9ea::1 prefixlen 128" +ifconfig_gif0_ipv6="inet6 2001:470:xx06:9ea::2/64" Or, you do not need to configure a client side global address in subnet of the inter-router link if you use his endpoint as the default router. Reducing the number of global addresses on a box is healthy for packet filtering rule management: -ifconfig_gif0_ipv6="inet6 2001:470:xx06:9ea::2 2001:470:xx06:9ea::1 prefixlen 128" +ifconfig_gif0_ipv6="inet6 auto_linklocal" -ipv6_defaultrouter="2001:470:xx06:9ea::1" +ipv6_defaultrouter="-interface gif0" And if your box works as a router for subnet 2001:470:xx07:9ea::/64, please add subnet-router anycast address. This is mandatory in RFC: +ifconfig_em0_ipv6_alias0="inet6 2001:470:xx07:9ea::/64 anycast" I think HE's endpoint is properly configured. You can ping6 to 2001:470:xx06:9ea:: from 2001:470:xx07:9ea:1::1. -- Hiroki ----Security_Multipart(Thu_Sep_11_12_21_05_2014_745)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEABECAAYFAlQRFSEACgkQTyzT2CeTzy07LQCgoZrWd8PL/27uGob+TZs/ETto w8wAn0Qj3rSANYH41soNbyPLbbNx/bNo =oU9R -----END PGP SIGNATURE----- ----Security_Multipart(Thu_Sep_11_12_21_05_2014_745)---- From owner-freebsd-net@FreeBSD.ORG Thu Sep 11 05:48:56 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1CB8E89D; Thu, 11 Sep 2014 05:48:56 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A4A3FC6; Thu, 11 Sep 2014 05:48:55 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id s8B5mpFp054740 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 11 Sep 2014 09:48:51 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id s8B5mpOW054739; Thu, 11 Sep 2014 09:48:51 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 11 Sep 2014 09:48:51 +0400 From: Gleb Smirnoff To: Andriy Gapon Subject: Re: dhclient[357]: send_packet: Invalid argument Message-ID: <20140911054851.GQ17059@glebius.int.ru> References: <5410AE40.9040301@FreeBSD.org> <5410B026.5020703@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5410B026.5020703@FreeBSD.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Olivier =?utf-8?B?Q29jaGFyZC1MYWJi77+9?= , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2014 05:48:56 -0000 On Wed, Sep 10, 2014 at 11:10:14PM +0300, Andriy Gapon wrote: A> On 10/09/2014 23:02, Andriy Gapon wrote: A> > A> > I recently performed an upgrade from a very old HEAD to a fresh one and after A> > that I started to see a lot of the following messages: A> > A> > dhclient[357]: send_packet: Invalid argument A> > A> > This is how ktrace sees them: A> > 357 dhclient CALL sendmsg(0x7,0x7fffffffd9f0,0) A> > 357 dhclient STRU struct sockaddr { AF_INET, 82.xx.xx.xx:67 } A> > 357 dhclient RET sendmsg -1 errno 22 Invalid argument A> > A> > This where the error originates according to DTrace: A> > 0 32096 rip_output:return A> > kernel`rip_send+0x46 A> > kernel`sosend_generic+0x56c A> > kernel`sosend+0x1e A> > kernel`kern_sendit+0x201 A> > kernel`sendit+0x10c A> > kernel`sys_sendmsg+0x61 A> > kernel`amd64_syscall+0x3dd A> > kernel`0xffffffff808414ab A> > A> > Looking at rip_output() I discovered r270929, https://wiki.freebsd.org/SOCK_RAW A> > Seems like the base DHCP client is affected? A> A> Oh, I've just actually noticed it: sbin/dhclient/packet.c, assemble_udp_ip_header() A> A> /* A> * While the BPF -- used for broadcasts -- expects a "true" IP header A> * with all the bytes in network byte order, the raw socket interface A> * which is used for unicasts expects the ip_len field to be in host A> * byte order. In both cases, the checksum has to be correct, so this A> * is as good a place as any to turn the bytes around again. A> */ A> if (to != INADDR_BROADCAST) A> ip.ip_len = ntohs(ip.ip_len); Fixed, thanks. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Thu Sep 11 17:50:28 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 203EF8FC; Thu, 11 Sep 2014 17:50:28 +0000 (UTC) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "nyi.unixathome.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B685BD98; Thu, 11 Sep 2014 17:50:27 +0000 (UTC) Received: from nyi.unixathome.org (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id 78CB55082E; Thu, 11 Sep 2014 17:50:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by nyi.unixathome.org (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WaA53bVauDfU; Thu, 11 Sep 2014 17:50:10 +0000 (UTC) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id E49DF50823 ; Thu, 11 Sep 2014 17:50:09 +0000 (UTC) Content-Type: multipart/signed; boundary="Apple-Mail=_FCF60D1E-2926-473B-9FC6-86C250078C0D"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Configuration for IPv6 over tunnel From: Dan Langille In-Reply-To: <20140911.122105.2066013438047221946.hrs@allbsd.org> Date: Thu, 11 Sep 2014 13:50:01 -0400 Message-Id: <94C9C202-EFEC-4689-A5CF-B3E6FE20F4CC@langille.org> References: <14E3A97C-4FCB-4A2C-B22F-3D0849CECA2D@langille.org> <20140911.122105.2066013438047221946.hrs@allbsd.org> To: Hiroki Sato X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2014 17:50:28 -0000 --Apple-Mail=_FCF60D1E-2926-473B-9FC6-86C250078C0D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-2022-jp On Sep 10, 2014, at 11:21 PM, Hiroki Sato wrote: > Dan Langille wrote > in <14E3A97C-4FCB-4A2C-B22F-3D0849CECA2D@langille.org>: >=20 > da> IPv6 Tunnel Endpoints > da> Server IPv4 Address: 209.51.x.y > da> Server IPv6 Address: 2001:470:xx06:9ea::1/64 > da> Client IPv4 Address: 96.245.100.201 > da> Client IPv6 Address: 2001:470:xx06:9ea::2/64 > da> > da> Routed /64: 2001:470:xx07:9ea::/64 > da> > da> My /etc/rc.conf includes > da> > da> cloned_interfaces=3D"gif0=1B$B!I=1B(B > da> ifconfig_gif0=3D"tunnel 96.245.100.201 209.51.x.y mtu 1480=1B$B!I=1B= (B > da> ifconfig_gif0_ipv6=3D"inet6 2001:470:xx06:9ea::2 = 2001:470:xx06:9ea::1 prefixlen 128" > da> ifconfig_em0_ipv6=3D"inet6 2001:470:xx07:9ea:1::1=1B$B!I=1B(B > da> ipv6_defaultrouter=3D"2001:470:xx06:9ea::1" > da> ipv6_gateway_enable=3D=1B$B!H=1B(BYES" > da> rtadvd_enable=3D=1B$B!H=1B(BYES=1B$B!I=1B(B >=20 > The following line is enough for ifconfig_gif0_ipv6. A /128 > configuration works but ugly: >=20 > -ifconfig_gif0_ipv6=3D"inet6 2001:470:xx06:9ea::2 = 2001:470:xx06:9ea::1 prefixlen 128" > +ifconfig_gif0_ipv6=3D"inet6 2001:470:xx06:9ea::2/64" >=20 > Or, you do not need to configure a client side global address in > subnet of the inter-router link if you use his endpoint as the > default router. Reducing the number of global addresses on a box is > healthy for packet filtering rule management: >=20 > -ifconfig_gif0_ipv6=3D"inet6 2001:470:xx06:9ea::2 = 2001:470:xx06:9ea::1 prefixlen 128" > +ifconfig_gif0_ipv6=3D"inet6 auto_linklocal" > -ipv6_defaultrouter=3D"2001:470:xx06:9ea::1" > +ipv6_defaultrouter=3D"-interface gif0" >=20 > And if your box works as a router for subnet > 2001:470:xx07:9ea::/64, please add subnet-router anycast address. > This is mandatory in RFC: >=20 > +ifconfig_em0_ipv6_alias0=3D"inet6 2001:470:xx07:9ea::/64 anycast" >=20 > I think HE's endpoint is properly configured. You can ping6 to > 2001:470:xx06:9ea:: from 2001:470:xx07:9ea:1::1. I added in the anycast just now. Before: $ ifconfig re0 re0: flags=3D8843 metric 0 mtu = 1500 = options=3D8209b ether e0:cb:4e:24:f0:ff inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255 inet6 fe80::e2cb:4eff:fe42:f0ff%re0 prefixlen 64 scopeid 0x2=20 inet6 2001:470:xx07:9ea:1::1 prefixlen 64=20 nd6 options=3D21 media: Ethernet autoselect (1000baseT ) status: active # ifconfig re0 inet6 2001:470:xx07:9ea::/64 anycast alias After: $ ifconfig re0 re0: flags=3D8843 metric 0 mtu = 1500 = options=3D8209b ether e0:cb:4e:42:f0:ff inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255 inet6 fe80::e2cb:4eff:fe42:f0ff%re0 prefixlen 64 scopeid 0x2=20 inet6 2001:470:xx07:9ea:1::1 prefixlen 64=20 inet6 2001:470:xx07:9ea:: prefixlen 64 anycast=20 nd6 options=3D21 media: Ethernet autoselect (1000baseT ) status: active Then I manually configured my Macbook to have: Router: 2001:470:xx07:9ea:1::1 IPv6 Address: 2001:470:xx07:9ea:1::1111 Prefix length: 64 $ ifconfig gif0 gif0: flags=3D8051 metric 0 mtu 1480 tunnel inet 96.245.xx.yy --> 209.51.161.14 inet6 fe80::21b:21ff:fe51:ab2d%gif0 prefixlen 64 scopeid 0xd=20 inet6 2001:470:xx06:9ea::2 --> 2001:470:xx06:9ea::1 prefixlen = 128=20 nd6 options=3D21 options=3D1 Let=1B$B!G=1B(Bs see how this goes. --Apple-Mail=_FCF60D1E-2926-473B-9FC6-86C250078C0D Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iKYEARECAGYFAlQR4MlfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDA3REZBQjJGRUQ3NEE5QkE0NTNGOUJCNzBB MEIxNzE0Q0ZGQjlEM0MACgkQCgsXFM/7nTyHaACg9HINSdC4pzkuRjCfR7E3OM4t nuIAnAvzJJvZS+KP6NVpKd5vjWxoZpt5 =omYV -----END PGP SIGNATURE----- --Apple-Mail=_FCF60D1E-2926-473B-9FC6-86C250078C0D-- From owner-freebsd-net@FreeBSD.ORG Thu Sep 11 18:13:36 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A677E486 for ; Thu, 11 Sep 2014 18:13:36 +0000 (UTC) Received: from mail-ig0-x236.google.com (mail-ig0-x236.google.com [IPv6:2607:f8b0:4001:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76E9591 for ; Thu, 11 Sep 2014 18:13:36 +0000 (UTC) Received: by mail-ig0-f182.google.com with SMTP id h18so3280143igc.15 for ; Thu, 11 Sep 2014 11:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5rveQ2oPj39ACn1UWUzvAA/OWrDI9cgrI4GeA/mR8r8=; b=0UK0Ts41jfjHf4520LkbFkn2wOIfswqeoJkq0mbEASJjkHXnwuo0W1EjH8IbyPh9Y9 XQwnpuAF7BA2IWyON/Q0kYGilSDJPaT86m9atgQS7ITCt0m8nb6YDOB64Xct0ZI3Br96 8SngLroN1tJtZ6eJz+T6NMqimymfR6NJuwcgB9FVCpj4UWEg5rP23D1yXTswsgLOQdr9 y1YL3GSzRyVgxLz6aNsD7m6on9rLzQKpu2QKGyNMaswcm+O1UoysklH5gPcrzVuzvSRB iy91k4+SFtVM4Mvw0Aep0l5quIwqFlT88cJwqXkz3tM5XCXQcZUR3rrVXQX8BdxtchXg kdRQ== MIME-Version: 1.0 X-Received: by 10.43.136.134 with SMTP id ik6mr4151416icc.6.1410459215674; Thu, 11 Sep 2014 11:13:35 -0700 (PDT) Received: by 10.107.8.132 with HTTP; Thu, 11 Sep 2014 11:13:35 -0700 (PDT) Date: Thu, 11 Sep 2014 12:13:35 -0600 Message-ID: Subject: netmap and jumbo packets From: David To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2014 18:13:36 -0000 Hi, I am learning to use netmap. Currently I got pkt-gen working (without drivers). When I try to set packet size above 2048 I got an error. Searching through the code I found in sys/dev/netmap/netmap_mem2.c the size constrain. I change it like this: diff --git a/sys/dev/netmap/netmap_mem2.c b/sys/dev/netmap/netmap_mem2.c index a15a8c7..8c561cb 100644 --- a/sys/dev/netmap/netmap_mem2.c +++ b/sys/dev/netmap/netmap_mem2.c @@ -25,6 +25,7 @@ #ifdef linux #include "bsd_glue.h" +#include #endif /* linux */ #ifdef __APPLE__ @@ -54,7 +55,7 @@ __FBSDID("$FreeBSD: head/sys/dev/netmap/netmap.c 241723 2012-1 #include #include "netmap_mem2.h" -#define NETMAP_BUF_MAX_NUM 20*4096*2 /* large machine */ +#define NETMAP_BUF_MAX_NUM 10*4096*2 /* large machine */ #define NETMAP_POOL_MAX_NAMSZ 32 @@ -183,7 +184,7 @@ struct netmap_obj_params netmap_params[NETMAP_POOLS_NR] =3D { .num =3D 200, }, [NETMAP_BUF_POOL] =3D { - .size =3D 2048, + .size =3D 8192, .num =3D NETMAP_BUF_MAX_NUM, }, }; (first change is not related) I had to change NETMAP_BUF_MAX_NUM because of the amount of memory needed. This way I can send packets as big as 8192 (which is what I need). Is there any problems this will bring on? (maybe decrease the performance or any other side effect) thanks --=20 David D=C3=ADaz Barquero Ingenier=C3=ADa en Computadores Tecnol=C3=B3gico de Costa Rica From owner-freebsd-net@FreeBSD.ORG Thu Sep 11 18:17:53 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF33B708 for ; Thu, 11 Sep 2014 18:17:53 +0000 (UTC) Received: from mail-ig0-x22b.google.com (mail-ig0-x22b.google.com [IPv6:2607:f8b0:4001:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A8D4D0 for ; Thu, 11 Sep 2014 18:17:53 +0000 (UTC) Received: by mail-ig0-f171.google.com with SMTP id r10so1439279igi.16 for ; Thu, 11 Sep 2014 11:17:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=oXIPGsw60ToNko8rCZgV/hmdrZjvOeXVW8rM95gNMMo=; b=Zr9NZf3uvJznuIYjRs5S+4mh8mnj3Oq3n+ueFyQfrOezq+3zsJY/qEnMzRFsM9miHu p2Fj0J03mW7d/1ZHGD9qFs/HhNvadPG7DIQmzML+mt3yolgOP4aSD1vDJcEqrvaH7If3 JTgHe56x/oj+9m9R0VA5CenN3Sv6vAgQ/s7cIOYGSVuZeG91I8cjC8/QitgInwWJwqZ6 d283s8Z7cCgWtuLvE6IcBl8LqW7hLbEp9je1c2DBiS7Lpaw5+n/0Y8ZZaZUAkKSC3y0Q b7if1NDSIIItJDE9yrxx2RDO5vZ5UqAV4H+AaHnDE45KO5GlLcyAwGD9zcDSLKrEs0vJ cVKA== MIME-Version: 1.0 X-Received: by 10.50.41.8 with SMTP id b8mr4815955igl.11.1410459472873; Thu, 11 Sep 2014 11:17:52 -0700 (PDT) Received: by 10.107.8.132 with HTTP; Thu, 11 Sep 2014 11:17:52 -0700 (PDT) In-Reply-To: References: Date: Thu, 11 Sep 2014 12:17:52 -0600 Message-ID: Subject: Re: help porting netmap to new driver From: David To: Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2014 18:17:53 -0000 I was able to work this around. The problem was in the driver. Now I start working on customizing the driver to use netmap optimized regards 2014-09-08 9:54 GMT-06:00 David : > Hi > > Sorry for the late response, I'll be using an ARM processor for a 1G link= . > > My first issue came when doing what you suggested, and try the "emulated > netmap mode". I need to cross compile the code from my x86, which I think= I > did correctly by setting up my arm toolchain and passing the kernel sourc= e > when doing the "make". > > Now when I run the pkt-gen example I get : > > [ 57.602575] Unable to handle kernel paging request at virtual address > f143ca04 > [ 57.609740] pgd =3D 894c4000 > [ 57.612420] [f143ca04] *pgd=3D00000000 > [ 57.615973] Internal error: Oops: 5 [#1] PREEMPT SMP ARM > > Where I'm stuck since a while ago trying to figure it out. > > regards > > 2014-08-27 16:23 GMT-06:00 Luigi Rizzo : > > >> >> >> On Wed, Aug 27, 2014 at 3:18 PM, David wrote: >> >>> Hi, >>> >>> I'm needing to use netmap on a custom driver, I don't understand the >>> content of the functions I need to implement and that are detailed on >>> "PORTING" file. >>> >>> >> =E2=80=8Bsometimes (often, actually) the hw has bottlenecks that make na= tive >> netmap mode almost useless. >> =E2=80=8BOne thing you could try is the "emulated netmap mode" which >> is used by default=E2=80=8B if the driver has no native support. >> This is in the code on code.google.com/p/netmap/ , >> the branch "next" (which is basically the code now in FreeBSD) >> has some fixes for that specific feature >> >> cheers >> luigi >> >> >> can someone give a hand to understand it better? >>> >>> regards >>> >>> -- >>> David D=C3=ADaz Barquero >>> >>> Ingenier=C3=ADa en Computadores >>> Tecnol=C3=B3gico de Costa Rica >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> >> >> -- >> -----------------------------------------+------------------------------= - >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2211611 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------= - >> > > > > -- > David D=C3=ADaz Barquero > > Ingenier=C3=ADa en Computadores > Tecnol=C3=B3gico de Costa Rica > --=20 David D=C3=ADaz Barquero Ingenier=C3=ADa en Computadores Tecnol=C3=B3gico de Costa Rica From owner-freebsd-net@FreeBSD.ORG Thu Sep 11 21:59:59 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4038B271 for ; Thu, 11 Sep 2014 21:59:59 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3C79B52 for ; Thu, 11 Sep 2014 21:59:58 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id ty20so9326198lab.9 for ; Thu, 11 Sep 2014 14:59:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=UkOR+mPCH3nbGzBVNqauodF3UYPTUeMHjjdct1kKn1g=; b=TOHYIuc4kJWMD68zMy49ImIXwXFG8gfHm9NblfzPA3hRBpSy9zmZJ3hUxyjPnJyNIT HiIBUSQXJx6ANeZYLNvuWbHsJkdcjWn1tR3mytSg9savNJ+ZHCKV9vzaEtM6xRx/4xTa +fSOXyCuBMv0TilhvAj9gHMJh3N50YYQVMApsZWzD98jd6QphVgVyVAeB8JQ/JmreuVC LpCwLSbHVWmTJBi5s81B88r7zTGhiShZKvMm9Sj6FoL2gmvkJ1kIp8l6QOGVuVskDo82 r2MvP0fwZmRncng34tDoGm5uxwxg5cCayIJYx8Q8UvVh7Jq6RUGww1JnfHZGM2a24KU2 cfdA== MIME-Version: 1.0 X-Received: by 10.112.34.143 with SMTP id z15mr4150630lbi.86.1410472795672; Thu, 11 Sep 2014 14:59:55 -0700 (PDT) Received: by 10.25.21.166 with HTTP; Thu, 11 Sep 2014 14:59:55 -0700 (PDT) Date: Thu, 11 Sep 2014 17:59:55 -0400 Message-ID: Subject: [PATCH] Fix integer truncation in systat -ifstat From: Ryan Stone To: freebsd-net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2014 21:59:59 -0000 systat -ifstat currently truncates byte counters down to 32-bit integers. The following fixes the issue, but I'm not very happy with it. u_long is what the rest of our code uses for network counters, but that ends up meaning that our counters are 32-bits wide on 32-bit platforms. I could make it uint64_t but that's not very future proof. RIght now I'm leaning towards punting on the issue and using u_long as there is an awful lot of code that would have to be modified for extended byte counters to actually work on all platforms. [rstone@rstone-laptop systat]svn diff Index: ifstat.c =================================================================== --- ifstat.c (revision 271439) +++ ifstat.c (working copy) @@ -269,8 +269,8 @@ struct if_stat *ifp = NULL; struct timeval tv, new_tv, old_tv; double elapsed = 0.0; - u_int new_inb, new_outb, old_inb, old_outb = 0; - u_int new_inp, new_outp, old_inp, old_outp = 0; + u_long new_inb, new_outb, old_inb, old_outb = 0; + u_long new_inp, new_outp, old_inp, old_outp = 0; SLIST_FOREACH(ifp, &curlist, link) { /* From owner-freebsd-net@FreeBSD.ORG Fri Sep 12 02:04:25 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DFE899BA for ; Fri, 12 Sep 2014 02:04:25 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 970BD60C for ; Fri, 12 Sep 2014 02:04:24 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-235-108.lns20.per1.internode.on.net [121.45.235.108]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id s8C24DCY058233 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 11 Sep 2014 19:04:16 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <54125491.8010708@freebsd.org> Date: Fri, 12 Sep 2014 10:04:01 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John Case , freebsd-net@freebsd.org Subject: Re: How can sshuttle be used properly with FreeBSD (and with DNS) ? References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 02:04:26 -0000 On 9/6/14, 10:52 AM, John Case wrote: > > I would like to use sshuttle (http://github.com/apenwarr/sshuttle) > on FreeBSD. > > I have it working for TCP connections, but it does not properly > tunnel DNS requests. The documentation for sshuttle says that ipfw > forward rules will not properly forward UDP packets, and so when it > runs on FreeBSD, sshuttle inserts divert rules instead. The project > author believes that this will work properly (inserting divert rules > to tunnel UDP) but I am not having any success. > > BUT, I already have a divert rule (and natd running) on this system > even before I run sshuttle at all - because the system won't > function as a normal gateway unless I use divert/natd. I prefer to > run a gateway without divert/natd, but since both sides of this > gateway are non-routable IPs, I cannot do that - in order to > function as a gateway with 10.x.x.x networks on both sides, I need > to run natd/divert. > > So that means that when sshuttle inserts its own divert rules, they > conflict with the existing ones, and I am not running a second natd > daemon, so I think it all just falls apart. > > How can this be fixed ? > > Is anyone out there using sshuttle on FreeBSD with the --dns switch ? > > Here is what my ipfw.conf looks like BEFORE I run sshuttle: > > > add 1000 divert natd ip from any to any in via xl0 > add 2000 divert natd ip from any to any out via xl0 > > and in rc.conf: > > > gateway_enable="yes" > natd_enable="yes" > natd_interface="xl0" > > > Again, this works fine - I have a functioning internet gateway and > both of the interfaces on it have non-routable IP address. > > Then I run sshuttle and it *also* works fine - but only for TCP. It > does not tunnel UDP (dns) properly like it is supposed to, and I > think the reason is that I already have diverting/natd going on and > then I run sshuttle and it inserts another two divert rules into ipfw. > > But I am not sure wha the fix would be ... what's on the other end of the link? I do similar but I use the built in ppp daemon, piping it through an ssh pipe. No extra components needed (if both ends are FreeBSD, or both ends can take a tcp session as transport for their ppp implementation.) > > Thanks. > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Fri Sep 12 05:59:44 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FC26D7E for ; Fri, 12 Sep 2014 05:59:44 +0000 (UTC) Received: from mx12.netapp.com (mx12.netapp.com [216.240.18.77]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx12.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C6EFC34 for ; Fri, 12 Sep 2014 05:59:44 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.04,511,1406617200"; d="asc'?scan'208";a="189352146" Received: from hioexcmbx06-prd.hq.netapp.com ([10.122.105.39]) by mx12-out.netapp.com with ESMTP; 11 Sep 2014 22:59:44 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx06-prd.hq.netapp.com (10.122.105.39) with Microsoft SMTP Server (TLS) id 15.0.913.22; Thu, 11 Sep 2014 22:59:28 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::79b6:a9a2:f48b:f065%21]) with mapi id 15.00.0913.011; Thu, 11 Sep 2014 22:59:28 -0700 From: "Eggert, Lars" To: Luigi Rizzo Subject: netmap wishlist Thread-Topic: netmap wishlist Thread-Index: AQHPzk61Hrt2PZpnokKRDfZWlEftWg== Date: Fri, 12 Sep 2014 05:59:27 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.1878.6) x-originating-ip: [10.120.60.34] Content-Type: multipart/signed; boundary="Apple-Mail=_E2A7E95E-A439-4D85-8545-B52EEDCCC627"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 05:59:44 -0000 --Apple-Mail=_E2A7E95E-A439-4D85-8545-B52EEDCCC627 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Luigi, I've started to play with netmap, like it a lot, and would like it to = grow support for some additional features that I'd need. I wonder if you = could comment on how likely support for any of the following is in = netmap in the foreseeable future? * IP/TCP/UDP checksum offload * TCP/UDP segmentation offload * TCP/UDP large receive offload * jumbograms (I saw the email earlier today, so maybe that's addressed) Thanks, Lars --Apple-Mail=_E2A7E95E-A439-4D85-8545-B52EEDCCC627 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBVBKLx9ZcnpRveo1xAQJm6QQAmtPxvLmZKwoLsUTQ04PvM1PLReoy7eBX cGfpvarco50Uu3oLVqh+MJs/HWKxhPvlhq9k2il5Jk9vyFVse7o1qn/yp4a0OzmL RVwFOzXpRnXdhdx5sTVEP0bIMWKc3tJuQSRVZ0mx5obMN8QmdAiPLhVxf69bs/QX Qkke3M0euPo= =bXCK -----END PGP SIGNATURE----- --Apple-Mail=_E2A7E95E-A439-4D85-8545-B52EEDCCC627-- From owner-freebsd-net@FreeBSD.ORG Fri Sep 12 07:10:12 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C92C2DDC for ; Fri, 12 Sep 2014 07:10:12 +0000 (UTC) Received: from mail105.syd.optusnet.com.au (mail105.syd.optusnet.com.au [211.29.132.249]) by mx1.freebsd.org (Postfix) with ESMTP id 8BB55325 for ; Fri, 12 Sep 2014 07:10:11 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail105.syd.optusnet.com.au (Postfix) with ESMTPS id 8C3E91041D82; Fri, 12 Sep 2014 16:41:43 +1000 (EST) Date: Fri, 12 Sep 2014 16:41:42 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Ryan Stone Subject: Re: [PATCH] Fix integer truncation in systat -ifstat In-Reply-To: Message-ID: <20140912154903.L923@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=BdjhjNd2 c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=NN3ANoA1gPMA:10 a=B5yeGf3Ib8EA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=e0rXU7nX-OYX5NGfszAA:9 a=CjuIK1q_8ugA:10 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 07:10:13 -0000 On Thu, 11 Sep 2014, Ryan Stone wrote: > systat -ifstat currently truncates byte counters down to 32-bit > integers. The following fixes the issue, but I'm not very happy with > it. u_long is what the rest of our code uses for network counters, > but that ends up meaning that our counters are 32-bits wide on 32-bit > platforms. I could make it uint64_t but that's not very future proof. > RIght now I'm leaning towards punting on the issue and using u_long as > there is an awful lot of code that would have to be modified for > extended byte counters to actually work on all platforms. Only differences in the counters are used except in 1 place that is broken in other ways, so overflow is only a large problem starting at about 40 Gbps. At only 10 Gbps, 32-bit counters are enough with a refresh interval of 1 second but not quite enough with the default interval of 5 seconds (this default is not documented in the man page. It seems to only be documented (with a grammar error -- comma splice) in the status message for mode switches). 5 seconds at nearly 1.125 GBps exceeds UINT32_MAX. Packet counter overflow isn't a problem until about 600 Gbps with the default interval. 32-bit systems would have other problems supporting 600 GBps interfaces. > [rstone@rstone-laptop systat]svn diff > Index: ifstat.c > =================================================================== > --- ifstat.c (revision 271439) > +++ ifstat.c (working copy) > @@ -269,8 +269,8 @@ > struct if_stat *ifp = NULL; > struct timeval tv, new_tv, old_tv; > double elapsed = 0.0; > - u_int new_inb, new_outb, old_inb, old_outb = 0; > - u_int new_inp, new_outp, old_inp, old_outp = 0; > + u_long new_inb, new_outb, old_inb, old_outb = 0; > + u_long new_inp, new_outp, old_inp, old_outp = 0; > > SLIST_FOREACH(ifp, &curlist, link) { > /* u_long was technically and practically correct in 1990 when long was the largest integer type and the kernel counters had type u_long. Except u_long was too large then (it should actually be long, thus 2*32 bits on 32-bit machines, making it too large and slow to use for almost anything including these counters then). Now the counters have type uint64_t in the kernel, but apparently not many applications kept up with this change (I think netstat did). DIfferences between these counters are assigned to struct member variables like if_in_curtraffic. These haven't kept up with the change either, but they were correct in 1990 since they have type u_long. The place that is broken in other ways: % /* Display interface if it's received some traffic. */ % if (new_inb > 0 && old_inb == 0) { % ifp->display = 1; % needsort = 1; % } The bugs here are very minor: - "it's" expands to "it is", so it is a grammar and/or semantics error - in the long term, old_inb always overflows if the interface is used at all. Sometimes it overflows to precisely 0. This breaks the logic for detecting the first activity on the interface. But the result of misdetcting non-first activity as first seems to be harmless -- just sort and redisplay. Bruce From owner-freebsd-net@FreeBSD.ORG Fri Sep 12 07:31:22 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 12C6E204 for ; Fri, 12 Sep 2014 07:31:22 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78D787C4 for ; Fri, 12 Sep 2014 07:31:21 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id z11so368832lbi.40 for ; Fri, 12 Sep 2014 00:31:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=YUW8jVFBEQHNWCE+hCak4jmWR6J9XGWlUi0mv/lBjqA=; b=nw6y4D0JJJJ7w4LKztLFSCrDJm8UXpEcKlZTPRyqjUv6SstgTrPi3Ojz/gvqD/gXNn R7iGX3SQvSWJlhaob1L7Wq+DgQsrDd3l8v1vDHvThlsIbs2OoFUHcP6r0a849gc6eLr6 82qM4/oxu7Rc0En/YFvQizDj9/pdw8L9HdHldRTg/8ApV2HniN18pQzRIC54k7T1jc6L k/J0xyePZH5YCd2cqSw1LiJfywKGt+SAMojkInj3vFXNZNgCvwBi6P92fkUI1m5/FFnD W/NbH9MDHs7wWQhK01l/rTdE+zq9t76CUVoOu+ZsGxUIL9ooztqWJJzbLtjsXlrD1XtM Ry4g== MIME-Version: 1.0 X-Received: by 10.152.42.136 with SMTP id o8mr6618099lal.71.1410507079435; Fri, 12 Sep 2014 00:31:19 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.26.37 with HTTP; Fri, 12 Sep 2014 00:31:19 -0700 (PDT) In-Reply-To: References: Date: Fri, 12 Sep 2014 09:31:19 +0200 X-Google-Sender-Auth: uh9j1Jihg2fYq1jwXb_gZJJitxI Message-ID: Subject: Re: netmap wishlist From: Luigi Rizzo To: "Eggert, Lars" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 07:31:22 -0000 On Fri, Sep 12, 2014 at 7:59 AM, Eggert, Lars wrote: > Hi Luigi, > > I've started to play with netmap, like it a lot, and would like it to gro= w > support for some additional features that I'd need. I wonder if you could > comment on how likely support for any of the following is in netmap in th= e > foreseeable future? > > * IP/TCP/UDP checksum offload > * TCP/UDP segmentation offload > * TCP/UDP large receive offload > * jumbograms (I saw the email earlier today, so maybe that's addressed) > =E2=80=8BHi Lars: there is something already available/in progress for some of the above, but here are my thoughts on the various subjects: - netmap is designed to work with large frames, by setting the buffer size to something suitable (using a sysctl). There might be some lurking bugs (e.g. some NICs need to be told about the maximum frame size or they will refuse to send/receive them even though the slot in the NIC ring specifies a large buffer), but this is trivial to fix on a case by case basis. The downside is some waste on buffers (they are fixed size so having to allocate say 16K for a 64 byte frame is a bit annoying). - checksums offloading can be added trivially in the *_txsync(), once again on a per-nic basis. Problem is, is we start adding per-packet features (say, checksums, scatter-gather I/O, segmentation) in the inner loop of *_txsync() we are going to lose some performance for high rate applications. Now we are running at about 20ns/pkt (because we assume a flat data format), having a few extra conditionals in the inner loop could easily eat another 5..20ns/pkt, and this makes me a bit uncomfortable, especially because the situations where these offloadings matter are typically with large packets, where we are not CPU bound. - the VALE switch has support for segmentation and checksum avoidance. Clients can register as virtio-net capable: in this case the port will accept/deliver large segments across that port, and do segmentation and checksum as required for ports that are not virtio-net enabled (e.g. physical NICs attached to the same VALE switch). This was developed earlier this year by Vincenzo Maffione. At the moment this only works on top of VALE ports, not NICs, and the reason is that there is a big win if the VM can deliver a large segments in one shot to another local VM. Much less useful if you are talking across a physical device, in which case the OS should be able to do a reasonable job in segmenting packets (see also next item). We could probably leverage this code to work also on top of NICs connected through netmap, e.g. programming the NIC to use its own native offloading, but i am skeptical about the usefulness and concerned about the potential performance loss in *_txsync(). - Stefano Garzarella has some code to do software GSO (this is for FreeBSD, linux already has something similar), which will be presented at EuroBSDCon later this month in Sofia. This should address the segmentation issue on the host stack. - on the receive side, both FreeBSD and Linux have an efficient RLO software fallback in case the NIC does not support it natively, i think we do not need this at the NIC/switch level. cheers luigi From owner-freebsd-net@FreeBSD.ORG Fri Sep 12 07:51:03 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F92E6BE for ; Fri, 12 Sep 2014 07:51:03 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx11.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A1B18EE for ; Fri, 12 Sep 2014 07:51:02 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.04,511,1406617200"; d="asc'?scan'208";a="147240790" Received: from hioexcmbx06-prd.hq.netapp.com ([10.122.105.39]) by mx11-out.netapp.com with ESMTP; 12 Sep 2014 00:51:02 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx06-prd.hq.netapp.com (10.122.105.39) with Microsoft SMTP Server (TLS) id 15.0.913.22; Fri, 12 Sep 2014 00:50:52 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::79b6:a9a2:f48b:f065%21]) with mapi id 15.00.0913.011; Fri, 12 Sep 2014 00:50:52 -0700 From: "Eggert, Lars" To: Luigi Rizzo Subject: Re: netmap wishlist Thread-Topic: netmap wishlist Thread-Index: AQHPzk61Hrt2PZpnokKRDfZWlEftWpv9j9GAgAAFf4A= Date: Fri, 12 Sep 2014 07:50:52 +0000 Message-ID: <6D7C4630-98FA-45C9-A8DE-357065E47F0C@netapp.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.1878.6) x-originating-ip: [10.122.56.79] Content-Type: multipart/signed; boundary="Apple-Mail=_7B1F0C82-0090-4A4D-984A-12DB8F989C5E"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 07:51:03 -0000 --Apple-Mail=_7B1F0C82-0090-4A4D-984A-12DB8F989C5E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, On 2014-9-12, at 9:31, Luigi Rizzo wrote: > there is something already available/in progress for some of the = above, > but here are my thoughts on the various subjects: >=20 > - netmap is designed to work with large frames, by setting the buffer > size to something suitable (using a sysctl). ... > The downside is some waste on buffers (they are fixed size so = having > to allocate say 16K for a 64 byte frame is a bit annoying). that's OK for what I'm doing. > - checksums offloading can be added trivially in the *_txsync(), > once again on a per-nic basis. > Problem is, is we start adding per-packet features (say, checksums, > scatter-gather I/O, segmentation) in the inner loop of *_txsync() > we are going to lose some performance for high rate applications. What about making these things compile-time options? I totally see that = if you want to use netmap for fast switching, you wouldn't want these. = But if you use netmap for operating on IP and transport protocol = packets, they become really essential. (Esp. at 40G - which reminds me = that I forgot to add netmap support for the ixl driver to the = wishlist...) > - the VALE switch has support for segmentation and checksum avoidance. > Clients can register as virtio-net capable: in this case the port = will > accept/deliver large segments across that port, and do segmentation = and > checksum as required for ports that are not virtio-net enabled > (e.g. physical NICs attached to the same VALE switch). > This was developed earlier this year by Vincenzo Maffione. I may look into this. I'm unclear if adding a VALE layer into the system = just to get this feature would be wort it in terms of performance. > We could probably leverage this code to work also on top of NICs > connected through netmap, e.g. programming the NIC to use its own > native offloading, but i am skeptical about the usefulness and > concerned about the potential performance loss in *_txsync(). I totally see that, but maybe a compile-time option would work. There = are several distinct use cases for netmap at the moment, and it's = unlikely that the same system would need to support several of them, so = compile-time specialization may be sufficient here. > - Stefano Garzarella has some code to do software GSO (this is for = FreeBSD, > linux already has something similar), which will be presented at > EuroBSDCon later this month in Sofia. This should address the > segmentation issue on the host stack. Nice, I will take a look. > - on the receive side, both FreeBSD and Linux have an efficient > RLO software fallback in case the NIC does not support it > natively, i think we do not need this at the NIC/switch level. OK, I need to look into this. Oh, and my list was prioritized - I think the checksum offload would be = the real winner when dealing munging IP and transport packets. Thanks, Lars --Apple-Mail=_7B1F0C82-0090-4A4D-984A-12DB8F989C5E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBVBKl5NZcnpRveo1xAQLDdgQAsLU+tmwjVgEeFBChqSq+74yyMZSc7sOU ggV/qLBaazcY0514o5P3KoHCKU9qLIytZ6+u5Me4TckUgJ3YADxEH9TzYW7drs7t qQpUtaJcqFs7tV+BoBU7gNBFxa9+7mki5LBrDSiTMmRiKzmowijBTqDEN7lKvSX6 UaJk0tJYgVE= =bSa6 -----END PGP SIGNATURE----- --Apple-Mail=_7B1F0C82-0090-4A4D-984A-12DB8F989C5E-- From owner-freebsd-net@FreeBSD.ORG Fri Sep 12 08:10:07 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26E8B8AB for ; Fri, 12 Sep 2014 08:10:07 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B37AAA80 for ; Fri, 12 Sep 2014 08:10:06 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id x13so326240wgg.21 for ; Fri, 12 Sep 2014 01:10:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=iON2UB0DkibZCrzzT/Sd5K8NcItv/gGbntT9quM6xL8=; b=x7xW3oYguPCaaKwNfrQyTW894J2A8DRkOqUQh+MvrINNkes9/bfC0Ufcl/4yR4JFa9 rF2NlxY32o9i7tIioMmNlaKgLLSNLotjX0rGvmNG2Yba2Q1eBEa8tJzRHklLOLfVGzeO EFinqlhaJAR1E6jJqf8MwHLclAP9j9rbmQaAiqfxlxVYFxDpCYru5PYFCZJ50D2/aEIl j4C8kjHQ06JEQS4vBsV7Pn//+ak3opZkkl8YiNI4NvWD43+0GNuAssw3Psi0kcaR+iNv fhgbTUySSUCIX63Z5DkwHNJ60NRXi1SbK7a5Vw/6mf+Hc5UXhVcWER+xOn0ny8mgd0sn dJXg== X-Received: by 10.194.59.18 with SMTP id v18mr9101581wjq.64.1410509404837; Fri, 12 Sep 2014 01:10:04 -0700 (PDT) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.194.90.2 with HTTP; Fri, 12 Sep 2014 01:09:44 -0700 (PDT) In-Reply-To: <20140912154903.L923@besplex.bde.org> References: <20140912154903.L923@besplex.bde.org> From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Fri, 12 Sep 2014 10:09:44 +0200 X-Google-Sender-Auth: nEzRZC6f6vpOrx2cFDZIJKCKiyc Message-ID: Subject: Re: [PATCH] Fix integer truncation in systat -ifstat To: Bruce Evans Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-net , Ryan Stone X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 08:10:07 -0000 On Fri, Sep 12, 2014 at 8:41 AM, Bruce Evans wrote: > Only differences in the counters are used except in 1 place that is > broken in other ways, so overflow is only a large problem starting at > about 40 Gbps. At only 10 Gbps, 32-bit counters are enough with a > refresh interval of 1 second but not quite enough with the default > interval of 5 seconds (this default is not documented in the man > page. It seems to only be documented (with a grammar error -- comma > splice) in the status message for mode switches). 5 seconds at > nearly 1.125 GBps exceeds UINT32_MAX. Packet counter overflow isn't > a problem until about 600 Gbps with the default interval. 32-bit > systems would have other problems supporting 600 GBps interfaces. > I confirm this behavior reported in this PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182448 From owner-freebsd-net@FreeBSD.ORG Fri Sep 12 17:02:40 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 857EEB5A for ; Fri, 12 Sep 2014 17:02:40 +0000 (UTC) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56187C06 for ; Fri, 12 Sep 2014 17:02:40 +0000 (UTC) Received: by mail-ie0-f177.google.com with SMTP id rd18so1266167iec.22 for ; Fri, 12 Sep 2014 10:02:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ZE5of5+N7+hEo8vPl1FJ5qVD/DTgX6uQ58OCxFSq9Cw=; b=qP2JfnVX0wzCnqtZngZuablCge6BR8IhwU3joRuzEoHznFe01dWz4V7+fxoafs4Gpq t0kcmLMxTdVCb0EZEV370ueAcPBnM6nHWfUfA9hMT4KjSshboNBsFQZJbFZzcVU7Xkhs ryyxSeZ2nS8SwRd1RQZ7Qg3yWuiE7ungTTkBfZuZnAmYI4g/vZRJZxhCIkINa35wxXgS 7/7e6fFxf4HPZ5mOg4rSg6Vn+2KKsRIbmEPOz5rrWJAEfTzCpQo7QFji+HMU+phjy6mo elc48zPYcyHP0k7U/eH6IVB+8fmMLB4Uo7DuFsoms4TZXBisNIvW1TqKdvRWPjg3ivqZ mo9A== MIME-Version: 1.0 X-Received: by 10.50.88.98 with SMTP id bf2mr3904551igb.11.1410541359774; Fri, 12 Sep 2014 10:02:39 -0700 (PDT) Received: by 10.107.8.132 with HTTP; Fri, 12 Sep 2014 10:02:39 -0700 (PDT) Date: Fri, 12 Sep 2014 11:02:39 -0600 Message-ID: Subject: help creating a send-receive test suit using netmap From: David To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 17:02:40 -0000 Hi I am trying to compare the performance of sending packets using netmap, socket and packet mmap. Right now I am working on top of pkt-gen and some other implementations for socket and packet mmap. I'm interested on the relation between packet size and packets I can send per second. I was following the code to check all the steps, and I found it is going throught a memcpy to set the packet on the netmap buffer (as defined on nm_pkt_copy inside netmap_user.h). I am afraid that the data I'm reading is mostly driven by the time the memcpy takes. Is there any other way around to check the performance in "send" to avoid this bottleneck? regards --=20 David D=C3=ADaz Barquero Ingenier=C3=ADa en Computadores Tecnol=C3=B3gico de Costa Rica From owner-freebsd-net@FreeBSD.ORG Sat Sep 13 20:09:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8281EE8A; Sat, 13 Sep 2014 20:09:55 +0000 (UTC) Received: from mail-pd0-x229.google.com (mail-pd0-x229.google.com [IPv6:2607:f8b0:400e:c02::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D4A32EB; Sat, 13 Sep 2014 20:09:55 +0000 (UTC) Received: by mail-pd0-f169.google.com with SMTP id fp1so3628861pdb.14 for ; Sat, 13 Sep 2014 13:09:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=0tdqeIHeQU2SDAJs5vcUa4aDl/OyerjwIJ89xBj70Rk=; b=jzG1xDqZbvATswcBbXwkHyH7CNtpk9zLLqxjaklnYeHuflLQdgKL3BinDi/F6Ufglk eohzr756+fCm0a2fNSzO2wBe+E9lYjE7vTZTA7oPdVpNqjYbqGFpff3GxcFnAjqeNBk+ Stff3zhdbNpQimDgR3s6LrlnPEtRZgvgoAqCJl5b1IQ5xq2uoIjePVdRJ2kjeiCAlIGh fP7eOF+SjIYkJWNZ4RNNzE9nsgmB7XbMiXXa0qIXXLmSm8zamhHpw9mfx4mj1XlX0456 llM91YxaXCIwvSYaUDfGGVoJsL3Ry1zqJbwOTUMRiKbQZVrGiFf3Lvtd21e1MXcKpdVi cohg== X-Received: by 10.68.161.197 with SMTP id xu5mr25661217pbb.160.1410638994808; Sat, 13 Sep 2014 13:09:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.70.55.162 with HTTP; Sat, 13 Sep 2014 13:09:14 -0700 (PDT) In-Reply-To: References: From: Eric Joyner Date: Sat, 13 Sep 2014 13:09:14 -0700 Message-ID: Subject: Re: [igb] add DROP_EN to each RX queue config if TX flow control is disabled To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Jack F Vogel , FreeBSD Net , "Joyner, Eric" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Sep 2014 20:09:55 -0000 This looks good to me. The only comment I have is that according to the I350 datasheet, DROP_EN is already set on all the queues but 0 by default, but I haven't checked for the other adapters covered by igb. --- - Eric Joyner On Mon, Sep 8, 2014 at 10:12 PM, Adrian Chadd wrote: > Hi, > > This patch enables the DROP_EN flag to each RX queue if TX flow > control is disabled. It (mostly) mirrors what the ixgbe driver does. > > This prevents a single full RX ring from stalling the rest of the RX rings. > > How's it look? (indenting and such aside, thanks google.) > > > > -a > > Index: sys/dev/e1000/if_igb.c > =================================================================== > --- sys/dev/e1000/if_igb.c (revision 271290) > +++ sys/dev/e1000/if_igb.c (working copy) > @@ -4712,6 +4712,18 @@ > rctl |= E1000_RCTL_SZ_2048; > } > > + /* > + * If TX flow control is disabled and there's >1 queue defined, > + * enable DROP. > + * > + * This drops frames rather than hanging the RX MAC for all queues. > + */ > + if ((adapter->num_queues > 1) && > + (adapter->fc == e1000_fc_none || > + adapter->fc == e1000_fc_rx_pause)) { > + srrctl |= E1000_SRRCTL_DROP_EN; > + } > + > /* Setup the Base and Length of the Rx Descriptor Rings */ > for (int i = 0; i < adapter->num_queues; i++, rxr++) { > u64 bus_addr = rxr->rxdma.dma_paddr; > @@ -6255,6 +6267,7 @@ > > adapter->hw.fc.current_mode = adapter->hw.fc.requested_mode; > e1000_force_mac_fc(&adapter->hw); > + /* XXX TODO: update DROP_EN on each RX queue if appropriate */ > return (error); > } > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Sat Sep 13 21:16:22 2014 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDA0175C for ; Sat, 13 Sep 2014 21:16:22 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B3D9EADF for ; Sat, 13 Sep 2014 21:16:22 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s8DLGMrl077742 for ; Sat, 13 Sep 2014 21:16:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 185967] [lagg] [patch] Link Aggregation LAGG: LACP not working in 10.0 Date: Sat, 13 Sep 2014 21:16:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: olivier@cochard.me X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Sep 2014 21:16:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=185967 --- Comment #11 from olivier@cochard.me --- (In reply to kvedulv from comment #10) > Hi Oliver, > > (In reply to olivier from comment #8) > > The patch didn't solve Lagg regression between 9.2 and 10-stable (r267244). > > > > My configuration only use one lagg member (the second will be added later): > > _Maybe_ #179926 is your Bug and you could try the patch over there... > (Although that problem seems to have existed at least in 9.1) I believe it's not a regression in my case: On FreeBSD 9.2, the lagg-lacp mode by not implementing a 'strict' mode was equivalent to a 'static+optionnal LACP' mode. The behavior of lagg-lacp on 9.2 seems: - If no LACP detected, allow a minimum of 1 lagg member to transmit. The behavior on 10 seems: - If no LACP detected, don't allow any of the lagg member to transmit (even with strict_mode disabled) And in my case: I didn't have a LACP device in front of my FreeBSD. Then I simply need to use a 'static' mode (like loadbalanced or roundrobin). -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-net@FreeBSD.ORG Sat Sep 13 23:20:45 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EFB7CD55; Sat, 13 Sep 2014 23:20:45 +0000 (UTC) Received: from mail.ipfw.ru (mail.ipfw.ru [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B5A73753; Sat, 13 Sep 2014 23:20:45 +0000 (UTC) Received: from v6.mpls.in ([2a02:978:2::5] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1XSsdu-000GZz-W3; Sat, 13 Sep 2014 23:05:55 +0400 Message-ID: <5414D10E.7020000@FreeBSD.org> Date: Sun, 14 Sep 2014 03:19:42 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "net@freebsd.org" Subject: if_lagg(4) accounting changes X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Sep 2014 23:20:46 -0000 Hello list. I'd like to commit some changes to lagg counters which might be worth discussion. Diff is available at https://reviews.freebsd.org/D781 Quoting its summary: While counting packets using per-cpu counters might not introduce any significant overhead at current rates, we do not need to do such accounting at all. lagg in general is pure control-plane interface, its action on receive should be just to change packet src if pointer. Its action on transmit should be just selecting output interface based on flowid. It should not generate any errors on its own. In fact, RX lagg path can be skipped by setting correct ifp inside NIC driver. TX path should be handled by generic multipath L2 nexthops inside routing code. This is first step for implementing this scenario. One side effect is that we're now collecting all counters (including errors) from underlying interfaces. Generally most networking HW vendors implement this behavior for their equipment and this is really the reasonable thing to do.