From owner-freebsd-net@freebsd.org Wed Apr 27 20:28:10 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2AFAB1E87B for ; Wed, 27 Apr 2016 20:28:10 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F4721701 for ; Wed, 27 Apr 2016 20:28:10 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-io0-x22b.google.com with SMTP id u185so66928778iod.3 for ; Wed, 27 Apr 2016 13:28:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=1Ev3mqoNtbMx651f/aw1YebPiLUzK1rhSBDWK4R53Wc=; b=j6mLQX2AYuSRR6TF7u5Fk4q+MK8bfDfKopYRXxVwHTsuqeIsW9cp0ZZSBHuZ/1jeEh 48u0tlLNbJcwd5G7rVe+ywMktaxamNnA0yVSn+tAuPrdxl4q5P+MEuNsH9Fd+kRMF7h0 S7Y0KF9C7F0ngg7E01rcZzSP0h5DTXqim6j33dmXSWnAvKGHfVQGsyLoLsWGthhU8Q60 aioDcfBzme6n1bUb8JnHJnbanVkzPGVZbDydOfxD6PTrx79COWhRDIIertS+vJoyMMpc 7AuQH9+8CUudnVZ+zLBKCKieT0MbwBBHnklxovEv+ZF1V8PnkhUcN8bEkbdI2r7sUF1E yXKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=1Ev3mqoNtbMx651f/aw1YebPiLUzK1rhSBDWK4R53Wc=; b=FuY/bjEo7AjnDIUv7WOf0E2S1s7mM11ZdF76EPjHFYvE+RrkbsehHLndjm/pThpIfr WvcLQoAxHMMMR4OK2pCjMJu9wUVRMHYcAQFZWU0j2XJXdegogf1qimlPdFZwY6OPrIaN q5K7Eg2iXhEc+Orm2550IiNEees6//SFyh7hLNxyQNpWB/DlrMjQ1ZzkltbdFN5pFbQm iNUL0lpJpDeU43Br6zeZlpvPrOpgqEfzKBqCZerXlLyLR//gp7/8PJ7RQpMQpPiUKjwl yVW2BT5QO66FyZeK1DZrbMbGrdEiX2uwCvRVSzJ5vKovI/gLjszVTxijEoqPSpBDXrN6 D19Q== X-Gm-Message-State: AOPr4FUZIBpCR2aTCOupWyIrpD3afsjs8/jIh/SBIbTljNGqzWogB9lhqNdFtq3K5wL5yrKjsu206rpRGXg01A== MIME-Version: 1.0 X-Received: by 10.107.159.137 with SMTP id i131mr12122828ioe.29.1461788889959; Wed, 27 Apr 2016 13:28:09 -0700 (PDT) Received: by 10.107.133.65 with HTTP; Wed, 27 Apr 2016 13:28:09 -0700 (PDT) In-Reply-To: References: Date: Wed, 27 Apr 2016 16:28:09 -0400 Message-ID: Subject: Re: Regression? VLAN packet drop after upgrading from r281235 From: Ryan Stone To: =?UTF-8?Q?Z=C3=A9_Claudio_Pastore?= Cc: freebsd-net Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Apr 2016 20:28:10 -0000 >From a quick look at the vlan code, I can identify a few cases that might cause that counter to increment: 1) Error from the underlying ixgbe device. Does "netstat -dI ix0" show that the driver has been dropping packets? 2) Link down events on the underlying NIC. I believe that link flaps will be logged to /var/log/messages and dmesg; do you see anything there that might correspond to the time of the packet drops? 3) If VLAN_HWTAGGING is disabled through ifconfig on the port, then in theory a low memory event could cause the packet to be dropped. Does "netstat -m" show that "requests for mbufs denied" increasing? On Wed, Apr 27, 2016 at 2:41 PM, Z=C3=A9 Claudio Pastore wrote: > Hello, > > On a BGP border router I help manage, we run FreeBSD 10.1-STABLE, > version r281235 and it works fine for several years now. > > We have around 4Gbit/s and 1.8Mpps routed on peak while per port interfac= e > we peak at 300Kpps. > > Our quality metrics are measured with: > > ping -s 1472 -i 0.1 > > As well as iperf bidirecional. > > This metric is similar to what Speedy Test and SIMET tests are done and o= ur > customers reference. > > Systems working w/o problem: > - 10.1-STABLE / r281235 > > Systems tested with drops: > - 10.2-STABLE / r292035M > - 10.3-STABLE / r298705 > - 11.0-CURRENT / r295683 (downloaded snapshot from ftp.freebsd.org) > - 11.0-CURRENT Melifaro Routing Branch / r297731M > > While testing, when errors happen I can see output errs on the vlan port = on > the output from "netstat -w1 -I vlan6" > > input vlan6 output > packets errs idrops bytes packets errs bytes colls > 1 0 0 66 30557 2 33310968 0 > 1 0 0 105 31458 3 33912219 0 > 2 0 0 2954 32001 8 34983986 0 > 1 0 0 1512 33150 6 35942558 0 > 1 0 0 1512 33654 4 37311862 0 > 1 0 0 1512 34825 3 38213793 0 > 3 0 0 1683 35376 4 39488912 0 > 5 0 0 7280 32423 3 35551869 0 > > Problems may happen under high load (~200Kpps) or low load (~30Kpps) on a > vlan port. The observed frame loss never happens on untagged ports, only > vlan related. The observed loss happens with packets sized 900 bytes and > above but noticeably loss rate is higher with packets close to 1400 (1472 > is my reference size). > > Loss rate on all listed systems different from r281235 is 9-19% with > ping(1) and iperf, while it's 0% on r281235. > > First I believed it to be a Intel driver error on systems newer than 10.1= . > My reference card are dual port 82599EB 10-Gigabit SFI/SFP+ Network > Connection (2x2 on x8 PCIe bus, total 4x10G). But yesterday I replaced > Intel by Chelsio T5 and the problem is still exactly the same, so it's no= t > related to card vendor. > > I always test the very same hardware, I have two SSD drives in this route= r, > one for the 10.1 which just runs fine and the other disk to test the > various versions of FreeBSD. > > Only minor loader and sysctl confs are tweaked: > > kern.hz=3D2000 > net.inet.ip.redirect=3D1 # do not send IP redirects > net.inet.ip.accept_sourceroute=3D0 # drop source routed packets sinc= e > they ca > net.inet.ip.sourceroute=3D0 # if source routed packets are > accepted th > net.inet.tcp.drop_synfin=3D1 # SYN/FIN packets get dropped on > initial c > net.inet.udp.blackhole=3D1 # drop udp packets destined for > closed soc > net.inet.tcp.blackhole=3D2 # drop tcp packets destined for > closed por > security.bsd.see_other_uids=3D0 > > Can anyone suggest what might be a fix/tuning for this behavior? Was ther= e > any relevant change on vlan code from particular revisions close to the o= ne > I run on 10.1 and later which would lead to such a big difference? > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >