From owner-freebsd-net@freebsd.org Wed Mar 25 22:31:51 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 38B4A26566B for ; Wed, 25 Mar 2020 22:31:51 +0000 (UTC) (envelope-from vmaffione@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48njXQ18qSz3NFH for ; Wed, 25 Mar 2020 22:31:50 +0000 (UTC) (envelope-from vmaffione@freebsd.org) Received: from mail-qv1-f49.google.com (mail-qv1-f49.google.com [209.85.219.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: vmaffione) by smtp.freebsd.org (Postfix) with ESMTPSA id BE33918C1A for ; Wed, 25 Mar 2020 22:31:43 +0000 (UTC) (envelope-from vmaffione@freebsd.org) Received: by mail-qv1-f49.google.com with SMTP id o7so1983141qvq.8 for ; Wed, 25 Mar 2020 15:31:43 -0700 (PDT) X-Gm-Message-State: ANhLgQ0ywjsJ4ddU1kXods8M0250MfBzbMlTy1TAIbeC9QpdcGLkrluW KpYijwNYTYxpdTCKEVAztW5HQCjXXtGENZEWJhs= X-Google-Smtp-Source: ADFU+vtnDmBTweNDTDnV/NRXDCWonXs8CEZTB7jP0SljpAqBfleaWcIB8Ybig2ZVV/huakj0tzwDoB4+QKY2HyufZPU= X-Received: by 2002:ad4:46e6:: with SMTP id h6mr5313194qvw.208.1585175502945; Wed, 25 Mar 2020 15:31:42 -0700 (PDT) MIME-Version: 1.0 References: <20200324123721.GA26248@staff.retn.net> <20200324140839.GB26248@staff.retn.net> In-Reply-To: <20200324140839.GB26248@staff.retn.net> From: Vincenzo Maffione Date: Wed, 25 Mar 2020 23:31:30 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: netmap/ixl and crc addition.. To: Alexandre Snarskii Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2020 22:31:51 -0000 Hi Alexandre, Thanks. Your patch looks good to me. I assume you have tested it? I will commit that to stable/11. The issue you report on stable/12 is more worrisome. The 'no space in TX ring' condition (head==cur==tail) is ok per-se: on a subsequent poll() wakeup (e.g. TX interrupt) or explicit ioctl(NIOCTXSYNC) you should see tail moving forward, therefore freeing some space to be used in the ring. However, the ring_reinit means that something is going wrong: either your application is using the TX ring incorrectly, or there is a bug in the netmap iflib code. Since FreeBSD 12, netmap support is provided by iflib, while before netmap support was provided directly by the ixl driver. In any case, it would probably help if you could provide some more detailed info (how to reproduce the problem). Cheers, Vincenzo Il giorno mar 24 mar 2020 alle ore 15:12 Alexandre Snarskii < snar@snar.spb.ru> ha scritto: > On Tue, Mar 24, 2020 at 03:37:36PM +0300, Alexandre Snarskii wrote: > > > > Hi! > > > > Long story short: looks like intel x722 does not by default add CRC to > > outbound frames, so with FreeBSD 11-stable netmap-generated traffic is > > dropped on the next port.. Fix is simple, attached. > > ... add missing attach :( > > > > > The same behaviour of 'unconditionally ask card to compute crc' can > > be found in both if_ixl: > > > https://svnweb.freebsd.org/base/stable/11/sys/dev/ixl/ixl_txrx.c?view=markup#l408 > > and in DPDK i40e driver: > > > https://github.com/DPDK/dpdk/blob/master/drivers/net/i40e/i40e_rxtx.c#L1105 > > so, I guess, it's safe. > > > > PS: of course, first idea was to upgrade to FreeBSD 12-stable, but while > > this upgrade solved the crc problem, this version shows 'stalled tx > queue' > > problem: after CTXSYNC tail == head == cur, 'no space in ring' > condition. > > Attempts to ignore this condition led to continuous ring resets in > txcsync: > > > > Mar 17 20:21:08 host kernel: 668.224836 [1679] nm_txsync_prologue > ixl1 TX3: fail 'head < kring->rhead || head > kring->rtail' h 136 c 136 t > 135 rh 135 rc 135 rt 135 hc 135 ht 135 > > Mar 17 20:21:08 host kernel: 668.238300 [1787] netmap_ring_reinit > called for ixl1 TX3 > > > > PPS: hardware details: Dell VEP4600, based on Xeon D-2100 with > > two onboard X722 ports (actually, four, but two of them are not > > wired). > > > > CPU: Intel(R) Xeon(R) D-2187NT CPU @ 2.00GHz (2000.06-MHz K8-class CPU) > > Origin="GenuineIntel" Id=0x50654 Family=0x6 Model=0x55 Stepping=4 > > > > ixl0: 1.11.9-k> mem 0xfa000000-0xfaffffff,0xfb018000-0xfb01ffff irq 11 at device > 0.0 numa-domain 0 on pci12 > > ixl0: using 1024 tx descriptors and 1024 rx descriptors > > ixl0: fw 3.1.57069 api 1.5 nvm 3.33 etid 80001007 oem 1.263.0 > > ixl0: PF-ID[0]: VFs 32, MSIX 129, VF MSIX 5, QPs 384, I2C > > ixl0: Using MSIX interrupts with 9 vectors > > ixl0: Allocating 8 queues for PF LAN VSI; 8 queues active > > ixl0: Ethernet address: 3c:2c:30:30:59:85 > > ixl0: SR-IOV ready > > ixl0: netmap queues/slots: TX 8/1024, RX 8/1024 > > > > ixl0@pci0:184:0:0: class=0x020000 card=0x00008086 chip=0x37d38086 > rev=0x04 hdr=0x00 > > vendor = 'Intel Corporation' > > device = 'Ethernet Connection X722 for 10GbE SFP+' > > class = network > > subclass = ethernet > > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >