From owner-freebsd-net@freebsd.org Sat Nov 3 13:11:01 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 194BA10FD5DC for ; Sat, 3 Nov 2018 13:11:01 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81]) by mx1.freebsd.org (Postfix) with ESMTP id B2E4F81860 for ; Sat, 3 Nov 2018 13:11:00 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:9579:fe59:c90b:488c]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 33C1E6C0 for ; Sat, 3 Nov 2018 16:10:54 +0300 (MSK) Date: Sat, 3 Nov 2018 16:10:54 +0300 From: Lev Serebryakov Organization: FreeBSD Message-ID: <717725514.20181103161054@serebryakov.spb.ru> To: freebsd-net@freebsd.org Subject: Strange behavior of forwarding between igb and ix on 11-STABLE: a lot of droppet packets no matter which pps is used MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Nov 2018 13:11:01 -0000 Hello Freebsd-net, I'm setting up simple experiment: Host A with em0 and ix0 (12.BETA2) Host B with igb1 and ix0 (11.2-STABLE) A:em0 is connected to B:igb1 A:ix0 is connected to B:ix0 Host B has static ARP for 192.168.10.2-192.168.10.9 set to A:ix0's MAC. Host B has forwarding enabled. I run packet generator on A:em0 to send packets for A:ix0 IP's (192.168.10.2-192.168.10.9) via host B with rate 64 pps (yes, pps): A# pkt-gen -f tx -i em0 -d 192.168.10.2:2000-192.168.10.9 -D -R 64 I also run packet receiver on A:ix0 to measure these 64 pps. A# pkt-gen -f rx -i ix0 And receiver shows about 1/3 of pps! ~22pps! At host B "netstat -sp ip" shows: 1790 packets forwarded (1790 packets fast forwarded) 44 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 1295 packets sent from this host 0 packets sent with fabricated ip header 1784 output packets dropped due to no bufs, etc. 1790 forwarded packets are EXACTLY what receiver has been seen! And "1784 output packets dropped due to no bufs, etc." is counting when packets are forwarded, but not when host B sends traffic by itself If I increase PPS picture will be the same: only between 1/4 and 1/3 of all packets are forwarded. What do I do wrong? This ix0-ix0 link is fully capable of full 10G/s transfers when it is tested end-to-end, without any problems. If I test UDP transfer from B to A these "output packets dropped" are NOT counting even at 100Kpps! Interface is perfectly Ok! -- Best regards, Lev mailto:lev@FreeBSD.org