From owner-freebsd-net@freebsd.org Fri Dec 7 12:37:33 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF62B130CA2F for ; Fri, 7 Dec 2018 12:37:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7731D89A84 for ; Fri, 7 Dec 2018 12:37:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 3B2A8130CA2D; Fri, 7 Dec 2018 12:37:32 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18845130CA2C for ; Fri, 7 Dec 2018 12:37:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB00A89A7D for ; Fri, 7 Dec 2018 12:37:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id D63885FE9 for ; Fri, 7 Dec 2018 12:37:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id wB7CbUId086230 for ; Fri, 7 Dec 2018 12:37:30 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id wB7CbUje086224 for net@FreeBSD.org; Fri, 7 Dec 2018 12:37:30 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 233759] igb (I210) + net.inet.ipsec.async_crypto=1 + aesni kill receiving queues and traffic Date: Fri, 07 Dec 2018 12:37:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: lev@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 7731D89A84 X-Spamd-Result: default: False [-2.94 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-0.98)[-0.983,0]; NEURAL_HAM_SHORT(-0.96)[-0.961,0]; NEURAL_HAM_LONG(-0.99)[-0.994,0]; ASN(0.00)[asn:10310, ipnet:2001:1900:2254::/48, country:US] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Dec 2018 12:37:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233759 --- Comment #6 from Lev A. Serebryakov --- (In reply to Sean Bruno from comment #5) I have three systems (they are separate physical systems, not VMs). (1) Manager. (2) Device Under Test ("DUT") (3) Mirror. Each system has 3 interfaces. One interface of each system is management on= e to connect from outside work, and these interfaces is not in scope of this description. Manager system has two interfaces in question: "outbound" and "inbound". - outbound has IP 10.1.0.2/24 and it is connected with "inbound" interfac= e of DUT (via dedicated switch). - inbound has IP 10.10.10.2/24 and it is connected with "outbound" inter= face of "Mirror". Manager system doesn't have any special routing record. DUT system has two interfaces: "outbound" (igb1 in this ticket) and "inboun= d" (igb0 in this ticket). - "outbound" (igb1) has IP 10.2.0.1/24 and it is connected with "inbound" interface of "Mirror". - "inbound" (igb0) has IP 10.1.0.1/24 and it is connected with "outbound" interface of "Manager" (via dedicated switch). DUT has routing enabled and has "route -net 10.10.10.0/24 10.2.0.1". DUT has such IPSec settings: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D add 10.2.0.1 10.2.0.2 esp 0x10001 -m tunnel -E aes-gcm-16 "wxyz0123456789abcdef"; add 10.2.0.1 10.2.0.` esp 0x10002 -m tunnel -E aes-gcm-16 "wxyz0123456789abcdef"; spdadd 10.1.0.0/24 10.10.10.0/24 udp -P out ipsec esp/tunnel/10.2.0.1-10.2.0.2/require; spdadd 10.10.10.0/24 10.1.0.0/24 udp -P in ipsec esp/tunnel/10.2.0.2-10.2.0.1/require; =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Mirror system has two interfaces in question: "outbound" and "inbound". - outbound has IP 10.10.10.1/24 and it is connected with "inbound" interf= ace of Manager. - inbound has IP 10.2.0.2/24 and it is connected with "outbound" interfa= ce of DUT. Mirror has routing enabled and has "route -net 10.1.0.0/24 10.2.0.2". Mirror has static ARP for 10.10.10.2-10.10.10.254 points to "Manager" "Inbo= und" interface. Mirror has such IPSec settings: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D add 10.2.0.1 10.2.0.2 esp 0x10001 -m tunnel -E aes-gcm-16 "wxyz0123456789abcdef"; add 10.2.0.1 10.2.0.` esp 0x10002 -m tunnel -E aes-gcm-16 "wxyz0123456789abcdef"; spdadd 10.10.10.0/24 10.1.0.0/24 udp -P out ipsec esp/tunnel/10.2.0.2-10.2.0.1/require; spdadd 10.1.0.0/24 10.10.10.0/24 udp -P in ipsec esp/tunnel/10.2.0.1-10.2.0.2/require; =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Ok, it is config. Really, it is loop "Manager -> DUT -> Mirror -> Manager" where connection between DUT and Mirror has additional IPsec config. Manager and Mirror are much more powerful than DUT and could pass full-wire-speed traffic without any problems with and without encryption. Now to test. Manager generates (with netmap's pkt-gen) UDP traffic with such characteristics: Transmit interface: "outbound" Dst MAC: DUT "inbound" Src IPs: 10.1.0.2:2000-10.1.0.5:2004 Dst IPs: 10.10.10.2:2000-10.10.10.128:2006 Manager receives all traffic (with netmap's pkt-gen) at "inbound" interface= and measure bandwidth. Now, if DUT has default setting for async IPsec (turned off) it could pass 690Mbit/s or 199Kp/s. Any traffic lower than that passes without any losses. For example, if I generate traffic and speed 64P/s (without any prefixes!) I see each and any packet returned to Manager from Mirror via DUT. No problems here. If I turn on async IPsec ("sysctl net.inet.ipsec.async_crypto=3D1" on DUT),= no matter which traffic is generated (I've tested with 64 packets per second, = not kilo-packets, simple packets!) receive queues of DUT inbound interface (igb= 0) stop to work one by one. --=20 You are receiving this mail because: You are the assignee for the bug.=