From owner-freebsd-net@freebsd.org Tue Apr 19 04:06:29 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44C6DB14885 for ; Tue, 19 Apr 2016 04:06:29 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04D291822 for ; Tue, 19 Apr 2016 04:06:29 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: by mail-io0-x22b.google.com with SMTP id o126so6792252iod.0 for ; Mon, 18 Apr 2016 21:06:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to; bh=KNY44VJH4HWYfbCL23vUXPs9iwyrZgAFWXj9SGYBIX8=; b=Jtwa1UyLkykV5lbGCyDfcWQ9gJ46S8F8ddotK9OOxNuGigHZScKSP0Qy7uRX4ruBQF Cdxz0pnqtAbhlcSvge3rvJ3JnNn15QRwyv5ehNazFQk1fD3F+kfrAXlSJfp58dnBPyki dDajQydz7rivc6mfBcoXonO4RXp+BkYII+ui06M4PTXKrptSllvR2ZRTbmtKqFQBSkOC HE7ZjvvfjLRePWvj3uXBhEp1dIs/dSwfl7+2pQRyKWSWYREHyNKgjnzDN5p2H16S5H9c 6e649mryqjbCovxDfYN5MZD8bg9rUFYDYtyVPOCLvnnKargtFfVbFYjUqrMbUgDWKJv/ Es4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to; bh=KNY44VJH4HWYfbCL23vUXPs9iwyrZgAFWXj9SGYBIX8=; b=Z3i7XAKr34DtuDkirgT2ItWy8MvJIle2KXSwJgmuY0JmLN+ujoYBxZP1rhfOYQesFs Mju6+Ht7SAbMFXB7W4IKHKFo1AApr/ro5VzEuMCyIgBvOJgXEgGZCxy1YXApPV4Gud6+ LktXLXkur05LmjNYsXzQvT79FDeYUCkdC4ySnpjQ2IT5B+ZpDm8lO2HkYr/LlVFaeAr5 y44WccSA6y6AMhe9GouxoRVyXhT4h7iGLZ+4gWRZC+JDSqmiNASBApCvUxrMo6VI5Sz2 97okpFbe1px+iQMthJfm3kUIDG3rCAKP9rbSBe4gbChoEVwXMI36c4+x0kZuCVp30Ry8 UtZA== X-Gm-Message-State: AOPr4FVyMv5UgG3/jnOBnt3xAQmTzNodki4qOJsvA/1wBX+3sGwqhq7gVdGdr/p7YcWdGU+JMDhZkTn777N1Qw== MIME-Version: 1.0 X-Received: by 10.107.46.158 with SMTP id u30mr1337430iou.162.1461038788335; Mon, 18 Apr 2016 21:06:28 -0700 (PDT) Received: by 10.107.156.132 with HTTP; Mon, 18 Apr 2016 21:06:28 -0700 (PDT) Date: Mon, 18 Apr 2016 23:06:28 -0500 Message-ID: Subject: Issues with ixl(4) From: Dustin Marquess To: freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Apr 2016 04:06:29 -0000 I'm having some strange issues with ixl(4) and a X710-DA4 card in a new-ish Intel-based server. I'm pretty much replicating an existing setup from an older AMD machine that used 2 x X520-DA2 cards and ixgbe(4). This is all on -CURRENT. It's meant to be a bhyve server, so the 4x10GE ports are put into a LACP-based lagg(4), then vlan(4) interfaces are bound to the lagg, and then if_bridge(4) interfaces are created to bind the vlan and tap interfaces together. The X710-DA4 is running the latest NVM from Intel (5.02): dev.ixl.3.fw_version: nvm 5.02 etid 80002284 oem 0.0.0 dev.ixl.2.fw_version: nvm 5.02 etid 80002284 oem 0.0.0 dev.ixl.1.fw_version: nvm 5.02 etid 80002284 oem 0.0.0 dev.ixl.0.fw_version: nvm 5.02 etid 80002284 oem 0.0.0 I've tried both the ixl driver that comes with -CURRENT (1.4.3?) and the 1.4.27 driver from Intel and am having the same problem. The problem is this exactly (sorry it's taken me so long to get to it!): Using just one interface, one interface + VLANs, the lagg without VLANs, etc, everything works perfectly fine. As soon as I combine lagg+vlan+bridge, all hell breaks loose. One machine can ping one alias on the server but not the other while other machines can. The server itself can't ping the DNS server nor the default route, but can ping things through the default route, etc. The behavior is very unpredictable. ssh can take a few times to get in, and then once it, "svn update" will work for a few seconds and then bomb out, etc. He is the working config from the X520-DA2 system: ifconfig_ix0="-lro -tso -txcsum up" ifconfig_ix1="-lro -tso -txcsum up" ifconfig_ix2="-lro -tso -txcsum up" ifconfig_ix3="-lro -tso -txcsum up" cloned_interfaces="lagg0 tap0 tap1 bridge0 bridge1 vlan1 vlan2" ifconfig_lagg0="laggproto lacp laggport ix0 laggport ix1 laggport ix2 laggport ix3" ifconfig_vlan1="vlan 1 vlandev lagg0" ifconfig_vlan2="vlan 2 vlandev lagg0" ifconfig_bridge0="inet 192.168.1.100/24 addm vlan1 addm tap0" ifconfig_bridge1="addm vlan2 addm tap1" defaultrouter="192.168.1.1" Here is the "broken" config from the X710-DA4 system: ifconfig_ixl0="-rxcsum -txcsum -lro -tso -vlanmtu -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up" ifconfig_ixl1="-rxcsum -txcsum -lro -tso -vlanmtu -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up" ifconfig_ixl2="-rxcsum -txcsum -lro -tso -vlanmtu -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up" ifconfig_ixl3="-rxcsum -txcsum -lro -tso -vlanmtu -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up" cloned_interfaces="lagg0 tap0 tap1 bridge0 bridge1 vlan1 vlan2" ifconfig_lagg0="laggproto lacp laggport ixl0 laggport ixl1 laggport ixl2 laggport ixl3" ifconfig_vlan1="vlan 1 vlandev lagg0" ifconfig_vlan2="vlan 2 vlandev lagg0" ifconfig_bridge0="inet 192.168.1.101/24 addm vlan1 addm tap0" ifconfig_bridge1="addm vlan2 addm tap1" defaultrouter="192.168.1.1" I've changed the various flags in the ifconfig_ixl# lines without any obvious differences. Both machines are connected to the same HPe 5820X switch with the same exact config, so I don't believe it's a switch issue. Any ideas? Has anybody seen something like this before? Thanks! -Dustin