From owner-freebsd-net@freebsd.org Fri Jun 26 11:55:58 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31FC198D42F for ; Fri, 26 Jun 2015 11:55:58 +0000 (UTC) (envelope-from pushkar.kothavade@msystechnologies.com) Received: from mail-pd0-f170.google.com (mail-pd0-f170.google.com [209.85.192.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F3AE01B0E for ; Fri, 26 Jun 2015 11:55:57 +0000 (UTC) (envelope-from pushkar.kothavade@msystechnologies.com) Received: by pdcu2 with SMTP id u2so73953747pdc.3 for ; Fri, 26 Jun 2015 04:55:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type; bh=gs+nnHBJKcKJ7RdJicURVkt9LweeSV7mK670AYVlu/I=; b=B0jFftmV4NSH152KceG/nHZqdpUYQ1KN9xH8aoswZhD4m44UL+6cJCQJ+NuaPfrMWk 3QasfHGmprR2x4SXQoM1SYkloRZthG1AYacUlk52+437dQDkEDtc8tRsgGHOVufny4Mf +Wo+xgb78oikK615DbL8mKHzrRa/llu1AW3qX+uag0bmpFEXcSXHqilNximFGEzdah66 S7j2BxRatIKXocUC+iR+Ofy4NU2duE9gMbbHmtOQQ/P90M5Q1g/n3MHcf+8ZcyG6MxkK KkHxDS+ktsz9Ukz1R4Z+RqFd++7QKoZb5tijJLQTgiFbo1PZyxZnySkFZjQRR9qRcOVg GL/Q== X-Gm-Message-State: ALoCoQlaNAW2fZMYC2FCwWmUhbfcLr+rHFpRsDAenYf7p5M6wr8zVaJXrIjR3XWlpDNPdbK5GO6V X-Received: by 10.70.49.73 with SMTP id s9mr2536823pdn.149.1435319751444; Fri, 26 Jun 2015 04:55:51 -0700 (PDT) Received: from [172.30.33.149] ([113.193.28.226]) by mx.google.com with ESMTPSA id de4sm33098199pbb.95.2015.06.26.04.55.46 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 26 Jun 2015 04:55:49 -0700 (PDT) Message-ID: <558D3DC0.8020703@msystechnologies.com> Date: Fri, 26 Jun 2015 17:25:44 +0530 From: Pushkar Kothavade User-Agent: Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: freebsd-net@freebsd.org CC: flewis@panasas.com, rtopannavar@panasas.com, sreen@panasas.com, rpokala@panasas.com, lakshmi.n@msystechnologies.com, "sr >> Tallam, Sreen" , rpokala@panasas.com Subject: Re: Lagg-LACP is not working with Intel Fortville NIC on FreeBSD HOL (11.0 Current) References: <557FD197.1020805@msystechnologies.com> <5582659F.6060900@msystechnologies.com> In-Reply-To: <5582659F.6060900@msystechnologies.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2015 11:55:58 -0000 Dear Members, To further narrow down the problem, I am evaluating single ixl link (Intel Fortville NIC) on FreeBSD platform. Single ixl link is _NOT_ a part of Lagg. *Issue* Change in MAC address on a ixl (Intel Fortville NIC) link does not reflect on NIC. Ping stops working after change in MAC address. Ping works, if PROMISC mode is set. *Steps to reproduce the problem* _/*# ifconfig ixl1*/_ ixl1: flags=8843 metric 0 mtu 9000 options=6407bb ether 68:05:ca:35:97:29 nd6 options=29 media: Ethernet autoselect (10Gbase-Twinax ) status: active _# ping XYZ/*/* */*/_Ping Works_/*/* */*/_ _/*# ifconfig ixl1 ether 58:04:89:12:34:56 */_Assign new MAC to the link. _# ping XYZ/*/* */*/_Ping does _NOT_ Work *Bug Analysis* While working on this issue, I found bug in the code. Refer 'ixl_init_locked(struct ixl_pf *pf) function' present in 'dev/ixl/if_ixl.c' file. 1080 /* Get the latest mac address... User might use a LAA */ 1081 bcopy(IF_LLADDR(vsi->ifp), tmpaddr, 1082 I40E_ETH_LENGTH_OF_ADDRESS); * 1083 if (!cmp_etheraddr(hw->mac.addr, tmpaddr) && ** ** 1084 i40e_validate_mac_addr(tmpaddr)) {* 1085 bcopy(tmpaddr, hw->mac.addr, 1086 I40E_ETH_LENGTH_OF_ADDRESS); 1087 ret = i40e_aq_mac_address_write(hw, 1088 I40E_AQC_WRITE_TYPE_LAA_ONLY, 1089 hw->mac.addr, NULL); 1090 if (ret) { 1091 device_printf(dev, "LLA address" 1092 "change failed!!\n"); 1093 return; 1094 } 1095 } Refer lines 1083 & 1084 - '*i40e_validate_mac_addr(tmpaddr)*' function returns 0 on success. Therefore 'i40e_aq_mac_address_write()' function is not getting called. Bug Fix is as Follows - if (!cmp_etheraddr(hw->mac.addr, tmpaddr) && i40e_validate_mac_addr(tmpaddr) *== I40E_SUCCESS*) After this fix, 'i40e_aq_mac_address_write()' function is getting called. *Conclusion* This 'bug fix' could not resolve the Ping issue. Appreciate your views on this matter. Thanks, Pushkar Kothavade On Thursday 18 June 2015 12:00 PM, Pushkar Kothavade wrote: > Dear Members, > > I am part of Panasas. I am evaluating 'Lagg-LACP performance and > robustness' over Intel Fortville NIC (IXL) on Intel Taylor Pass > platform with FreeBSD-HOL [11-CURRENT] installed. Lagg-LACP feature > works fine (with satisfactory performance) when Promiscuous mode is > enabled on the Lagg. > If Promiscuous mode is disabled, in that case even basic Lagg > functionality does not work. > > *Setup Information : * > > - Machine (Intel Taylor Pass) is running FreeBSD-HOL (11 current) > - Machine has 'Intel Fortville NIC (Intel Ethernet Controller XL710)' > installed with special firmware to support four 10Gbps ports with > breakout cable > - Breakout cable is used with one of the 40GB ports which creates four > 10GB ports > - All four 10Gbps ports are connected to Dell Force 10 Switch > - Lagg-LACP has been configured on all four 10GB ports of the Machine > and on Switch ports as well > > *Problem Statement :* > > # When machine sends an Arp-request then interface does not receive > Arp-reply > - Arp request sent by the machine is successful > - Switch is able to get Arp broadcast request sent by the machine > - Switch sends back unicast Arp-reply to machine > - But still driver does not receive Arp-reply > > # Even after adding static Arp entry in machine's Arp table, ping fails > > # Configured same MAC address on all 4 interfaces which are part of > Lagg, still ping fails > Is there any problem with MAC filtering ? > > *Workaround : * > > - After putting Lagg interface in promiscuous mode, network interface > seems to work properly. > > *Platform Information :* > > Machine - Intel Taylor Pass server board > OS - FreeBSD HOL (11.0 Current) > NIC - Intel Fortville NIC (Intel Ethernet Controller XL710) - Find > more information as below > Switch - Dell Force 10 Switch > > > ixl0: mem > 0x383ffe800000-0x383ffeffffff,0x383fff818000-0x383fff81ffff irq 42 at > device 0.0 on pci4 > ixl0: Using MSIX interrupts with 13 vectors > ixl0: f4.33 a1.2 n04.42 e80001935 > ixl0: Ethernet address: 68:05:ca:35:97:28 > ixl0: PCI Express Bus: Speed 8.0GT/s Width x8 > ixl1: mem > 0x383ffe000000-0x383ffe7fffff,0x383fff810000-0x383fff817fff irq 42 at > device 0.1 on pci4 > ixl1: Using MSIX interrupts with 13 vectors > ixl1: f4.33 a1.2 n04.42 e80001935 > ixl1: Ethernet address: 68:05:ca:35:97:29 > ixl1: PCI Express Bus: Speed 8.0GT/s Width x8 > ixl2: mem > 0x383ffd800000-0x383ffdffffff,0x383fff808000-0x383fff80ffff irq 42 at > device 0.2 on pci4 > ixl2: Using MSIX interrupts with 13 vectors > ixl2: f4.33 a1.2 n04.42 e80001935 > ixl2: Ethernet address: 68:05:ca:35:97:2a > ixl2: PCI Express Bus: Speed 8.0GT/s Width x8 > ixl3: mem > 0x383ffd000000-0x383ffd7fffff,0x383fff800000-0x383fff807fff irq 42 at > device 0.3 on pci4 > ixl3: Using MSIX interrupts with 13 vectors > ixl3: f4.33 a1.2 n04.42 e80001935 > ixl3: Ethernet address: 68:05:ca:35:97:2b > ixl3: PCI Express Bus: Speed 8.0GT/s Width x8 > acpi0: Power Button (fixed) > ixl0: mem > 0x383ffe800000-0x383ffeffffff,0x383fff818000-0x383fff81ffff irq 42 at > device 0.0 on pci4 > ixl0: Using MSIX interrupts with 13 vectors > ixl0: f4.33 a1.2 n04.42 e80001935 > ixl0: Ethernet address: 68:05:ca:35:97:28 > ixl0: PCI Express Bus: Speed 8.0GT/s Width x8 > ixl1: mem > 0x383ffe000000-0x383ffe7fffff,0x383fff810000-0x383fff817fff irq 42 at > device 0.1 on pci4 > ixl1: Using MSIX interrupts with 13 vectors > ixl1: f4.33 a1.2 n04.42 e80001935 > ixl1: Ethernet address: 68:05:ca:35:97:29 > ixl1: PCI Express Bus: Speed 8.0GT/s Width x8 > ixl2: mem > 0x383ffd800000-0x383ffdffffff,0x383fff808000-0x383fff80ffff irq 42 at > device 0.2 on pci4 > ixl2: Using MSIX interrupts with 13 vectors > ixl2: f4.33 a1.2 n04.42 e80001935 > ixl2: Ethernet address: 68:05:ca:35:97:2a > ixl2: PCI Express Bus: Speed 8.0GT/s Width x8 > ixl3: mem > 0x383ffd000000-0x383ffd7fffff,0x383fff800000-0x383fff807fff irq 42 at > device 0.3 on pci4 > ixl3: Using MSIX interrupts with 13 vectors > ixl3: f4.33 a1.2 n04.42 e80001935 > ixl3: Ethernet address: 68:05:ca:35:97:2b > ixl3: PCI Express Bus: Speed 8.0GT/s Width x8 > > > Thanks, > Pushkar Kothavade > > >