From owner-freebsd-net@freebsd.org Thu Mar 23 18:38:50 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA72DD1A0B7 for ; Thu, 23 Mar 2017 18:38:50 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pg0-x22f.google.com (mail-pg0-x22f.google.com [IPv6:2607:f8b0:400e:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E1B61C40 for ; Thu, 23 Mar 2017 18:38:50 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pg0-x22f.google.com with SMTP id w20so25419779pgc.0 for ; Thu, 23 Mar 2017 11:38:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=MZEZ7eraxmGS0M14CDjRXI4iKQL8YwYQmecwiA4XfKg=; b=OMUfvqrKvjN+ROhybRnCNuOT37PtbhX1Qe2/SpCMywonHbT3PtwsCJRLSxiYpfEXTs Y9wmX8qu0/G8oEhdokTGX5Z8g/DClxHfA28mG+7Z8W9fGNY/b3kMLlY/vbloQJro3uJv dhgKRZSrY2cBjjybDQC0iRYoXOB5fKSdEmAw0g2yJBgHA7eLti2wDm3lbsMkjKu5j+i5 JWe9GB/nXzjvEEDPvPo7209dM0lk9ijeNkCytJzYoUtG+dTTCHcXmjBOem6+rZeguUvd O2cuY8bItgvIUu+L3s3Me/IcE0edxRRp7Y0jTxIjtj6XNHnOlhsl5XgJMYzN9vTafmLX L4aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=MZEZ7eraxmGS0M14CDjRXI4iKQL8YwYQmecwiA4XfKg=; b=Dn3RPWimABXguBZgBmfU1+0NtFF7bOdPKXUEFreZmWjhKTpEidpsPVHEquR0614WrQ /DrIueNQBhCGrwhBAE/cl4JCyTcxxjL7M4ivXpcaj8TM4gubLA5j6AkuxDM/Na+RrKOX WxeTsIndkDYl5s09CH+dlLMNLzTX8WKaKo0leXSo4zQaqsNI+qzk9t9So1xFpWQUVdEa 7djzwrs4SwGP611qC9B7GQtarKVtWfeJ8FtJ34ZsQmT1vJjFIyih+KFLUP7Sk67mI7W5 nq1/mH7yi5dm3GHrpIj8+pq+eTdY+D525N+/5qOQwa7LAcJJiRVzSm+xbUrABKEFooSx U81w== X-Gm-Message-State: AFeK/H113JpLLSsIlb5AuTo649PR6eV/6AaBAhMUFyIi/IjXQmvFlDD7HYSpX8s9s33W/8b/F6J6Khi1I/omUA== X-Received: by 10.98.54.196 with SMTP id d187mr4712935pfa.33.1490294330053; Thu, 23 Mar 2017 11:38:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.168.13 with HTTP; Thu, 23 Mar 2017 11:38:49 -0700 (PDT) In-Reply-To: <58D3C6F4.6010500@stream-technologies.com> References: <58D3C6F4.6010500@stream-technologies.com> From: Navdeep Parhar Date: Thu, 23 Mar 2017 11:38:49 -0700 Message-ID: Subject: Re: cxgbe netmap promiscuous mode? To: Joe Jones Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2017 18:38:50 -0000 Your netmap application should be using the 'vcxl' interface, not the cxl interface. Even though they share a physical port they have different MAC addresses and are totally autonomous. The peer should use the vcxl interface's MAC if it wants to reach the netmap application. Do you have the panic message and stack? I know of a couple of panics that have been fixed in -STABLE -- one was one related to emulated mode and the second one was an illegal lock acquisition. Regards, Navdeep On Thu, Mar 23, 2017 at 6:00 AM, Joe Jones wrote: > Hello, > > We have a T520-SO and have made a new install of 11.0, to begin with the box > would panic every time we tried to switch the card into netmap mode. So we > recompiled the kernel with netmap removed, then compiled the netmap kernel > module from github, as this in our experience generally leads to a more > stable netmap. > > we have > > uname -a > FreeBSD goose2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0: Wed Mar 22 > 16:52:35 UTC 2017 joe@goose2:/usr/obj/usr/src/sys/GENERIC amd64 > > and the following in /boot/loader.conf > > t4fw_cfg_load="YES" > t5fw_cfg_load="YES" > if_cxgbe_load="YES" > hw.cxgbe.fl_pktshift=0 > hw.cxgbe.toecaps_allowed=0 > hw.cxgbe.nnmtxq10g=8 > hw.cxgbe.nnmrxq10g=8 > hw.cxgbe.num_vis=2 > > Before I run our application I run > > ifconfig cxl1 promisc -vlanhwtag up > > Now our application can now start without panicking the kernel. Here is > where it gets interesting, our application is able to announce it's self via > ARP, I can see the ethernet switch learning which port it's on, and other > hosts adding it to their ARP tables. When I try an ICMP ping it goes > missing. After watching the TX packet graph for the connected port on the > switch while starting and stopping a flood ping to the application, I'm sure > the packets are getting sent to the card, however I don't see them in the > netmap ring. If I kill our application, then use ifconfig to create and > configure a vlan port I can confirm that the card is working and has > connectivity. > > Here's what I think is happening. ARP requests are received because they are > sent to the broadcast address. Our application then announces it's self. > However traffic destined for the application is send to a MAC address which > is neither the broadcast or the MAC programed into the hardware and is > dropped. My understanding of promiscuous it that it informs the card that we > want these dropped packets. It looks to me like, when the card is in netmap > mode the promisc flag is being ignored. > > I have also tried using freebsd-update to update to p8. As with the p0 > kernel we get a panic when we switch the card into netmap mode. > > We did previously have these cards working in netmap mode. We were using a > pre 11 snapshot of the svn head though . > > Many Thanks > > Joe Jones > Stream Technologies > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"