From owner-freebsd-net@freebsd.org Thu Mar 23 18:20:42 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4564ED1A9DF for ; Thu, 23 Mar 2017 18:20:42 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-ot0-x22a.google.com (mail-ot0-x22a.google.com [IPv6:2607:f8b0:4003:c0f::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03CDA1FF1 for ; Thu, 23 Mar 2017 18:20:42 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-ot0-x22a.google.com with SMTP id a12so123583122ota.0 for ; Thu, 23 Mar 2017 11:20:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9qcUhwERearcaIDvmy9HEbJZ8OgdYBwZTAk8MsrOlsE=; b=AFBivKnkJtrrtJIiG8CiIM1uhgFQMk3gOULwX4PQSF4WbRNeqnKV0EvP5T82RnqffQ yH2T+sT227/KYsuKof8pzN+V+brDwN6MovHxVFfzDmQZ5zA7mRK2zP6GVvGS5fupoxu9 Hqu1fJh0bfDRakwBs4CIf/t5k9rB1+9S4rWrBrYXEuYf+RjKuQsCPGpPQBCo+ZhQX19C oJJwvCYrLYILlE8m8MLUiaAREaq9XaGF0djtTL3DjonzWu84TOYjXdzfLVQor4c+ZPOi 1UXEOuXXTIuJy43GXQXU+YLFRohUBTTwsDWF25yeIWBTbhrItdCGGPZ2EvNJntbGUo7h l1Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9qcUhwERearcaIDvmy9HEbJZ8OgdYBwZTAk8MsrOlsE=; b=bRgcElAwd2SfXbUw4FjXv7VyLec1ungGUI/IEcAMumpJAd+GkmCo8lcLmcnm6lEQs4 Gz7WG+dF6pVbzgNiU7gBgynRSfwS7AvmMTFp4UTc5sm0ofUMD/xsJEh7Si4PlwKsP79i Min2F5hy3POKvKA3HybwMrossysd6aTibpkd7qov7bPCvht8mz5l8NyHNJFmWFFZJU57 k8oi3d+jo6qo+N55awcgX8Kcipbkc+zd8g+uwmeJod6M5f9W9cq8CQMSXTaEn4HFICiv AX8E1C7AVZISgHtwEp/e90twzeEV5Xi/RrlfcqyYteV+lr/hHnDs4rJFPL23rLjLJpcJ Ix8A== X-Gm-Message-State: AFeK/H0C/cJuychvu83KqhGzWiWnznuoW0je1EiDwEGfVfxOA/rZMjKf9gLUdKSpxciW55i4yUdqK9sNBoBdPA== X-Received: by 10.157.52.116 with SMTP id v107mr287768otb.61.1490293241290; Thu, 23 Mar 2017 11:20:41 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.50.45 with HTTP; Thu, 23 Mar 2017 11:20:40 -0700 (PDT) In-Reply-To: <58D3C6F4.6010500@stream-technologies.com> References: <58D3C6F4.6010500@stream-technologies.com> From: Vincenzo Maffione Date: Thu, 23 Mar 2017 19:20:40 +0100 Message-ID: Subject: Re: cxgbe netmap promiscuous mode? To: Joe Jones Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2017 18:20:42 -0000 Hi, You could try to use netmap in emulated mode (sysctl dev.netmap.admode=2). If this works, at least you know that the problem is in the cxgbe netmap support and not in the netmap core itself. Cheers, Vincenzo 2017-03-23 14:00 GMT+01:00 Joe Jones : > Hello, > > We have a T520-SO and have made a new install of 11.0, to begin with the > box would panic every time we tried to switch the card into netmap mode. So > we recompiled the kernel with netmap removed, then compiled the netmap > kernel module from github, as this in our experience generally leads to a > more stable netmap. > > we have > > uname -a > FreeBSD goose2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0: Wed Mar 22 > 16:52:35 UTC 2017 joe@goose2:/usr/obj/usr/src/sys/GENERIC amd64 > > and the following in /boot/loader.conf > > t4fw_cfg_load="YES" > t5fw_cfg_load="YES" > if_cxgbe_load="YES" > hw.cxgbe.fl_pktshift=0 > hw.cxgbe.toecaps_allowed=0 > hw.cxgbe.nnmtxq10g=8 > hw.cxgbe.nnmrxq10g=8 > hw.cxgbe.num_vis=2 > > Before I run our application I run > > ifconfig cxl1 promisc -vlanhwtag up > > Now our application can now start without panicking the kernel. Here is > where it gets interesting, our application is able to announce it's self > via ARP, I can see the ethernet switch learning which port it's on, and > other hosts adding it to their ARP tables. When I try an ICMP ping it goes > missing. After watching the TX packet graph for the connected port on the > switch while starting and stopping a flood ping to the application, I'm > sure the packets are getting sent to the card, however I don't see them in > the netmap ring. If I kill our application, then use ifconfig to create and > configure a vlan port I can confirm that the card is working and has > connectivity. > > Here's what I think is happening. ARP requests are received because they > are sent to the broadcast address. Our application then announces it's > self. However traffic destined for the application is send to a MAC address > which is neither the broadcast or the MAC programed into the hardware and > is dropped. My understanding of promiscuous it that it informs the card > that we want these dropped packets. It looks to me like, when the card is > in netmap mode the promisc flag is being ignored. > > I have also tried using freebsd-update to update to p8. As with the p0 > kernel we get a panic when we switch the card into netmap mode. > > We did previously have these cards working in netmap mode. We were using a > pre 11 snapshot of the svn head though . > > Many Thanks > > Joe Jones > Stream Technologies > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Vincenzo Maffione