From owner-freebsd-net@freebsd.org Fri Mar 24 14:17:55 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C8CAD1B6FD for ; Fri, 24 Mar 2017 14:17:55 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-oi0-x22e.google.com (mail-oi0-x22e.google.com [IPv6:2607:f8b0:4003:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E07B51A90 for ; Fri, 24 Mar 2017 14:17:54 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-oi0-x22e.google.com with SMTP id r203so1907474oib.3 for ; Fri, 24 Mar 2017 07:17:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=NVMK6deVRcEPfacgnG0unobouDn+EWW1ZgnUS5xmxFE=; b=swQx+4+4D997hUiLbV7doAIKge/BQBakRyjWLxfrz4jy1UNz1dX5QguAjZw5HHKpqC TmD06Vn+copEMPI7czA77qQXHMAfaoNUEnw9azwdT48RcH+W5j9Ti5XKNw678Pl3uYtO APrVSa6ZQQxEVx0GaUdXzq7AXYO5A+iC/mitzvew7IucrWcMOExOb9u/sgWucRcTDRpR xKaMhfZ8hVSMHDPk3ZsBvb/kI0WZizMWNzIdE7ww90Syl8megIzREnA7RE5xnmetvVxw yoI/0H0w6kZv59kBWlmTpAn9eVzPH3h/hV1uhN+sRinSAWnXDDqbPfU0tiiC3jc1Vqry I3gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=NVMK6deVRcEPfacgnG0unobouDn+EWW1ZgnUS5xmxFE=; b=P/LNwZ8mMvE5mA7YA/AWgzIpL4GLROlGRj3mdkGkzrhJDiz3RfkfyfHzqCtgEdP00g 70Cj6SUErIUwmr1ZLay0IODfgI/bdf5DQXseioqJm0L/C7k8CRi6LsNcQjvqbHbaDoLH fGeu8EOiaKKjrlwYicPm4PONgge1BPVCu5FwGz9VsaM6oD7Mdy7myek/3uoOeMZQYadl DTXZo/8LN6xb0nPKE/aKJydOKdM9ofh3bogTqJj3i05PlN4FMaXj4FTBewg0Fu+PHzJW qvuR2iQo7sFpsAS3T76EUPlhrygXOT6O1qf+urj5KmDeAOEjzEATd9hKLJ+qIljHuj+P TGQw== X-Gm-Message-State: AFeK/H1V3DBlVffc4IDgO5DcyVBK04tAB8ZGeipEcz4DK2HYU8fHJnJSIDequEyfNPXNLWKaOGeUQEgSMNnaZw== X-Received: by 10.202.105.147 with SMTP id e141mr4148933oic.103.1490365074017; Fri, 24 Mar 2017 07:17:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.50.45 with HTTP; Fri, 24 Mar 2017 07:17:53 -0700 (PDT) In-Reply-To: <58D5242F.9040706@stream-technologies.com> References: <58D3C6F4.6010500@stream-technologies.com> <58D4F2C5.9020204@stream-technologies.com> <58D5242F.9040706@stream-technologies.com> From: Vincenzo Maffione Date: Fri, 24 Mar 2017 15:17:53 +0100 Message-ID: Subject: Re: cxgbe netmap promiscuous mode? To: Joe Jones Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Mar 2017 14:17:55 -0000 Hi Joe, From your description it's pretty clear that you hit the FreeBSD11 bug that is now fixed in 11.0-stable (in addition to the upstream netmap version on github). The fix was committed in r312783. Cheers, Vincenzo 2017-03-24 14:50 GMT+01:00 Joe Jones : > Hi Vincenzo, > > we can get rid if the panic if we compile the kernel without netmap, then > load an up to date netmap as a module. We can live with that for now. > > Some time last year before Free BSD 11 was branched from head, we compiled > a checkout and it worked without problem on the same hardware setup. I may > try and figure out what the regression is, I'm not familiar with the > FreeBSD release process either though. > > Thanks, > Joe Jones > > On 24/03/17 10:29, Vincenzo Maffione wrote: > >> Hi Joe, >> There was a fix for a panic in emulated mode that was applied stable/11 >> branch, so I guess it also ended up into FreeBSD-11.0-STABLE. >> I don't know whether the same fix ended up into in 11.0-RELEASE-p8 (I'm >> not familiar with FreeBSD releasing process, sorry!). >> >> Or maybe this panic happens with netmap upstream? >> If this is a new bug, it would be nice to have the kernel with the debug >> symbols enabled, so that we can get more detailed information from the >> stack trace. >> >> Cheers, >> Vincenzo >> >> 2017-03-24 11:19 GMT+01:00 Joe Jones > >: >> >> >> Hi Vincenzo, >> >> I just tried with that sysctl set to 2, I get a similar looking >> panic to before >> >> Fatal trap 12: page fault while in kernel mode >> cpuid = 7; apic id = 0e >> fault virtual address = 0x1 >> fault code = supervisor read data, page not present >> instruction pointer = 0x20:0xffffffff806e38aa >> stack pointer = 0x28:0xfffffe047ba18440 >> frame pointer = 0x28:0xfffffe047ba18490 >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags = interrupt enabled, resume, IOPL = 0 >> current process = 2205 (main) >> trap number = 12 >> panic: page fault >> cpuid = 7 >> KDB: stack backtrace: >> #0 0xffffffff80b240f7 at kdb_backtrace+0x67 >> #1 0xffffffff80ad9462 at vpanic+0x182 >> #2 0xffffffff80ad92d3 at panic+0x43 >> #3 0xffffffff80fa1d51 at trap_fatal+0x351 >> #4 0xffffffff80fa1f43 at trap_pfault+0x1e3 >> #5 0xffffffff80fa14ec at trap+0x26c >> #6 0xffffffff80f841c1 at calltrap+0x8 >> #7 0xffffffff806e5a80 at generic_netmap_txsync+0x330 >> #8 0xffffffff806e06f9 at netmap_ioctl+0x279 >> #9 0xffffffff8098624f at devfs_ioctl_f+0x13f >> #10 0xffffffff80b41b34 at kern_ioctl+0x2d4 >> #11 0xffffffff80b417f1 at sys_ioctl+0x171 >> #12 0xffffffff80fa26ae at amd64_syscall+0x4ce >> #13 0xffffffff80f844ab at Xfast_syscall+0xfb >> >> This is on 11.0-RELEASE-p8 >> >> Thanks, >> Joe Jones >> >> On 23/03/17 18:20, Vincenzo Maffione wrote: >> >> Hi, >> You could try to use netmap in emulated mode (sysctl >> dev.netmap.admode=2). If this works, at least you know that >> the problem is in the cxgbe netmap support and not in the >> netmap core itself. >> >> Cheers, >> Vincenzo >> >> 2017-03-23 14:00 GMT+01:00 Joe Jones >> > >> > >> >>: >> >> Hello, >> >> We have a T520-SO and have made a new install of 11.0, to >> begin >> with the box would panic every time we tried to switch the >> card >> into netmap mode. So we recompiled the kernel with netmap >> removed, >> then compiled the netmap kernel module from github, as >> this in our >> experience generally leads to a more stable netmap. >> >> we have >> >> uname -a >> FreeBSD goose2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0: >> Wed Mar >> 22 16:52:35 UTC 2017 joe@goose2:/usr/obj/usr/src/sys/GENERIC >> amd64 >> >> and the following in /boot/loader.conf >> >> t4fw_cfg_load="YES" >> t5fw_cfg_load="YES" >> if_cxgbe_load="YES" >> hw.cxgbe.fl_pktshift=0 >> hw.cxgbe.toecaps_allowed=0 >> hw.cxgbe.nnmtxq10g=8 >> hw.cxgbe.nnmrxq10g=8 >> hw.cxgbe.num_vis=2 >> >> Before I run our application I run >> >> ifconfig cxl1 promisc -vlanhwtag up >> >> Now our application can now start without panicking the >> kernel. >> Here is where it gets interesting, our application is able to >> announce it's self via ARP, I can see the ethernet switch >> learning >> which port it's on, and other hosts adding it to their ARP >> tables. >> When I try an ICMP ping it goes missing. After watching the TX >> packet graph for the connected port on the switch while >> starting >> and stopping a flood ping to the application, I'm sure the >> packets >> are getting sent to the card, however I don't see them in the >> netmap ring. If I kill our application, then use ifconfig to >> create and configure a vlan port I can confirm that the >> card is >> working and has connectivity. >> >> Here's what I think is happening. ARP requests are received >> because they are sent to the broadcast address. Our >> application >> then announces it's self. However traffic destined for the >> application is send to a MAC address which is neither the >> broadcast or the MAC programed into the hardware and is >> dropped. >> My understanding of promiscuous it that it informs the >> card that >> we want these dropped packets. It looks to me like, when >> the card >> is in netmap mode the promisc flag is being ignored. >> >> I have also tried using freebsd-update to update to p8. As >> with >> the p0 kernel we get a panic when we switch the card into >> netmap mode. >> >> We did previously have these cards working in netmap mode. >> We were >> using a pre 11 snapshot of the svn head though . >> >> Many Thanks >> >> Joe Jones >> Stream Technologies >> >> _______________________________________________ >> freebsd-net@freebsd.org >> > > mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> >> > > >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org >> >> > >" >> >> >> >> >> -- Vincenzo Maffione >> >> >> >> >> >> -- >> Vincenzo Maffione >> > > -- Vincenzo Maffione