Date: Fri, 24 Mar 2017 13:50:39 +0000 From: Joe Jones <joe@stream-technologies.com> To: Vincenzo Maffione <v.maffione@gmail.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: cxgbe netmap promiscuous mode? Message-ID: <58D5242F.9040706@stream-technologies.com> In-Reply-To: <CA%2B_eA9itNjXmsu26ZsFZCp4T=txA0nDb%2BozCTrxpZaA1ym_O7Q@mail.gmail.com> References: <58D3C6F4.6010500@stream-technologies.com> <CA%2B_eA9hvvMrp8Amc8EJy3jGq51x0vD=Ad=ruTacP4oR0X66=Ew@mail.gmail.com> <58D4F2C5.9020204@stream-technologies.com> <CA%2B_eA9itNjXmsu26ZsFZCp4T=txA0nDb%2BozCTrxpZaA1ym_O7Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Vincenzo, we can get rid if the panic if we compile the kernel without netmap, then load an up to date netmap as a module. We can live with that for now. Some time last year before Free BSD 11 was branched from head, we compiled a checkout and it worked without problem on the same hardware setup. I may try and figure out what the regression is, I'm not familiar with the FreeBSD release process either though. Thanks, Joe Jones On 24/03/17 10:29, Vincenzo Maffione wrote: > Hi Joe, > There was a fix for a panic in emulated mode that was applied > stable/11 branch, so I guess it also ended up into FreeBSD-11.0-STABLE. > I don't know whether the same fix ended up into in 11.0-RELEASE-p8 > (I'm not familiar with FreeBSD releasing process, sorry!). > > Or maybe this panic happens with netmap upstream? > If this is a new bug, it would be nice to have the kernel with the > debug symbols enabled, so that we can get more detailed information > from the stack trace. > > Cheers, > Vincenzo > > 2017-03-24 11:19 GMT+01:00 Joe Jones <joe@stream-technologies.com > <mailto:joe@stream-technologies.com>>: > > Hi Vincenzo, > > I just tried with that sysctl set to 2, I get a similar looking > panic to before > > Fatal trap 12: page fault while in kernel mode > cpuid = 7; apic id = 0e > fault virtual address = 0x1 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff806e38aa > stack pointer = 0x28:0xfffffe047ba18440 > frame pointer = 0x28:0xfffffe047ba18490 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 2205 (main) > trap number = 12 > panic: page fault > cpuid = 7 > KDB: stack backtrace: > #0 0xffffffff80b240f7 at kdb_backtrace+0x67 > #1 0xffffffff80ad9462 at vpanic+0x182 > #2 0xffffffff80ad92d3 at panic+0x43 > #3 0xffffffff80fa1d51 at trap_fatal+0x351 > #4 0xffffffff80fa1f43 at trap_pfault+0x1e3 > #5 0xffffffff80fa14ec at trap+0x26c > #6 0xffffffff80f841c1 at calltrap+0x8 > #7 0xffffffff806e5a80 at generic_netmap_txsync+0x330 > #8 0xffffffff806e06f9 at netmap_ioctl+0x279 > #9 0xffffffff8098624f at devfs_ioctl_f+0x13f > #10 0xffffffff80b41b34 at kern_ioctl+0x2d4 > #11 0xffffffff80b417f1 at sys_ioctl+0x171 > #12 0xffffffff80fa26ae at amd64_syscall+0x4ce > #13 0xffffffff80f844ab at Xfast_syscall+0xfb > > This is on 11.0-RELEASE-p8 > > Thanks, > Joe Jones > > On 23/03/17 18:20, Vincenzo Maffione wrote: > > Hi, > You could try to use netmap in emulated mode (sysctl > dev.netmap.admode=2). If this works, at least you know that > the problem is in the cxgbe netmap support and not in the > netmap core itself. > > Cheers, > Vincenzo > > 2017-03-23 14:00 GMT+01:00 Joe Jones > <joe@stream-technologies.com > <mailto:joe@stream-technologies.com> > <mailto:joe@stream-technologies.com > <mailto:joe@stream-technologies.com>>>: > > Hello, > > We have a T520-SO and have made a new install of 11.0, to > begin > with the box would panic every time we tried to switch the > card > into netmap mode. So we recompiled the kernel with netmap > removed, > then compiled the netmap kernel module from github, as > this in our > experience generally leads to a more stable netmap. > > we have > > uname -a > FreeBSD goose2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0: > Wed Mar > 22 16:52:35 UTC 2017 > joe@goose2:/usr/obj/usr/src/sys/GENERIC amd64 > > and the following in /boot/loader.conf > > t4fw_cfg_load="YES" > t5fw_cfg_load="YES" > if_cxgbe_load="YES" > hw.cxgbe.fl_pktshift=0 > hw.cxgbe.toecaps_allowed=0 > hw.cxgbe.nnmtxq10g=8 > hw.cxgbe.nnmrxq10g=8 > hw.cxgbe.num_vis=2 > > Before I run our application I run > > ifconfig cxl1 promisc -vlanhwtag up > > Now our application can now start without panicking the > kernel. > Here is where it gets interesting, our application is able to > announce it's self via ARP, I can see the ethernet switch > learning > which port it's on, and other hosts adding it to their ARP > tables. > When I try an ICMP ping it goes missing. After watching the TX > packet graph for the connected port on the switch while > starting > and stopping a flood ping to the application, I'm sure the > packets > are getting sent to the card, however I don't see them in the > netmap ring. If I kill our application, then use ifconfig to > create and configure a vlan port I can confirm that the > card is > working and has connectivity. > > Here's what I think is happening. ARP requests are received > because they are sent to the broadcast address. Our > application > then announces it's self. However traffic destined for the > application is send to a MAC address which is neither the > broadcast or the MAC programed into the hardware and is > dropped. > My understanding of promiscuous it that it informs the > card that > we want these dropped packets. It looks to me like, when > the card > is in netmap mode the promisc flag is being ignored. > > I have also tried using freebsd-update to update to p8. As > with > the p0 kernel we get a panic when we switch the card into > netmap mode. > > We did previously have these cards working in netmap mode. > We were > using a pre 11 snapshot of the svn head though . > > Many Thanks > > Joe Jones > Stream Technologies > > _______________________________________________ > freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org> > <mailto:freebsd-net@freebsd.org > <mailto:freebsd-net@freebsd.org>> mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > <https://lists.freebsd.org/mailman/listinfo/freebsd-net>; > <https://lists.freebsd.org/mailman/listinfo/freebsd-net > <https://lists.freebsd.org/mailman/listinfo/freebsd-net>>; > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org > <mailto:freebsd-net-unsubscribe@freebsd.org> > <mailto:freebsd-net-unsubscribe@freebsd.org > <mailto:freebsd-net-unsubscribe@freebsd.org>>" > > > > > -- > Vincenzo Maffione > > > > > > -- > Vincenzo Maffione
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?58D5242F.9040706>