Date: Fri, 24 Mar 2017 11:29:36 +0100 From: Vincenzo Maffione <v.maffione@gmail.com> To: Joe Jones <joe@stream-technologies.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: cxgbe netmap promiscuous mode? Message-ID: <CA%2B_eA9itNjXmsu26ZsFZCp4T=txA0nDb%2BozCTrxpZaA1ym_O7Q@mail.gmail.com> In-Reply-To: <58D4F2C5.9020204@stream-technologies.com> References: <58D3C6F4.6010500@stream-technologies.com> <CA%2B_eA9hvvMrp8Amc8EJy3jGq51x0vD=Ad=ruTacP4oR0X66=Ew@mail.gmail.com> <58D4F2C5.9020204@stream-technologies.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Joe, There was a fix for a panic in emulated mode that was applied stable/11 branch, so I guess it also ended up into FreeBSD-11.0-STABLE. I don't know whether the same fix ended up into in 11.0-RELEASE-p8 (I'm not familiar with FreeBSD releasing process, sorry!). Or maybe this panic happens with netmap upstream? If this is a new bug, it would be nice to have the kernel with the debug symbols enabled, so that we can get more detailed information from the stack trace. Cheers, Vincenzo 2017-03-24 11:19 GMT+01:00 Joe Jones <joe@stream-technologies.com>: > Hi Vincenzo, > > I just tried with that sysctl set to 2, I get a similar looking panic to > before > > Fatal trap 12: page fault while in kernel mode > cpuid = 7; apic id = 0e > fault virtual address = 0x1 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff806e38aa > stack pointer = 0x28:0xfffffe047ba18440 > frame pointer = 0x28:0xfffffe047ba18490 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 2205 (main) > trap number = 12 > panic: page fault > cpuid = 7 > KDB: stack backtrace: > #0 0xffffffff80b240f7 at kdb_backtrace+0x67 > #1 0xffffffff80ad9462 at vpanic+0x182 > #2 0xffffffff80ad92d3 at panic+0x43 > #3 0xffffffff80fa1d51 at trap_fatal+0x351 > #4 0xffffffff80fa1f43 at trap_pfault+0x1e3 > #5 0xffffffff80fa14ec at trap+0x26c > #6 0xffffffff80f841c1 at calltrap+0x8 > #7 0xffffffff806e5a80 at generic_netmap_txsync+0x330 > #8 0xffffffff806e06f9 at netmap_ioctl+0x279 > #9 0xffffffff8098624f at devfs_ioctl_f+0x13f > #10 0xffffffff80b41b34 at kern_ioctl+0x2d4 > #11 0xffffffff80b417f1 at sys_ioctl+0x171 > #12 0xffffffff80fa26ae at amd64_syscall+0x4ce > #13 0xffffffff80f844ab at Xfast_syscall+0xfb > > This is on 11.0-RELEASE-p8 > > Thanks, > Joe Jones > > On 23/03/17 18:20, Vincenzo Maffione wrote: > >> Hi, >> You could try to use netmap in emulated mode (sysctl >> dev.netmap.admode=2). If this works, at least you know that the problem is >> in the cxgbe netmap support and not in the netmap core itself. >> >> Cheers, >> Vincenzo >> >> 2017-03-23 14:00 GMT+01:00 Joe Jones <joe@stream-technologies.com >> <mailto:joe@stream-technologies.com>>: >> >> Hello, >> >> We have a T520-SO and have made a new install of 11.0, to begin >> with the box would panic every time we tried to switch the card >> into netmap mode. So we recompiled the kernel with netmap removed, >> then compiled the netmap kernel module from github, as this in our >> experience generally leads to a more stable netmap. >> >> we have >> >> uname -a >> FreeBSD goose2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0: Wed Mar >> 22 16:52:35 UTC 2017 joe@goose2:/usr/obj/usr/src/sys/GENERIC amd64 >> >> and the following in /boot/loader.conf >> >> t4fw_cfg_load="YES" >> t5fw_cfg_load="YES" >> if_cxgbe_load="YES" >> hw.cxgbe.fl_pktshift=0 >> hw.cxgbe.toecaps_allowed=0 >> hw.cxgbe.nnmtxq10g=8 >> hw.cxgbe.nnmrxq10g=8 >> hw.cxgbe.num_vis=2 >> >> Before I run our application I run >> >> ifconfig cxl1 promisc -vlanhwtag up >> >> Now our application can now start without panicking the kernel. >> Here is where it gets interesting, our application is able to >> announce it's self via ARP, I can see the ethernet switch learning >> which port it's on, and other hosts adding it to their ARP tables. >> When I try an ICMP ping it goes missing. After watching the TX >> packet graph for the connected port on the switch while starting >> and stopping a flood ping to the application, I'm sure the packets >> are getting sent to the card, however I don't see them in the >> netmap ring. If I kill our application, then use ifconfig to >> create and configure a vlan port I can confirm that the card is >> working and has connectivity. >> >> Here's what I think is happening. ARP requests are received >> because they are sent to the broadcast address. Our application >> then announces it's self. However traffic destined for the >> application is send to a MAC address which is neither the >> broadcast or the MAC programed into the hardware and is dropped. >> My understanding of promiscuous it that it informs the card that >> we want these dropped packets. It looks to me like, when the card >> is in netmap mode the promisc flag is being ignored. >> >> I have also tried using freebsd-update to update to p8. As with >> the p0 kernel we get a panic when we switch the card into netmap mode. >> >> We did previously have these cards working in netmap mode. We were >> using a pre 11 snapshot of the svn head though . >> >> Many Thanks >> >> Joe Jones >> Stream Technologies >> >> _______________________________________________ >> freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org> mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> <https://lists.freebsd.org/mailman/listinfo/freebsd-net>; >> To unsubscribe, send any mail to >> "freebsd-net-unsubscribe@freebsd.org >> <mailto:freebsd-net-unsubscribe@freebsd.org>" >> >> >> >> >> -- >> Vincenzo Maffione >> > > -- Vincenzo Maffione
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B_eA9itNjXmsu26ZsFZCp4T=txA0nDb%2BozCTrxpZaA1ym_O7Q>