Date: Fri, 24 Mar 2017 10:19:49 +0000 From: Joe Jones <joe@stream-technologies.com> To: Vincenzo Maffione <v.maffione@gmail.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: cxgbe netmap promiscuous mode? Message-ID: <58D4F2C5.9020204@stream-technologies.com> In-Reply-To: <CA%2B_eA9hvvMrp8Amc8EJy3jGq51x0vD=Ad=ruTacP4oR0X66=Ew@mail.gmail.com> References: <58D3C6F4.6010500@stream-technologies.com> <CA%2B_eA9hvvMrp8Amc8EJy3jGq51x0vD=Ad=ruTacP4oR0X66=Ew@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Vincenzo,
I just tried with that sysctl set to 2, I get a similar looking panic
to before
Fatal trap 12: page fault while in kernel mode
cpuid = 7; apic id = 0e
fault virtual address = 0x1
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff806e38aa
stack pointer = 0x28:0xfffffe047ba18440
frame pointer = 0x28:0xfffffe047ba18490
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 2205 (main)
trap number = 12
panic: page fault
cpuid = 7
KDB: stack backtrace:
#0 0xffffffff80b240f7 at kdb_backtrace+0x67
#1 0xffffffff80ad9462 at vpanic+0x182
#2 0xffffffff80ad92d3 at panic+0x43
#3 0xffffffff80fa1d51 at trap_fatal+0x351
#4 0xffffffff80fa1f43 at trap_pfault+0x1e3
#5 0xffffffff80fa14ec at trap+0x26c
#6 0xffffffff80f841c1 at calltrap+0x8
#7 0xffffffff806e5a80 at generic_netmap_txsync+0x330
#8 0xffffffff806e06f9 at netmap_ioctl+0x279
#9 0xffffffff8098624f at devfs_ioctl_f+0x13f
#10 0xffffffff80b41b34 at kern_ioctl+0x2d4
#11 0xffffffff80b417f1 at sys_ioctl+0x171
#12 0xffffffff80fa26ae at amd64_syscall+0x4ce
#13 0xffffffff80f844ab at Xfast_syscall+0xfb
This is on 11.0-RELEASE-p8
Thanks,
Joe Jones
On 23/03/17 18:20, Vincenzo Maffione wrote:
> Hi,
> You could try to use netmap in emulated mode (sysctl
> dev.netmap.admode=2). If this works, at least you know that the
> problem is in the cxgbe netmap support and not in the netmap core itself.
>
> Cheers,
> Vincenzo
>
> 2017-03-23 14:00 GMT+01:00 Joe Jones <joe@stream-technologies.com
> <mailto:joe@stream-technologies.com>>:
>
> Hello,
>
> We have a T520-SO and have made a new install of 11.0, to begin
> with the box would panic every time we tried to switch the card
> into netmap mode. So we recompiled the kernel with netmap removed,
> then compiled the netmap kernel module from github, as this in our
> experience generally leads to a more stable netmap.
>
> we have
>
> uname -a
> FreeBSD goose2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0: Wed Mar
> 22 16:52:35 UTC 2017 joe@goose2:/usr/obj/usr/src/sys/GENERIC amd64
>
> and the following in /boot/loader.conf
>
> t4fw_cfg_load="YES"
> t5fw_cfg_load="YES"
> if_cxgbe_load="YES"
> hw.cxgbe.fl_pktshift=0
> hw.cxgbe.toecaps_allowed=0
> hw.cxgbe.nnmtxq10g=8
> hw.cxgbe.nnmrxq10g=8
> hw.cxgbe.num_vis=2
>
> Before I run our application I run
>
> ifconfig cxl1 promisc -vlanhwtag up
>
> Now our application can now start without panicking the kernel.
> Here is where it gets interesting, our application is able to
> announce it's self via ARP, I can see the ethernet switch learning
> which port it's on, and other hosts adding it to their ARP tables.
> When I try an ICMP ping it goes missing. After watching the TX
> packet graph for the connected port on the switch while starting
> and stopping a flood ping to the application, I'm sure the packets
> are getting sent to the card, however I don't see them in the
> netmap ring. If I kill our application, then use ifconfig to
> create and configure a vlan port I can confirm that the card is
> working and has connectivity.
>
> Here's what I think is happening. ARP requests are received
> because they are sent to the broadcast address. Our application
> then announces it's self. However traffic destined for the
> application is send to a MAC address which is neither the
> broadcast or the MAC programed into the hardware and is dropped.
> My understanding of promiscuous it that it informs the card that
> we want these dropped packets. It looks to me like, when the card
> is in netmap mode the promisc flag is being ignored.
>
> I have also tried using freebsd-update to update to p8. As with
> the p0 kernel we get a panic when we switch the card into netmap mode.
>
> We did previously have these cards working in netmap mode. We were
> using a pre 11 snapshot of the svn head though .
>
> Many Thanks
>
> Joe Jones
> Stream Technologies
>
> _______________________________________________
> freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org> mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> <https://lists.freebsd.org/mailman/listinfo/freebsd-net>;
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe@freebsd.org
> <mailto:freebsd-net-unsubscribe@freebsd.org>"
>
>
>
>
> --
> Vincenzo Maffione
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?58D4F2C5.9020204>