Date: Fri, 14 Apr 2017 16:10:59 +0100 From: Joe Jones <joe@stream-technologies.com> To: Navdeep Parhar <nparhar@gmail.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: cxgbe netmap promiscuous mode? Message-ID: <58F0E683.7050806@stream-technologies.com> In-Reply-To: <CAPFoGT9EFNcjpjszMVyGhB7ktKG2ir-srG5noPziPK=4mbXj_g@mail.gmail.com> References: <58D3C6F4.6010500@stream-technologies.com> <CAPFoGT-F86Y-yxEK03rmmJkr0LbP4ocgq27pqw3g=yqkdud4=Q@mail.gmail.com> <58D521C0.1000804@stream-technologies.com> <CAPFoGT9EFNcjpjszMVyGhB7ktKG2ir-srG5noPziPK=4mbXj_g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Navdeep,
I think I have found a driver bug. Earlier today I set up the switch I'm
using so that two of the ports mirror the traffic on one of the other
ports. We are planning on using a similar setup to allow packet tracing
without stressing the boxes our application is running on any more then
they are already.
I connected both ports to one of our cxgbe cards, My intention was to
use tcpdump to check that my switch config was doing what I thought it
should. I ran
ifconfig cxl? promisc -vlanhwtag up
on both interfaces, this is what the interfaces looked like
cxl0:
flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC>
metric 0 mtu 1500
options=ec07ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:07:43:33:8a:20
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-Twinax <full-duplex>
status: active
vcxl0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:07:43:33:8a:22
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-Twinax <full-duplex>
status: active
cxl1:
flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC>
metric 0 mtu 1500
options=ec07ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:07:43:33:8a:28
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-Twinax <full-duplex>
status: active
vcxl1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:07:43:33:8a:2a
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-Twinax <full-duplex>
status: active
The interesting thing is, a tcpdump on cxl0 showed all the traffic I
expected to see, while tcpdump on cxl1 showed only broadcast traffic.
After playing with the switch config to make sure the difference was not
on the switch I pulled both patch cables out and into another server
with the same card. On the second server I could use tcpdump and see all
the traffic I expected on either interface.
Then back on the original server, I reloaded the device driver and tried
again. Now I got only broadcast on cxl0 and cxl1. Then finally I got all
the traffic to show up by doing
ifconfig cxl1 -promisc
ifconfig cxl1 promisc
It would appearer to me that the card can get into a state where
ifconfig reports that it is in promiscuous mode when it is not.
Joe Jones
On 24/03/17 16:44, Navdeep Parhar wrote:
> On Fri, Mar 24, 2017 at 6:40 AM, Joe Jones <joe@stream-technologies.com> wrote:
>> Hello Navdeep,
>>
>> ...
>>
>> We were using our own MACs, we can fix the problem by using the mac from the
>> vcxl interface. Should we not be able to capture all traffic on the
>> interface regardless of what destination MAC it has.
> You should, but you'll need to put vcxl<n> in promiscous mode for
> that. The command that you posted had cxl<n> in promiscuous mode. As
> I said earlier they share the wire but operate as independent network
> interfaces otherwise.
>
> Regards,
> Navdeep
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?58F0E683.7050806>