Date: Sun, 30 Jun 2013 18:25:18 +0400 From: "Alexander V. Chernikov" <melifaro@FreeBSD.org> To: net@freebsd.org Cc: Navdeep Parhar <np@FreeBSD.org> Subject: cxgbetool & hw filtering issues Message-ID: <51D03FCE.1060102@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
Hello list!
While experimenting with Chelsio T440-CR (cxgbe) internal firewall, I'm
getting some kind of unexpected results:
filtering 'type ipv4 action drop' permits IPv4 TCP traffic with bad
checksum.
filtering 'type IPv6 action drop' permits IPv6 traffic to multicast
addresses (MLDv2, etc..)
filtering 'ethtype 34525 action drop' (drop all IPv6) results in
'CHELSIO_T4_SET_FILTER: Argument list too long' despite to what is said
in budget table from cxgbetool.8
filtering 'matchtype 4 action drop' or similar (4,5,4:0,4:4, 5:0, 5:5)
does not match anything despite some traffic definitely falls into that
conditions.
filtering 'action drop' and 'iport X action drop' filters IPv4 traffic only.
filter 'type ipv6 ...' can be set on (0,4,8,12,...) filter numbers
yelling 'CHELSIO_T4_SET_FILTER: Invalid argument' on other numbers.
What can I do to debug further/fix this behavior?
Some more questions:
Does anybody known how I can get/set total number of HW firewall
records? There is such tunable in Linux version.
Is there any way to retrieve _host_ interface statistic (e.g. how much
traffic in packets/bytes are thrown to NIC driver)?
Setup description:
[packet generator replaying small PCAP with 280kpps rate] -> cxgbe3
[[FreeBSD 10-CURRENT r248721]].
PCAP is captured on my host machine so
1) Outgoing TCP checksums are almost all wrong
2) DST macs are not modified (so they are all unknown to NIC).
cxgbe3:
flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC>
metric 0 mtu 1500
options=6c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:07:43:11:88:d8
inet6 fe80::207:43ff:fe11:88d8%cxgbe3 prefixlen 64 scopeid 0x9
inet6 2a02:6b8:0:401:207:43ff:fe11:88d8 prefixlen 64 detached
deprecated autoconf
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-Twinax <full-duplex>
status: active
dev.t4nex.0.%desc: Chelsio T440-CR NIC (rev 2), S/N:PT42110574,
E/C:01234567890123
dev.t4nex.0.%driver: t4nex
dev.t4nex.0.%location: slot=0 function=4
dev.t4nex.0.%pnpinfo: vendor=0x1425 device=0x4403 subvendor=0x1425
subdevice=0x0000 class=0x020000
dev.t4nex.0.%parent: pci8
dev.t4nex.0.nports: 4
dev.t4nex.0.hw_revision: 2
dev.t4nex.0.firmware_version: 1.8.4.0
dev.t4nex.0.cf: default
dev.t4nex.0.cfcsum: 4260083439
dev.t4nex.0.linkcaps: 0
dev.t4nex.0.niccaps: 1<NIC>
dev.t4nex.0.toecaps: 0
dev.t4nex.0.rdmacaps: 0
dev.t4nex.0.iscsicaps: 0
dev.t4nex.0.fcoecaps: 0
dev.t4nex.0.core_clock: 228125
dev.t4nex.0.holdoff_timers: 1 5 10 50 100 200
dev.t4nex.0.holdoff_pkt_counts: 1 8 16 32
dev.t4nex.0.fwq.abs_id: 0
dev.t4nex.0.fwq.cntxt_id: 0
dev.t4nex.0.fwq.cidx: 121
dev.t4nex.0.mgmtq.cntxt_id: 0
dev.t4nex.0.mgmtq.cidx: 95
dev.t4nex.0.mgmtq.pidx: 111
dev.t4nex.0.mgmtq.tx_wrs: 119
dev.t4nex.0.mgmtq.no_desc: 0
dev.t4nex.0.mgmtq.unstalled: 0
# kenv | grep cxgbe
hw.cxgbe.fcoecaps_allowed="0"
hw.cxgbe.iscsicaps_allowed="0"
hw.cxgbe.nrxq10g="4"
hw.cxgbe.ntxq10g="4"
hw.cxgbe.qsize_rxq="4096"
hw.cxgbe.qsize_txq="4096"
hw.cxgbe.rdmacaps_allowed="0"
hw.cxgbe.toecaps_allowed="0"
TRAFFIC PART:
input (cxgbe3) output
packets errs idrops bytes packets errs bytes colls
284368 0 0 85436494 0 0 0 0
284340 0 0 85442168 0 0 0 0
284205 0 0 85464055 0 0 0 0
...
(not changing, nearly constant rate, is not affected by filters)
# ipfw show 200
00200 16860 2685762 deny ip from any to any via cxgbe3
# Running counter to see how much is actually dropped/passed
# while true; do sleep 1; ipfw show 200 ; ipfw -q zero 200 ;done
[[ empty filters ]]
00200 281769 80351685 deny ip from any to any via cxgbe3
..
[[ ### (1) IPv4 EXPERIMENT ]]
[[ # ./cxgbetool t4nex0 filter 0 type ipv4 action drop ]]
00200 115263 15431259 deny ip from any to any via cxgbe3
00200 116523 15584332 deny ip from any to any via cxgbe3
[[# time tcpdump -i cxgbe3 -lnps0 -c 100 ip
18:18:42.621728 IP 95.108.170.36.39215 > 93.158.158.93.80: Flags [.],
ack 4252241156, win 995, options [nop,nop,TS val 538195932 ecr
1194270183], length 0
..
tcpdump -i cxgbe3 -lnps0 -c 100 ip 0,00s user 0,01s system 15% cpu
0,059 total
#]]
[[ ### (2) IPv6 EXPERIMENT ]]
[[ # ./cxgbetool t4nex0 filter 4 type ipv6 action drop ]]
00200 64962 10332022 deny ip from any to any via cxgbe3
00200 64878 10327694 deny ip from any to any via cxgbe3
...
[[# time tcpdump -i cxgbe3 -lnps0 -c 100 ip6
18:21:34.553596 IP6 fe80::884:a1e8:86ae:57f7 > ff02::16: HBH ICMP6,
multicast listener report v2, 3 group record(s), length 68
..
tcpdump -i cxgbe3 -lnps0 -c 100 ip6 0,00s user 0,00s system 0% cpu
0,483 total
#]]
Address in (1) is my host machine address, viewing resulting .pcap file
in wireshark shows incorrect TCP checksums for IPv4 packets.
Other pcaps not containing "bad" traffic are properly filtered by rules
above.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51D03FCE.1060102>