Date: Sun, 30 Jun 2013 12:41:13 -0700 From: Navdeep Parhar <np@FreeBSD.org> To: "Alexander V. Chernikov" <melifaro@FreeBSD.org> Cc: net@freebsd.org Subject: Re: cxgbetool & hw filtering issues Message-ID: <51D089D9.6080901@FreeBSD.org> In-Reply-To: <51D03FCE.1060102@FreeBSD.org> References: <51D03FCE.1060102@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 06/30/13 07:25, Alexander V. Chernikov wrote: > Hello list! > > While experimenting with Chelsio T440-CR (cxgbe) internal firewall, I'm > getting some kind of unexpected results: One bit of general advice to begin with: add "hitcnts 1" to all your filter rules and then you can see how many incoming packets hit that filter in the output of "cxgbetool t4nex0 filter list". I really should make hitcnts=1 the default in the driver. > filtering 'type ipv4 action drop' permits IPv4 TCP traffic with bad > checksum. It may be that a bad checksum makes it an invalid IPv4 packet to the chip and so it doesn't hit the "type ipv4" rule. There is an entirely separate knob available to have the chip drop bad packets if you don't want to see them. The default is to let them through so that users can examine them with tcpdump etc. > filtering 'type IPv6 action drop' permits IPv6 traffic to multicast > addresses (MLDv2, etc..) The DMAC is an L2 multicast address? Try "proto 58 hitcnts 1 action drop" to get these ICMP6 packets. > filtering 'ethtype 34525 action drop' (drop all IPv6) results in > 'CHELSIO_T4_SET_FILTER: Argument list too long' despite to what is said > in budget table from cxgbetool.8 This _would_ have gotten everything with ethertype ipv6 but the default filter mode doesn't have ethtype enabled, which is why it's complaining: # cxgbetool t4nex0 filter mode ipv4 ipv6 sip dip sport dport matchtype proto ivlan iport fcoe > filtering 'matchtype 4 action drop' or similar (4,5,4:0,4:4, 5:0, 5:5) > does not match anything despite some traffic definitely falls into that > conditions. > filtering 'action drop' and 'iport X action drop' filters IPv4 traffic > only. Strange. I use "iport X action drop hitcnts 1" as a packet black hole all the time. Were these the only filters when you tried them? Are you sure your packets didn't hit some other rule and were delivered as a result of that? Check the order in "cxgbetool t4nex0 filter list" Also, are you going by the ifnet rx stats as displayed by netstat etc.? Right now the driver fills the ifnet stats directly from hardware registers rather than counting the packets that it actually received from the chip. The hardware registers include packets that would have been delivered to the driver if no filters were present but are dropped due to a filter. > filter 'type ipv6 ...' can be set on (0,4,8,12,...) filter numbers > yelling 'CHELSIO_T4_SET_FILTER: Invalid argument' on other numbers. Yes, IPv6 filters take 4 tid's (non-IPv6 take 1) and these tid's have to start at a naturally aligned boundary. No way around this. > What can I do to debug further/fix this behavior? > > Some more questions: > Does anybody known how I can get/set total number of HW firewall > records? There is such tunable in Linux version. I will add a simple sysctl for this. For now you can indirectly figure this out from the output of "sysctl -n dev.t4nex.0.misc.tids" -- the FTIDs are the filter tids. For example I see 1456 filters on this card: trantor:~# sysctl -n dev.t4nex.0.misc.tids ATID range: 0-8191, in use: 0 TID range: 2048-18431, in use: 0 STID range: 0-511, in use: 0 FTID range: 512-1967 HW TID usage: 0 IP users, 0 IPv6 users trantor:~# echo $((1967 - 512 + 1)) 1456 > Is there any way to retrieve _host_ interface statistic (e.g. how much > traffic in packets/bytes are thrown to NIC driver)? cxgbe(4) doesn't count this stuff itself. Currently it just reads the hardware registers once per second and it's done. Software stats would have to be per queue (and then aggregated from time to time). I'll wait to see where the PCPU counter work in the kernel goes before reworking this part of the driver. Regards, Navdeep
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51D089D9.6080901>