Date: Wed, 18 Nov 2020 22:45:57 +0100 From: Vincenzo Maffione <vmaffione@freebsd.org> To: Rajesh Kumar <rajfbsd@gmail.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Netmap bridge not working with 10G Ethernet ports Message-ID: <CA%2B_eA9hR8ysiFGj-iriMpqXcDbc4X_h_C1sgNoO05KoLy5orCA@mail.gmail.com> In-Reply-To: <CAAO%2BANOg5MEfHf9bV5x4L_QXNY2O9vQk0s%2BJrD7yzeXCQfHt8w@mail.gmail.com> References: <CAAO%2BANOg5MEfHf9bV5x4L_QXNY2O9vQk0s%2BJrD7yzeXCQfHt8w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, Il giorno mer 18 nov 2020 alle ore 08:13 Rajesh Kumar <rajfbsd@gmail.com> ha scritto: > Hi, > > I am testing a 10G Network driver with Netmap "bridge" utility, where it > doesn't seem to work. Here is my setup details. > > *System under Test:* Running FreeBSD CURRENT. Has two inbuilt 10G NIC > ports. > *System 1:* Running Ubuntu, whose network port is connected to Port1 of > System Under Test > *System 2:* Running FreeBSD CURRENT, whose network port is connected to > Port 0 of System Under Test. > > Bridged the Port0 and Port1 of System Under Test using the Netmap "bridge" > utility. Able to see interfaces coming up active and Link UP. > # bridge -c -v -i netmap:ax0 -i netmap:ax1 > > This looks like if_axe(4) driver, and therefore there's no native netmap support, which means you are falling back on the emulated netmap adapter. Are these USB dongles? If so, how can they be 10G? > Then tried pinging from System 1 to System 2. It fails. > > *Observations:* > 1. ARP request from System 1 goes to bridge port 1 (netmap_rxsync) and then > forwarded to port 0 (netmap_txsync) > 2. ARP request is received in System 2 (via bridge port 0) and ARP reply is > being sent from System 2. > 3. ARP reply from System 2 seems to be not reaching bridge port 0 to get > forwarded to bridge 1 and hence to System 1. > 4. Above 3 steps happen 3 times for ARP resolution cycle and then fails. > Hence the ping fails. > > On Debugging, when the ARP reply is being sent from System 2, I don't see > any interrupt triggered on the bridge port 0 in system under test. > > In this kind of configuration it is mandatory to disable all the NIC offloads, because netmap does not program the NIC to honor them, e.g.: # ifconfig ax0 -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6 # ifconfig ax1 -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6 > Netstat in system under test, doesn't show any receive or drop counters > incremented. But as I understand netstat capture the stats above the netmap > stack. Hence not reflecting the counts. > Correct. > > *Note:* > a) I tried with another vendor 10G NIC card. It behaves the same way. So > this issue doesn't seem to be generic and not hardware specific. > Which driver are those NICs using? That makes the difference. I guess it's still a driver with no native netmap support, hence you are using the same emulated adapter. > b) Trying with another vendor 1G NIC card, things are working. So not > sure, what makes a difference here. The ports in System 1 and System 2 are > USB attached Ethernet capable of maximum speed of 1G. So does connecting > 1G to 10G bridge ports is having any impact? > I don't think so. On each p2p link the NICs will negotiate 1G speed. In any case, what driver was this one? > c) We have verified the same 10G driver with pkt-gen utility and things are > working. Facing issue only when using "bridge" utility. > That may be because pkt-gen does not care about checksums, whereas the TCP/IP stack does. Hence the need to disable offloads (see above). Cheers, Vincenzo > So, wondering how the ARP reply packet is getting lost here. Any ideas to > debug? > > Thanks, > Rajesh. > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B_eA9hR8ysiFGj-iriMpqXcDbc4X_h_C1sgNoO05KoLy5orCA>