Date: Fri, 29 May 2020 19:00:51 +0200 From: Anthony Arnaud <antho.arnaudisce@gmail.com> To: Vincenzo Maffione <vmaffione@freebsd.org> Cc: Luigi Rizzo <rizzo@iet.unipi.it>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Netmap - Vale switch - tcp problem Message-ID: <CAGCUUMG7HG%2B8X89AYrhQ4dh6-equsvOLNyn_3sNU8h55PjPv9A@mail.gmail.com> In-Reply-To: <CA%2B_eA9iukqVoDZFyyW2qEZ2OHfE9=fUj070uUXcYmrXt67mwqw@mail.gmail.com> References: <CAGCUUMGxxiv8j-W3S8Gs1g=ZbDLnU8_Hv6SVMCpmnrRXPNsUTA@mail.gmail.com> <CA%2BhQ2%2BhfDvKv8q%2BjUB-mv_TaUTHfTPabx-qem-0=PW3vu8Br=A@mail.gmail.com> <CAGCUUMHwRghU3AMgKhQNrUs248y=NaGFN2Fm-f4khqhRsWn-qg@mail.gmail.com> <CA%2B_eA9iukqVoDZFyyW2qEZ2OHfE9=fUj070uUXcYmrXt67mwqw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Vincenzo,
thanks for your hints!
I rebooted my guest FBSD 12.1 machine, and I have perfomed your steps
#ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 up promisc
vtnet1:
flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric
0 mtu 1500
options=6c04b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 0e:bd:ec:7a:08:06
media: Ethernet 10Gbase-T <full-duplex>
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
tcpdump is ok.
But after:
#vale-ctl
446.196827 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1
446.196855 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^
tcpdump not work anymore.
I don't understand why the tcp traffic disappears.
In my configuration vtnet1 is a mirror port created by Open vSwitch, but I
don't think that's the reason.
Below some info about my configuration:
dev.netmap.iflib_rx_miss_bufs: 0
dev.netmap.iflib_rx_miss: 0
dev.netmap.iflib_crcstrip: 1
dev.netmap.bridge_batch: 1024
dev.netmap.default_pipes: 0
dev.netmap.priv_buf_num: 4098
dev.netmap.priv_buf_size: 2048
dev.netmap.buf_curr_num: 163840
dev.netmap.buf_num: 163840
dev.netmap.buf_curr_size: 2048
dev.netmap.buf_size: 2048
dev.netmap.priv_ring_num: 4
dev.netmap.priv_ring_size: 20480
dev.netmap.ring_curr_num: 200
dev.netmap.ring_num: 200
dev.netmap.ring_curr_size: 36864
dev.netmap.ring_size: 36864
dev.netmap.priv_if_num: 2
dev.netmap.priv_if_size: 1024
dev.netmap.if_curr_num: 100
dev.netmap.if_num: 100
dev.netmap.if_curr_size: 1024
dev.netmap.if_size: 1024
dev.netmap.ptnet_vnet_hdr: 1
dev.netmap.generic_rings: 1
dev.netmap.generic_ringsize: 1024
dev.netmap.generic_mit: 100000
dev.netmap.generic_hwcsum: 0
dev.netmap.admode: 0
dev.netmap.fwd: 0
dev.netmap.txsync_retry: 2
dev.netmap.no_pendintr: 1
dev.netmap.no_timestamp: 0
dev.netmap.verbose: 0
dev.vtnet.1.txq0.rescheduled: 0
dev.vtnet.1.txq0.tso: 0
dev.vtnet.1.txq0.csum: 0
dev.vtnet.1.txq0.omcasts: 0
dev.vtnet.1.txq0.obytes: 0
dev.vtnet.1.txq0.opackets: 0
dev.vtnet.1.rxq0.rescheduled: 0
dev.vtnet.1.rxq0.csum_failed: 0
dev.vtnet.1.rxq0.csum: 66
dev.vtnet.1.rxq0.ierrors: 0
dev.vtnet.1.rxq0.iqdrops: 0
dev.vtnet.1.rxq0.ibytes: 11904780
dev.vtnet.1.rxq0.ipackets: 40984
dev.vtnet.1.tx_task_rescheduled: 0
dev.vtnet.1.tx_tso_offloaded: 0
dev.vtnet.1.tx_csum_offloaded: 0
dev.vtnet.1.tx_defrag_failed: 0
dev.vtnet.1.tx_defragged: 0
dev.vtnet.1.tx_tso_not_tcp: 0
dev.vtnet.1.tx_tso_bad_ethtype: 0
dev.vtnet.1.tx_csum_bad_ethtype: 0
dev.vtnet.1.rx_task_rescheduled: 0
dev.vtnet.1.rx_csum_offloaded: 0
dev.vtnet.1.rx_csum_failed: 0
dev.vtnet.1.rx_csum_bad_proto: 0
dev.vtnet.1.rx_csum_bad_offset: 0
dev.vtnet.1.rx_csum_bad_ipproto: 0
dev.vtnet.1.rx_csum_bad_ethtype: 0
dev.vtnet.1.rx_mergeable_failed: 0
dev.vtnet.1.rx_enq_replacement_failed: 0
dev.vtnet.1.rx_frame_too_large: 0
dev.vtnet.1.mbuf_alloc_failed: 0
dev.vtnet.1.act_vq_pairs: 1
dev.vtnet.1.requested_vq_pairs: 0
dev.vtnet.1.max_vq_pairs: 1
dev.vtnet.1.%parent: virtio_pci4
dev.vtnet.1.%pnpinfo:
dev.vtnet.1.%location:
dev.vtnet.1.%driver: vtnet
dev.vtnet.1.%desc: VirtIO Networking Adapter
dev.vtnet.0.txq0.rescheduled: 0
Cheers,
Anthony
Il giorno gio 28 mag 2020 alle ore 21:38 Vincenzo Maffione <
vmaffione@freebsd.org> ha scritto:
> Hi,
> I was trying to reproduce your problem (same FreeBSD release as yours).
> It looks like there is some sort of bad interaction with LRO.
>
> Starting from a fresh boot, if you keep lro enabled, e.g.
> # ifconfig vtnet0 -txcsum -rxcsum -tso4 -tso6
> # vale-ctl ....
> then I experience no problem (TCP works between vtnet0 and the host,
> tcpdump on vtnet0 works as expected).
>
> As soon as you disable LRO:
> # ifconfig vtnet0 -lro
> both TCP and tcpdump stop working.
> If I enable LRO again, TCP restarts working, but tcpdump doesn't. I need
> to reboot the machine to fix it.
>
> Btw, creating vi0 (persistent VALE port) is not relevant for this test.
> You may as well use ephemeral VALE ports (e.g. run pkt-gen -i vale0:1 -f
> rx).
>
> I will have a look at the LRO issue asap. In the meantime you could avoid
> disabling LRO and see if that works for you.
>
> Cheers,
> Vincenzo
>
> Il giorno gio 28 mag 2020 alle ore 17:16 Anthony Arnaud <
> antho.arnaudisce@gmail.com> ha scritto:
>
>> I already disabled the checksum, the vtnet config is:
>>
>> ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6
>> -vlanmtu
>> -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up promisc
>>
>> vtnet1:
>> flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC>
>> metric
>> 0 mtu 1500
>> options=1800a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE,NETMAP>
>> ether 0e:bd:ec:7a:08:06
>> media: Ethernet 10Gbase-T <full-duplex>
>> status: active
>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>>
>> Sorry for not having posted vtnet config before.
>> PS: VLAN_HWCSUM is not switchable off for some reason!
>> the problem is not that.
>>
>> Cheers
>> Anthony
>>
>>
>>
>>
>> Il giorno gio 28 mag 2020 alle ore 16:05 Luigi Rizzo <rizzo@iet.unipi.it>
>> ha scritto:
>>
>> >
>> >
>> > On Thursday, May 28, 2020, Anthony Arnaud <antho.arnaudisce@gmail.com>
>> > wrote:
>> >
>> >> Hi everyone!
>> >> I would like to create a vale switch with an interface attached with
>> the
>> >> host stack and some virtual.
>> >> My env is a VM with FBSD-12.1 12.1-RELEASE FreeBSD 12.1-RELEASE r354233
>> >> GENERIC amd64
>> >> and VirtIO support.
>> >>
>> >> I performed:
>> >>
>> >> vale-ctl -h vale0:vtnet1
>> >> vale-ctl -n vi0
>> >> vale-ctl -a vale0:vi0
>> >>
>> >> 615.925514 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1
>> >> 615.925559 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^
>> >> 615.925572 bdg_ctl [149] bridge:0 port:2 vale0:vi0
>> >>
>> >> vtnet1 is configured as mirror port.
>> >> But if:
>> >>
>> >> tcpdump -i vtnet1
>> >> or
>> >> tcpdump -i vale0:vi0
>> >>
>> >> why can't I see any TCP packets?
>> >> UDP and ICMP packet are ok.
>> >>
>> >> Without vale switch tcpdump show all TCP packets correctly.
>> >
>> >
>> > You have to disable checksum offloading on vtnet1.
>> >
>> > Cheers
>> > Luigi
>> >
>> > It is a bug?
>> >> Thanks to all!
>> >> <freebsd-net@freebsd.org>
>> >> _______________________________________________
>> >> freebsd-net@freebsd.org mailing list
>> >> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>> >>
>> >
>> >
>> > --
>> >
>> -----------------------------------------+-------------------------------
>> > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing.
>> dell'Informazione
>> > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
>> > TEL +39-050-2217533 . via Diotisalvi 2
>> > Mobile +39-338-6809875 . 56122 PISA (Italy)
>> >
>> -----------------------------------------+-------------------------------
>> >
>> >
>> _______________________________________________
>> freebsd-net@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGCUUMG7HG%2B8X89AYrhQ4dh6-equsvOLNyn_3sNU8h55PjPv9A>