From owner-freebsd-net@freebsd.org Sat May 30 12:29:57 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9D257331586 for ; Sat, 30 May 2020 12:29:57 +0000 (UTC) (envelope-from vmaffione@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Z13T3yXfz3ZY9 for ; Sat, 30 May 2020 12:29:57 +0000 (UTC) (envelope-from vmaffione@freebsd.org) Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: vmaffione) by smtp.freebsd.org (Postfix) with ESMTPSA id 7B08A19BD4 for ; Sat, 30 May 2020 12:29:57 +0000 (UTC) (envelope-from vmaffione@freebsd.org) Received: by mail-qt1-f172.google.com with SMTP id c12so4161062qtq.11 for ; Sat, 30 May 2020 05:29:57 -0700 (PDT) X-Gm-Message-State: AOAM533WgbqNEokvcnufFwopXilAAodRDw+kgKobMhG+ivYAmrPMhnxP YJeySNpGJYFanpoDeHnplv0CJAzzrKyJ/y/NemY= X-Google-Smtp-Source: ABdhPJxPOH8fo6MHauNOvWAUqHtNQJA05leyQronytBChqO0lftSGehoF4qs0AILokTPpFdL5r9tBZ5/VVDJOx/0XUw= X-Received: by 2002:ac8:6ec2:: with SMTP id f2mr13276353qtv.309.1590841796993; Sat, 30 May 2020 05:29:56 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Vincenzo Maffione Date: Sat, 30 May 2020 14:29:45 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Netmap - Vale switch - tcp problem To: Anthony Arnaud Cc: Luigi Rizzo , "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 May 2020 12:29:57 -0000 Il giorno ven 29 mag 2020 alle ore 19:01 Anthony Arnaud < antho.arnaudisce@gmail.com> ha scritto: > Hi Vincenzo, > > thanks for your hints! > I rebooted my guest FBSD 12.1 machine, and I have perfomed your steps > > #ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 up promisc > > vtnet1: > flags=28943 metric > 0 mtu 1500 > > options=6c04b8 > ether 0e:bd:ec:7a:08:06 > media: Ethernet 10Gbase-T > status: active > nd6 options=29 > > tcpdump is ok. > > Does it mean you see both ICMP, UDP and TCP traffic? > But after: > > #vale-ctl > > 446.196827 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1 > 446.196855 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^ > > tcpdump not work anymore. > Do you see ICMP/UDP only and not TCP? > I don't understand why the tcp traffic disappears. > In my configuration vtnet1 is a mirror port created by Open vSwitch, but I > don't think that's the reason. > > No, I don't think that's relevant. In my setup, vtnet0 is a guest interface backed by a host tap device (attached to a linux bridge), and hypervisor is QEMU/KVM. Here are the steps I follow in the VM (in this order): # ifconfig vtnet0 -txcsum -rxcsum -tso4 -tso6 up 192.168.100.2/24 # vale-ctl -h vale0:vtnet0 # nc 192.168.100.1 5555 # connect to listening netcat in the host. hello abc [...] # tcpdump -ni vtnet0 # This shows the TCP traffic. I start to see problems when I change the offloads: # ifconfig vtnet0 -lro Cheers, Vincenzo > Below some info about my configuration: > > dev.netmap.iflib_rx_miss_bufs: 0 > dev.netmap.iflib_rx_miss: 0 > dev.netmap.iflib_crcstrip: 1 > dev.netmap.bridge_batch: 1024 > dev.netmap.default_pipes: 0 > dev.netmap.priv_buf_num: 4098 > dev.netmap.priv_buf_size: 2048 > dev.netmap.buf_curr_num: 163840 > dev.netmap.buf_num: 163840 > dev.netmap.buf_curr_size: 2048 > dev.netmap.buf_size: 2048 > dev.netmap.priv_ring_num: 4 > dev.netmap.priv_ring_size: 20480 > dev.netmap.ring_curr_num: 200 > dev.netmap.ring_num: 200 > dev.netmap.ring_curr_size: 36864 > dev.netmap.ring_size: 36864 > dev.netmap.priv_if_num: 2 > dev.netmap.priv_if_size: 1024 > dev.netmap.if_curr_num: 100 > dev.netmap.if_num: 100 > dev.netmap.if_curr_size: 1024 > dev.netmap.if_size: 1024 > dev.netmap.ptnet_vnet_hdr: 1 > dev.netmap.generic_rings: 1 > dev.netmap.generic_ringsize: 1024 > dev.netmap.generic_mit: 100000 > dev.netmap.generic_hwcsum: 0 > dev.netmap.admode: 0 > dev.netmap.fwd: 0 > dev.netmap.txsync_retry: 2 > dev.netmap.no_pendintr: 1 > dev.netmap.no_timestamp: 0 > dev.netmap.verbose: 0 > > > dev.vtnet.1.txq0.rescheduled: 0 > dev.vtnet.1.txq0.tso: 0 > dev.vtnet.1.txq0.csum: 0 > dev.vtnet.1.txq0.omcasts: 0 > dev.vtnet.1.txq0.obytes: 0 > dev.vtnet.1.txq0.opackets: 0 > dev.vtnet.1.rxq0.rescheduled: 0 > dev.vtnet.1.rxq0.csum_failed: 0 > dev.vtnet.1.rxq0.csum: 66 > dev.vtnet.1.rxq0.ierrors: 0 > dev.vtnet.1.rxq0.iqdrops: 0 > dev.vtnet.1.rxq0.ibytes: 11904780 > dev.vtnet.1.rxq0.ipackets: 40984 > dev.vtnet.1.tx_task_rescheduled: 0 > dev.vtnet.1.tx_tso_offloaded: 0 > dev.vtnet.1.tx_csum_offloaded: 0 > dev.vtnet.1.tx_defrag_failed: 0 > dev.vtnet.1.tx_defragged: 0 > dev.vtnet.1.tx_tso_not_tcp: 0 > dev.vtnet.1.tx_tso_bad_ethtype: 0 > dev.vtnet.1.tx_csum_bad_ethtype: 0 > dev.vtnet.1.rx_task_rescheduled: 0 > dev.vtnet.1.rx_csum_offloaded: 0 > dev.vtnet.1.rx_csum_failed: 0 > dev.vtnet.1.rx_csum_bad_proto: 0 > dev.vtnet.1.rx_csum_bad_offset: 0 > dev.vtnet.1.rx_csum_bad_ipproto: 0 > dev.vtnet.1.rx_csum_bad_ethtype: 0 > dev.vtnet.1.rx_mergeable_failed: 0 > dev.vtnet.1.rx_enq_replacement_failed: 0 > dev.vtnet.1.rx_frame_too_large: 0 > dev.vtnet.1.mbuf_alloc_failed: 0 > dev.vtnet.1.act_vq_pairs: 1 > dev.vtnet.1.requested_vq_pairs: 0 > dev.vtnet.1.max_vq_pairs: 1 > dev.vtnet.1.%parent: virtio_pci4 > dev.vtnet.1.%pnpinfo: > dev.vtnet.1.%location: > dev.vtnet.1.%driver: vtnet > dev.vtnet.1.%desc: VirtIO Networking Adapter > dev.vtnet.0.txq0.rescheduled: 0 > > Cheers, > Anthony > > Il giorno gio 28 mag 2020 alle ore 21:38 Vincenzo Maffione < > vmaffione@freebsd.org> ha scritto: > >> Hi, >> I was trying to reproduce your problem (same FreeBSD release as yours). >> It looks like there is some sort of bad interaction with LRO. >> >> Starting from a fresh boot, if you keep lro enabled, e.g. >> # ifconfig vtnet0 -txcsum -rxcsum -tso4 -tso6 >> # vale-ctl .... >> then I experience no problem (TCP works between vtnet0 and the host, >> tcpdump on vtnet0 works as expected). >> >> As soon as you disable LRO: >> # ifconfig vtnet0 -lro >> both TCP and tcpdump stop working. >> If I enable LRO again, TCP restarts working, but tcpdump doesn't. I need >> to reboot the machine to fix it. >> >> Btw, creating vi0 (persistent VALE port) is not relevant for this test. >> You may as well use ephemeral VALE ports (e.g. run pkt-gen -i vale0:1 -f >> rx). >> >> I will have a look at the LRO issue asap. In the meantime you could avoid >> disabling LRO and see if that works for you. >> >> Cheers, >> Vincenzo >> >> Il giorno gio 28 mag 2020 alle ore 17:16 Anthony Arnaud < >> antho.arnaudisce@gmail.com> ha scritto: >> >>> I already disabled the checksum, the vtnet config is: >>> >>> ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6 >>> -vlanmtu >>> -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up promisc >>> >>> vtnet1: >>> flags=28943 >>> metric >>> 0 mtu 1500 >>> options=1800a8 >>> ether 0e:bd:ec:7a:08:06 >>> media: Ethernet 10Gbase-T >>> status: active >>> nd6 options=29 >>> >>> Sorry for not having posted vtnet config before. >>> PS: VLAN_HWCSUM is not switchable off for some reason! >>> the problem is not that. >>> >>> Cheers >>> Anthony >>> >>> >>> >>> >>> Il giorno gio 28 mag 2020 alle ore 16:05 Luigi Rizzo >> > >>> ha scritto: >>> >>> > >>> > >>> > On Thursday, May 28, 2020, Anthony Arnaud >>> > wrote: >>> > >>> >> Hi everyone! >>> >> I would like to create a vale switch with an interface attached with >>> the >>> >> host stack and some virtual. >>> >> My env is a VM with FBSD-12.1 12.1-RELEASE FreeBSD 12.1-RELEASE >>> r354233 >>> >> GENERIC amd64 >>> >> and VirtIO support. >>> >> >>> >> I performed: >>> >> >>> >> vale-ctl -h vale0:vtnet1 >>> >> vale-ctl -n vi0 >>> >> vale-ctl -a vale0:vi0 >>> >> >>> >> 615.925514 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1 >>> >> 615.925559 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^ >>> >> 615.925572 bdg_ctl [149] bridge:0 port:2 vale0:vi0 >>> >> >>> >> vtnet1 is configured as mirror port. >>> >> But if: >>> >> >>> >> tcpdump -i vtnet1 >>> >> or >>> >> tcpdump -i vale0:vi0 >>> >> >>> >> why can't I see any TCP packets? >>> >> UDP and ICMP packet are ok. >>> >> >>> >> Without vale switch tcpdump show all TCP packets correctly. >>> > >>> > >>> > You have to disable checksum offloading on vtnet1. >>> > >>> > Cheers >>> > Luigi >>> > >>> > It is a bug? >>> >> Thanks to all! >>> >> >>> >> _______________________________________________ >>> >> freebsd-net@freebsd.org mailing list >>> >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org >>> " >>> >> >>> > >>> > >>> > -- >>> > >>> -----------------------------------------+------------------------------- >>> > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >>> dell'Informazione >>> > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>> > TEL +39-050-2217533 . via Diotisalvi 2 >>> > Mobile +39-338-6809875 . 56122 PISA (Italy) >>> > >>> -----------------------------------------+------------------------------- >>> > >>> > >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>