From owner-freebsd-net@freebsd.org Fri May 29 17:01:03 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CA7A733CD68 for ; Fri, 29 May 2020 17:01:03 +0000 (UTC) (envelope-from antho.arnaudisce@gmail.com) Received: from mail-ot1-x341.google.com (mail-ot1-x341.google.com [IPv6:2607:f8b0:4864:20::341]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49YW6l53bzz449F; Fri, 29 May 2020 17:01:03 +0000 (UTC) (envelope-from antho.arnaudisce@gmail.com) Received: by mail-ot1-x341.google.com with SMTP id 69so2472423otv.2; Fri, 29 May 2020 10:01:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AkBvg34JLKWgcC/7pYn2N221KeKbJVyoxmByeNb8ETw=; b=alEnsPON4XwVt6O4Ratkt6LaaU8f2tAYqTXy95ZxU+bhwIwY99N9PGFtxKYMwHf1O7 7tq6NQAXSo4lLakGUatwuGRJPd7PPrv2dYuc2SaHdMZj9vK4L5F2QbsL1OJ60+5Hwwg0 yNwcs09UkgY3bbndQw4DlPW3htXpM7MNQKj2l7bsM9bREjiKH7JWzoIFm71lTSvdVhWM 8M5xw2c0cv5pCjgfzD05QZrDGKZNnPsI1onRd1hV+jGK/ledPrO1qtqtN8NPMjECy/H7 ZDXXl2raDtRsPPic9wnwdnFTsHRCP/dXnQ2+0lxPbRSuC/yRdCgSVOmNDHd1ip2/zY03 T7sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AkBvg34JLKWgcC/7pYn2N221KeKbJVyoxmByeNb8ETw=; b=DWvX2Kj77CHqsnBwUBy5uEiMURd8IwLlaPCbLV0IQIzjq+FmI38WaI/PMkTWbSf2MH gtBm8DzUB5V3jl9sHxpU24rYOgdRKetvrHm/BihCb5W411YCD/x8qp78UIhW9D/aDw+5 +OeMVZQPtVd6WIRTD9I0zOeB4nIWGrpYLl8MxjNHicfZCTYECT2YZwpTPB/dyKB5sDSJ BJyr8p++0Am2qdJNrA+avE4O22ASefna/S3+6S5KDfmX8qJ9if38/SlZJaxi/QvIZV36 kZTJXauE8rQuViZ5GiMl8ppSaUCNbufvjouauzXEdJAbUMIrD5nObli7stiYMN3e2am+ T2MA== X-Gm-Message-State: AOAM533frRYLpurZZHWkoG7j+kJnDW5XNgD7+1z+od+3jyiHqk/qvXRc 5oTavYgSZIJ6hGFbcFocyAgI6Qgd3Kf/35887kRIYFox X-Google-Smtp-Source: ABdhPJw9Ep9VE0ZABSfgXbNlBp5qAqbnmsbnhA35dhKkBJbSpXRnT3hwclwbP0mcDvEeAwEIEuItm1zPPtBqRTFtL9g= X-Received: by 2002:a05:6830:1d69:: with SMTP id l9mr7418830oti.127.1590771662163; Fri, 29 May 2020 10:01:02 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Anthony Arnaud Date: Fri, 29 May 2020 19:00:51 +0200 Message-ID: Subject: Re: Netmap - Vale switch - tcp problem To: Vincenzo Maffione Cc: Luigi Rizzo , "freebsd-net@freebsd.org" X-Rspamd-Queue-Id: 49YW6l53bzz449F X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 May 2020 17:01:03 -0000 Hi Vincenzo, thanks for your hints! I rebooted my guest FBSD 12.1 machine, and I have perfomed your steps #ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 up promisc vtnet1: flags=28943 metric 0 mtu 1500 options=6c04b8 ether 0e:bd:ec:7a:08:06 media: Ethernet 10Gbase-T status: active nd6 options=29 tcpdump is ok. But after: #vale-ctl 446.196827 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1 446.196855 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^ tcpdump not work anymore. I don't understand why the tcp traffic disappears. In my configuration vtnet1 is a mirror port created by Open vSwitch, but I don't think that's the reason. Below some info about my configuration: dev.netmap.iflib_rx_miss_bufs: 0 dev.netmap.iflib_rx_miss: 0 dev.netmap.iflib_crcstrip: 1 dev.netmap.bridge_batch: 1024 dev.netmap.default_pipes: 0 dev.netmap.priv_buf_num: 4098 dev.netmap.priv_buf_size: 2048 dev.netmap.buf_curr_num: 163840 dev.netmap.buf_num: 163840 dev.netmap.buf_curr_size: 2048 dev.netmap.buf_size: 2048 dev.netmap.priv_ring_num: 4 dev.netmap.priv_ring_size: 20480 dev.netmap.ring_curr_num: 200 dev.netmap.ring_num: 200 dev.netmap.ring_curr_size: 36864 dev.netmap.ring_size: 36864 dev.netmap.priv_if_num: 2 dev.netmap.priv_if_size: 1024 dev.netmap.if_curr_num: 100 dev.netmap.if_num: 100 dev.netmap.if_curr_size: 1024 dev.netmap.if_size: 1024 dev.netmap.ptnet_vnet_hdr: 1 dev.netmap.generic_rings: 1 dev.netmap.generic_ringsize: 1024 dev.netmap.generic_mit: 100000 dev.netmap.generic_hwcsum: 0 dev.netmap.admode: 0 dev.netmap.fwd: 0 dev.netmap.txsync_retry: 2 dev.netmap.no_pendintr: 1 dev.netmap.no_timestamp: 0 dev.netmap.verbose: 0 dev.vtnet.1.txq0.rescheduled: 0 dev.vtnet.1.txq0.tso: 0 dev.vtnet.1.txq0.csum: 0 dev.vtnet.1.txq0.omcasts: 0 dev.vtnet.1.txq0.obytes: 0 dev.vtnet.1.txq0.opackets: 0 dev.vtnet.1.rxq0.rescheduled: 0 dev.vtnet.1.rxq0.csum_failed: 0 dev.vtnet.1.rxq0.csum: 66 dev.vtnet.1.rxq0.ierrors: 0 dev.vtnet.1.rxq0.iqdrops: 0 dev.vtnet.1.rxq0.ibytes: 11904780 dev.vtnet.1.rxq0.ipackets: 40984 dev.vtnet.1.tx_task_rescheduled: 0 dev.vtnet.1.tx_tso_offloaded: 0 dev.vtnet.1.tx_csum_offloaded: 0 dev.vtnet.1.tx_defrag_failed: 0 dev.vtnet.1.tx_defragged: 0 dev.vtnet.1.tx_tso_not_tcp: 0 dev.vtnet.1.tx_tso_bad_ethtype: 0 dev.vtnet.1.tx_csum_bad_ethtype: 0 dev.vtnet.1.rx_task_rescheduled: 0 dev.vtnet.1.rx_csum_offloaded: 0 dev.vtnet.1.rx_csum_failed: 0 dev.vtnet.1.rx_csum_bad_proto: 0 dev.vtnet.1.rx_csum_bad_offset: 0 dev.vtnet.1.rx_csum_bad_ipproto: 0 dev.vtnet.1.rx_csum_bad_ethtype: 0 dev.vtnet.1.rx_mergeable_failed: 0 dev.vtnet.1.rx_enq_replacement_failed: 0 dev.vtnet.1.rx_frame_too_large: 0 dev.vtnet.1.mbuf_alloc_failed: 0 dev.vtnet.1.act_vq_pairs: 1 dev.vtnet.1.requested_vq_pairs: 0 dev.vtnet.1.max_vq_pairs: 1 dev.vtnet.1.%parent: virtio_pci4 dev.vtnet.1.%pnpinfo: dev.vtnet.1.%location: dev.vtnet.1.%driver: vtnet dev.vtnet.1.%desc: VirtIO Networking Adapter dev.vtnet.0.txq0.rescheduled: 0 Cheers, Anthony Il giorno gio 28 mag 2020 alle ore 21:38 Vincenzo Maffione < vmaffione@freebsd.org> ha scritto: > Hi, > I was trying to reproduce your problem (same FreeBSD release as yours). > It looks like there is some sort of bad interaction with LRO. > > Starting from a fresh boot, if you keep lro enabled, e.g. > # ifconfig vtnet0 -txcsum -rxcsum -tso4 -tso6 > # vale-ctl .... > then I experience no problem (TCP works between vtnet0 and the host, > tcpdump on vtnet0 works as expected). > > As soon as you disable LRO: > # ifconfig vtnet0 -lro > both TCP and tcpdump stop working. > If I enable LRO again, TCP restarts working, but tcpdump doesn't. I need > to reboot the machine to fix it. > > Btw, creating vi0 (persistent VALE port) is not relevant for this test. > You may as well use ephemeral VALE ports (e.g. run pkt-gen -i vale0:1 -f > rx). > > I will have a look at the LRO issue asap. In the meantime you could avoid > disabling LRO and see if that works for you. > > Cheers, > Vincenzo > > Il giorno gio 28 mag 2020 alle ore 17:16 Anthony Arnaud < > antho.arnaudisce@gmail.com> ha scritto: > >> I already disabled the checksum, the vtnet config is: >> >> ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6 >> -vlanmtu >> -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up promisc >> >> vtnet1: >> flags=28943 >> metric >> 0 mtu 1500 >> options=1800a8 >> ether 0e:bd:ec:7a:08:06 >> media: Ethernet 10Gbase-T >> status: active >> nd6 options=29 >> >> Sorry for not having posted vtnet config before. >> PS: VLAN_HWCSUM is not switchable off for some reason! >> the problem is not that. >> >> Cheers >> Anthony >> >> >> >> >> Il giorno gio 28 mag 2020 alle ore 16:05 Luigi Rizzo >> ha scritto: >> >> > >> > >> > On Thursday, May 28, 2020, Anthony Arnaud >> > wrote: >> > >> >> Hi everyone! >> >> I would like to create a vale switch with an interface attached with >> the >> >> host stack and some virtual. >> >> My env is a VM with FBSD-12.1 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 >> >> GENERIC amd64 >> >> and VirtIO support. >> >> >> >> I performed: >> >> >> >> vale-ctl -h vale0:vtnet1 >> >> vale-ctl -n vi0 >> >> vale-ctl -a vale0:vi0 >> >> >> >> 615.925514 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1 >> >> 615.925559 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^ >> >> 615.925572 bdg_ctl [149] bridge:0 port:2 vale0:vi0 >> >> >> >> vtnet1 is configured as mirror port. >> >> But if: >> >> >> >> tcpdump -i vtnet1 >> >> or >> >> tcpdump -i vale0:vi0 >> >> >> >> why can't I see any TCP packets? >> >> UDP and ICMP packet are ok. >> >> >> >> Without vale switch tcpdump show all TCP packets correctly. >> > >> > >> > You have to disable checksum offloading on vtnet1. >> > >> > Cheers >> > Luigi >> > >> > It is a bug? >> >> Thanks to all! >> >> >> >> _______________________________________________ >> >> freebsd-net@freebsd.org mailing list >> >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> > >> > >> > -- >> > >> -----------------------------------------+------------------------------- >> > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >> dell'Informazione >> > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> > TEL +39-050-2217533 . via Diotisalvi 2 >> > Mobile +39-338-6809875 . 56122 PISA (Italy) >> > >> -----------------------------------------+------------------------------- >> > >> > >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >