From owner-freebsd-net@freebsd.org Mon Jun 1 13:19:23 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 08A863363F0 for ; Mon, 1 Jun 2020 13:19:23 +0000 (UTC) (envelope-from antho.arnaudisce@gmail.com) Received: from mail-oo1-xc42.google.com (mail-oo1-xc42.google.com [IPv6:2607:f8b0:4864:20::c42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49bG3Z21ZDz3cm8; Mon, 1 Jun 2020 13:19:22 +0000 (UTC) (envelope-from antho.arnaudisce@gmail.com) Received: by mail-oo1-xc42.google.com with SMTP id h7so1390315ooc.9; Mon, 01 Jun 2020 06:19:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HWl0TDJ++sdrR+vUCr/bqmdv1p3p97kzADDcAe5dYPQ=; b=Vrg5u6L4+QPhf20f+XBHguOGrcUV5hDUp7KDM8iHc5m0KPXskTDWViGjRZyoHfaFXZ tB6kG89+osG9ycROHU2/2BjxRvkaPcTuS9+aRJ8cG0OPRR0MpSg+z/L2IUyqdlmJcXEO ADYT2u2jNFUQMSC/kfr+5z+YGZaVhSCyrfMJmY0Qzq0PhCoSWGvHTRH2k4RnbOs6V040 iZh6Upy0PSBlXMB1WdSqkpQym11c5DgSAcjEytmeep1X9yHAhSCHjG5dtGepppLrb+gm A4HoZJEog8wqwmPMMR+5U0LRBP+9A0jwUBOfa+RvOhd975pA5QT0IFvBdc92YKK8NusE r5aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HWl0TDJ++sdrR+vUCr/bqmdv1p3p97kzADDcAe5dYPQ=; b=ON9TlA6j1qyhxGccwGBz6n7CgvHKsbSn9iADbU7bsx+BeqitB/UKv4Ojiitm2E6ZxB wi2UKJj+pvNqFOKw8MEtWcs5bZ+HL6vVyZr7mpDWjvkK1nGI1B/uTsC7liO7oId7+2hu 7mZzNzZ0QLtvz03vijVqicb8qyopLV4OHOrLx4Cjl8Djof1iuLidmWJDTv0yoTEcLaHa Nt3Kb5iBqgZQZX+WMP0PsVKhoTsu3T4PGdMtWFuGcyKJukM2dCMt24PPOa3DjEBgm2hf lKPjiXgZpRL9QQ4aG7bajrVR2siWIyMVK6sA55udTekc56oRfZ8dLaLe9t/Kk3IFReSA hdng== X-Gm-Message-State: AOAM532IwQFROjlhHY793dOdmDK6Agr++yzaD/bblZcGuNVWbIyKjxqh koHQ39vzpBcw9JSqnQm/8sDLvuvjSB09CwLGynOhcLIH X-Google-Smtp-Source: ABdhPJy+SBTAI500Bxb8qSkTGL88fRMjqtncsesn2PhwtSyixNYdv6Sy6XVn6J+YCtu1lkeYfhyZN6WA8iSxfYer7bY= X-Received: by 2002:a4a:381:: with SMTP id 123mr15020982ooi.85.1591017560744; Mon, 01 Jun 2020 06:19:20 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Anthony Arnaud Date: Mon, 1 Jun 2020 15:19:09 +0200 Message-ID: Subject: Re: Netmap - Vale switch - tcp problem To: Vincenzo Maffione Cc: Luigi Rizzo , "freebsd-net@freebsd.org" X-Rspamd-Queue-Id: 49bG3Z21ZDz3cm8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Vrg5u6L4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of anthoarnaudisce@gmail.com designates 2607:f8b0:4864:20::c42 as permitted sender) smtp.mailfrom=anthoarnaudisce@gmail.com X-Spamd-Result: default: False [-3.29 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; NEURAL_HAM_LONG(-1.00)[-0.995]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::c42:from]; HTTP_TO_IP(1.00)[]; NEURAL_HAM_SHORT(-1.30)[-1.296]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2020 13:19:23 -0000 Hi Vincenzo, To simplify the scenario I have installed from scratch FBSD12.1 on a new machine, without any virtualization env. I have encountered the same problem, when i attach an ethernet interface to vale switch (in this case an intel card em5) the tcp traffic disappears and tcpdump shown only udp, icmp6 and stp packets. If I detach the NIC from vale0 tcpdump shown all tcp traffic. I'm using the netmap version included in FBSD 12.1, and I have compiled vale-ctl presents in kernel sources (/src/tools/tools/netmap/) I executed your steps. There is something dark about that behaviour... Cheers Anthon Il giorno sab 30 mag 2020 alle ore 14:29 Vincenzo Maffione < vmaffione@freebsd.org> ha scritto: > > > Il giorno ven 29 mag 2020 alle ore 19:01 Anthony Arnaud < > antho.arnaudisce@gmail.com> ha scritto: > >> Hi Vincenzo, >> >> thanks for your hints! >> I rebooted my guest FBSD 12.1 machine, and I have perfomed your steps >> >> #ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 up promisc >> >> vtnet1: >> flags=28943 metric >> 0 mtu 1500 >> >> options=6c04b8 >> ether 0e:bd:ec:7a:08:06 >> media: Ethernet 10Gbase-T >> status: active >> nd6 options=29 >> >> tcpdump is ok. >> >> Does it mean you see both ICMP, UDP and TCP traffic? > > >> But after: >> >> #vale-ctl >> >> 446.196827 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1 >> 446.196855 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^ >> >> tcpdump not work anymore. >> > > Do you see ICMP/UDP only and not TCP? > > >> I don't understand why the tcp traffic disappears. >> In my configuration vtnet1 is a mirror port created by Open vSwitch, but >> I don't think that's the reason. >> >> No, I don't think that's relevant. > > In my setup, vtnet0 is a guest interface backed by a host tap device > (attached to a linux bridge), and hypervisor is QEMU/KVM. > Here are the steps I follow in the VM (in this order): > > # ifconfig vtnet0 -txcsum -rxcsum -tso4 -tso6 up 192.168.100.2/24 > # vale-ctl -h vale0:vtnet0 > > # nc 192.168.100.1 5555 # connect to listening netcat in the host. > hello > abc > [...] > # tcpdump -ni vtnet0 # This shows the TCP traffic. > > I start to see problems when I change the offloads: > # ifconfig vtnet0 -lro > > Cheers, > Vincenzo > > >> Below some info about my configuration: >> >> dev.netmap.iflib_rx_miss_bufs: 0 >> dev.netmap.iflib_rx_miss: 0 >> dev.netmap.iflib_crcstrip: 1 >> dev.netmap.bridge_batch: 1024 >> dev.netmap.default_pipes: 0 >> dev.netmap.priv_buf_num: 4098 >> dev.netmap.priv_buf_size: 2048 >> dev.netmap.buf_curr_num: 163840 >> dev.netmap.buf_num: 163840 >> dev.netmap.buf_curr_size: 2048 >> dev.netmap.buf_size: 2048 >> dev.netmap.priv_ring_num: 4 >> dev.netmap.priv_ring_size: 20480 >> dev.netmap.ring_curr_num: 200 >> dev.netmap.ring_num: 200 >> dev.netmap.ring_curr_size: 36864 >> dev.netmap.ring_size: 36864 >> dev.netmap.priv_if_num: 2 >> dev.netmap.priv_if_size: 1024 >> dev.netmap.if_curr_num: 100 >> dev.netmap.if_num: 100 >> dev.netmap.if_curr_size: 1024 >> dev.netmap.if_size: 1024 >> dev.netmap.ptnet_vnet_hdr: 1 >> dev.netmap.generic_rings: 1 >> dev.netmap.generic_ringsize: 1024 >> dev.netmap.generic_mit: 100000 >> dev.netmap.generic_hwcsum: 0 >> dev.netmap.admode: 0 >> dev.netmap.fwd: 0 >> dev.netmap.txsync_retry: 2 >> dev.netmap.no_pendintr: 1 >> dev.netmap.no_timestamp: 0 >> dev.netmap.verbose: 0 >> >> >> dev.vtnet.1.txq0.rescheduled: 0 >> dev.vtnet.1.txq0.tso: 0 >> dev.vtnet.1.txq0.csum: 0 >> dev.vtnet.1.txq0.omcasts: 0 >> dev.vtnet.1.txq0.obytes: 0 >> dev.vtnet.1.txq0.opackets: 0 >> dev.vtnet.1.rxq0.rescheduled: 0 >> dev.vtnet.1.rxq0.csum_failed: 0 >> dev.vtnet.1.rxq0.csum: 66 >> dev.vtnet.1.rxq0.ierrors: 0 >> dev.vtnet.1.rxq0.iqdrops: 0 >> dev.vtnet.1.rxq0.ibytes: 11904780 >> dev.vtnet.1.rxq0.ipackets: 40984 >> dev.vtnet.1.tx_task_rescheduled: 0 >> dev.vtnet.1.tx_tso_offloaded: 0 >> dev.vtnet.1.tx_csum_offloaded: 0 >> dev.vtnet.1.tx_defrag_failed: 0 >> dev.vtnet.1.tx_defragged: 0 >> dev.vtnet.1.tx_tso_not_tcp: 0 >> dev.vtnet.1.tx_tso_bad_ethtype: 0 >> dev.vtnet.1.tx_csum_bad_ethtype: 0 >> dev.vtnet.1.rx_task_rescheduled: 0 >> dev.vtnet.1.rx_csum_offloaded: 0 >> dev.vtnet.1.rx_csum_failed: 0 >> dev.vtnet.1.rx_csum_bad_proto: 0 >> dev.vtnet.1.rx_csum_bad_offset: 0 >> dev.vtnet.1.rx_csum_bad_ipproto: 0 >> dev.vtnet.1.rx_csum_bad_ethtype: 0 >> dev.vtnet.1.rx_mergeable_failed: 0 >> dev.vtnet.1.rx_enq_replacement_failed: 0 >> dev.vtnet.1.rx_frame_too_large: 0 >> dev.vtnet.1.mbuf_alloc_failed: 0 >> dev.vtnet.1.act_vq_pairs: 1 >> dev.vtnet.1.requested_vq_pairs: 0 >> dev.vtnet.1.max_vq_pairs: 1 >> dev.vtnet.1.%parent: virtio_pci4 >> dev.vtnet.1.%pnpinfo: >> dev.vtnet.1.%location: >> dev.vtnet.1.%driver: vtnet >> dev.vtnet.1.%desc: VirtIO Networking Adapter >> dev.vtnet.0.txq0.rescheduled: 0 >> >> Cheers, >> Anthony >> >> Il giorno gio 28 mag 2020 alle ore 21:38 Vincenzo Maffione < >> vmaffione@freebsd.org> ha scritto: >> >>> Hi, >>> I was trying to reproduce your problem (same FreeBSD release as >>> yours). It looks like there is some sort of bad interaction with LRO. >>> >>> Starting from a fresh boot, if you keep lro enabled, e.g. >>> # ifconfig vtnet0 -txcsum -rxcsum -tso4 -tso6 >>> # vale-ctl .... >>> then I experience no problem (TCP works between vtnet0 and the host, >>> tcpdump on vtnet0 works as expected). >>> >>> As soon as you disable LRO: >>> # ifconfig vtnet0 -lro >>> both TCP and tcpdump stop working. >>> If I enable LRO again, TCP restarts working, but tcpdump doesn't. I need >>> to reboot the machine to fix it. >>> >>> Btw, creating vi0 (persistent VALE port) is not relevant for this test. >>> You may as well use ephemeral VALE ports (e.g. run pkt-gen -i vale0:1 -f >>> rx). >>> >>> I will have a look at the LRO issue asap. In the meantime you could >>> avoid disabling LRO and see if that works for you. >>> >>> Cheers, >>> Vincenzo >>> >>> Il giorno gio 28 mag 2020 alle ore 17:16 Anthony Arnaud < >>> antho.arnaudisce@gmail.com> ha scritto: >>> >>>> I already disabled the checksum, the vtnet config is: >>>> >>>> ifconfig vtnet1 -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6 >>>> -vlanmtu >>>> -vlanhwtag -vlanhwfilter -vlanhwtso -vlanhwcsum up promisc >>>> >>>> vtnet1: >>>> flags=28943 >>>> metric >>>> 0 mtu 1500 >>>> options=1800a8 >>>> ether 0e:bd:ec:7a:08:06 >>>> media: Ethernet 10Gbase-T >>>> status: active >>>> nd6 options=29 >>>> >>>> Sorry for not having posted vtnet config before. >>>> PS: VLAN_HWCSUM is not switchable off for some reason! >>>> the problem is not that. >>>> >>>> Cheers >>>> Anthony >>>> >>>> >>>> >>>> >>>> Il giorno gio 28 mag 2020 alle ore 16:05 Luigi Rizzo < >>>> rizzo@iet.unipi.it> >>>> ha scritto: >>>> >>>> > >>>> > >>>> > On Thursday, May 28, 2020, Anthony Arnaud >>> > >>>> > wrote: >>>> > >>>> >> Hi everyone! >>>> >> I would like to create a vale switch with an interface attached with >>>> the >>>> >> host stack and some virtual. >>>> >> My env is a VM with FBSD-12.1 12.1-RELEASE FreeBSD 12.1-RELEASE >>>> r354233 >>>> >> GENERIC amd64 >>>> >> and VirtIO support. >>>> >> >>>> >> I performed: >>>> >> >>>> >> vale-ctl -h vale0:vtnet1 >>>> >> vale-ctl -n vi0 >>>> >> vale-ctl -a vale0:vi0 >>>> >> >>>> >> 615.925514 bdg_ctl [149] bridge:0 port:0 vale0:vtnet1 >>>> >> 615.925559 bdg_ctl [149] bridge:0 port:1 vale0:vtnet1^ >>>> >> 615.925572 bdg_ctl [149] bridge:0 port:2 vale0:vi0 >>>> >> >>>> >> vtnet1 is configured as mirror port. >>>> >> But if: >>>> >> >>>> >> tcpdump -i vtnet1 >>>> >> or >>>> >> tcpdump -i vale0:vi0 >>>> >> >>>> >> why can't I see any TCP packets? >>>> >> UDP and ICMP packet are ok. >>>> >> >>>> >> Without vale switch tcpdump show all TCP packets correctly. >>>> > >>>> > >>>> > You have to disable checksum offloading on vtnet1. >>>> > >>>> > Cheers >>>> > Luigi >>>> > >>>> > It is a bug? >>>> >> Thanks to all! >>>> >> >>>> >> _______________________________________________ >>>> >> freebsd-net@freebsd.org mailing list >>>> >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> >> To unsubscribe, send any mail to " >>>> freebsd-net-unsubscribe@freebsd.org" >>>> >> >>>> > >>>> > >>>> > -- >>>> > >>>> -----------------------------------------+------------------------------- >>>> > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >>>> dell'Informazione >>>> > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>>> > TEL +39-050-2217533 . via Diotisalvi 2 >>>> > Mobile +39-338-6809875 . 56122 PISA (Italy) >>>> > >>>> -----------------------------------------+------------------------------- >>>> > >>>> > >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> >>>