From owner-freebsd-net@freebsd.org Sat Jun 6 18:32:32 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6DD0B33B16A for ; Sat, 6 Jun 2020 18:32:32 +0000 (UTC) (envelope-from antho.arnaudisce@gmail.com) Received: from mail-ot1-x32e.google.com (mail-ot1-x32e.google.com [IPv6:2607:f8b0:4864:20::32e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49fSmb16sMz4YRp; Sat, 6 Jun 2020 18:32:30 +0000 (UTC) (envelope-from antho.arnaudisce@gmail.com) Received: by mail-ot1-x32e.google.com with SMTP id o13so10353073otl.5; Sat, 06 Jun 2020 11:32:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Xoqz79JFIS1XoYN9MPC+9WE1JzZKJJrbOxDMPq8VU0E=; b=ZqzZqvssoX0cOMPzgaGHRq1bI5RPxOtV2k4dKeyB5ACF5B6E0ujGpdhRGvIvLup/BY txmM60ao1tMRZfLOAT+Qbk8sgPgxcKgzvT5PUoY5QMgjD8ZBqVPZHzWjkI1rAdfglEmS W3KKBv4J72jR+zQmG0RjPiLstlMtthDrqEErWI9kuPPVpTxDxTgSFYX4J4zuU6og+w2j H49psBvyKx2Yo4bnzVIqvE43It2tRdjwj0uX+CLPtvjV7aTT6HTv8TwCVQpe+XvDztRK 6zXXZ3vipfyjSOLo7R5S8xxQQO4GhGeQF98OXXaSsyE1m6NpgmxPpuE93Q9t5HJRes6U qnew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Xoqz79JFIS1XoYN9MPC+9WE1JzZKJJrbOxDMPq8VU0E=; b=hz+k/iuYqr1bj7wJ99fehKSr0WF0xOSXNdQPvrvOvCV5P9eDCVZWvGtLIERjc+sDxG rq06onC6tos4UXiMBhSUP4OSWzyJC2E2+o/lbVFzu2wrnNqiDjkBOEJhLAnpWSghCmTR Vf5WK8V/WmHQEgAI8Qz0/EAaSxdTFUP8g6P6Y6WOJ1HXRB6kJ1y36FUcFD2I0Gbr2M5l UdPaXd0FM8fdLR9Al/awVCYE23ykVwCLrcojXNEGaE/GbO+1oaJlOW8H/jji+ZPN2/c2 JlwJQflOk9W1u1OtoqFqJj1/yIREpUkFNlrlhj/h677Sz3mYQ4LHvc8GcrcYc5MgS9R/ ifUQ== X-Gm-Message-State: AOAM533L9SWPxJ+RtC3uTpPgR5WWSidK6Cv/glFGF7kmSAxaAQf53sWr q+WhRH8JU+3WA2SSuJNWGLrunkEYpyt1KwvkbFy0NDEG X-Google-Smtp-Source: ABdhPJzaWcUaUvkysD6X5+Zsf5A5653OYxxMSExQgjHZb30jl2t2HnJd5SokFpye72l0+Ep/DcFZMn925cgMiHstomw= X-Received: by 2002:a9d:798c:: with SMTP id h12mr12065855otm.297.1591468349116; Sat, 06 Jun 2020 11:32:29 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Anthony Arnaud Date: Sat, 6 Jun 2020 20:32:19 +0200 Message-ID: Subject: Re: Netmap - Vale switch - tcp problem To: Vincenzo Maffione Cc: Luigi Rizzo , "freebsd-net@freebsd.org" X-Rspamd-Queue-Id: 49fSmb16sMz4YRp X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2020 18:32:32 -0000 This works!! A good news! It works even if I connect the nic to vale switch directly from ssh, the connection does not drop like before your patch. Can you list the steps to setup the bridge in your tests? I thing "ifconfig bridge create" etc. Now I have to understand why it doesn't work in my previous case. Thanks, Antho Il giorno sab 6 giu 2020 alle ore 09:52 Vincenzo Maffione < vmaffione@freebsd.org> ha scritto: > > > Il giorno ven 5 giu 2020 alle ore 18:16 Anthony Arnaud < > antho.arnaudisce@gmail.com> ha scritto: > >> Hi Vincenzo, >> Thank you for your answers. >> I upgraded a guest machine in proxmox env to FreeBSD 13.0-CURRENT #0 >> r361830. >> After that I compiled the tools as usual from /src/tools/tools/netmap >> I configured 2 NIC vtnet >> >> vtnet0 with an ip 192.168.1.x >> vtnet1 without ip and csum disabled as mirror of vtnet0 (using Open >> vSwitch) >> Tcp traffic is generated by an ssh connection from my host to guest >> machine. >> > > Not clear where you run OVS. In the host or in the guest? > > >> "tcpdump -i vtnet1 tcp": at each keypress in ssh shell connected to >> 192.168.1.x I see few tcp packet sniffed in guest machine. >> But if I attach vtnet1 to vale switch, tcpdump no longer works as before. >> No tcp traffic is shown. >> Can I perform any other test about this? >> > > First, have you tried to leave vtnet1 alone and try: > > # ifconfig vtnet0 up 192.168.1.x -txcsum -rxcsum -tso4 -tso6 > # vale-ctl -h vale0:vtnet0 > and verified that you can ssh into 192.168.1.x from the host? > > At least you verify that the vale-ctl -h works also on your enivironment. > > Cheers, > Vincenzo > > >> Cheers, >> Antho >> >> Il giorno mer 3 giu 2020 alle ore 19:53 Vincenzo Maffione < >> vmaffione@freebsd.org> ha scritto: >> >>> Hi Anthony, >>> I fixed more bugs in the vtnet driver (in FreeBSD-CURRENT). As of >>> r361760, I'm now able to run the following steps in a VM with a vtnet device >>> >>> # ifconfig vtnet0 -txcsum -rxcsum -tso4 192.168.100.2/24 >>> # vale-ctl -h vale:vtnet0 >>> # netperf -H 192.168.100.1 # Run a netperf TCP_STREAM test to the host >>> bridge interface (br0) >>> >>> Since TCP works correctly at reasonable speed I'm confident that most of >>> the existing problems have gone. >>> Let me know if you have any questions about this. >>> >>> Cheers, >>> Vincenzo >>> >>> Il giorno lun 1 giu 2020 alle ore 18:22 Vincenzo Maffione < >>> vmaffione@freebsd.org> ha scritto: >>> >>>> Hi Anthony, >>>> I think there is more than a bug, drivers-related, that show up when >>>> you attach the interface to a vale switch. >>>> I've found and fixed some related to if_vtnet (see below). In any case, >>>> in my tests there is no difference between TCP traffic and other (UDP, >>>> ICMP, STP,...). >>>> The issues are not related to LRO, as I thought. >>>> There are still more bugs in vtnet and I'm trying to chase them. >>>> In the meanwhile it would help if you apply the patches below and try >>>> again with vtnet to see if the situation improves. They apply cleanly to >>>> 12.1 release. >>>> >>>> Regarding your problem with em devices, it is probably yet a different >>>> issue. It may be related to the iflib transition or not. It would help to >>>> try the same setup on stable/11 (which does not have iflib). I don't have >>>> an em device, but I will try with an emulated em in QEMU/KVM. >>>> >>>> Cheers, >>>> Vincenzo >>>> >>>> ------------------------------------------------------------------------ >>>> r361698 | vmaffione | 2020-06-01 16:14:29 +0000 (Mon, 01 Jun 2020) | 8 >>>> lines >>>> >>>> netmap: if_vtnet: avoid netmap ring wraparound >>>> >>>> netmap assumes the one "slot" is left unused to distinguish >>>> the empty ring and full ring conditions. This assumption was >>>> violated by vtnet_netmap_rxq_populate(). >>>> >>>> MFC after: 1 week >>>> >>>> ------------------------------------------------------------------------ >>>> r361697 | vmaffione | 2020-06-01 16:12:09 +0000 (Mon, 01 Jun 2020) | 8 >>>> lines >>>> >>>> netmap: if_vtnet: replace vtnet_free_used() >>>> >>>> The functionality contained in this function is duplicated, >>>> as it is already available in vtnet_txq_free_mbufs() >>>> and vtnet_rxq_free_mbufs(). >>>> >>>> MFC after: 1 week >>>> >>>> ------------------------------------------------------------------------ >>>> r361696 | vmaffione | 2020-06-01 16:10:44 +0000 (Mon, 01 Jun 2020) | 13 >>>> lines >>>> >>>> netmap: vtnet: fix RX virtqueue initialization bug >>>> >>>> The vtnet_netmap_rxq_populate() function erroneously assumed >>>> that kring->nr_hwcur = 0, i.e. the kring was in the initial >>>> state. However, this is not always the case: for example, >>>> when a vtnet reinit is triggered by some changes in the >>>> interface flags or capenable. >>>> This patch changes the behaviour of vtnet_netmap_kring_refill() >>>> so that it always starts publishing the netmap buffers starting >>>> from the current value of kring->nr_hwcur. >>>> >>>> MFC after: 1 week >>>> ------------------------------------------------------------------------ >>>> Il giorno lun 1 giu 2020 alle ore 15:19 Anthony Arnaud < >>>> antho.arnaudisce@gmail.com> ha scritto: >>>> >>>>> Hi Vincenzo, >>>>> >>>>> To simplify the scenario I have installed from scratch FBSD12.1 on a >>>>> new machine, without any virtualization env. >>>>> I have encountered the same problem, when i attach an ethernet >>>>> interface to vale switch (in this case an intel card em5) the tcp traffic >>>>> disappears and tcpdump shown only udp, icmp6 and stp packets. >>>>> If I detach the NIC from vale0 tcpdump shown all tcp traffic. >>>>> I'm using the netmap version included in FBSD 12.1, and I have >>>>> compiled vale-ctl presents in kernel sources (/src/tools/tools/netmap/) >>>>> I executed your steps. >>>>> There is something dark about that behaviour... >>>>> >>>>> Cheers >>>>> Anthon >>>>> >>>>>