From owner-freebsd-questions@freebsd.org Sat Mar 21 07:48:52 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CC3802597C0 for ; Sat, 21 Mar 2020 07:48:52 +0000 (UTC) (envelope-from tommyhp2@gmail.com) Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48kt7R6T6Fz4BdN for ; Sat, 21 Mar 2020 07:48:51 +0000 (UTC) (envelope-from tommyhp2@gmail.com) Received: by mail-wm1-x333.google.com with SMTP id c187so8753833wme.1 for ; Sat, 21 Mar 2020 00:48:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=1ipCGBV/n/pHhcn8EAX5sQ3aQO048zOofsMTQdAQzDs=; b=PHCLcA8BCtFTo9xjAffOOz92XoWrJf7IXas4V5IP8nZ1gmrxcVPiXU0wz5MrE/SGfw xYZU21O+QSd/z1isLo5bgn6yw/Ti3BPdiJn3rldiIN04qyQSbHBCPSZEho3ClO54ik76 I1nDHI5cIgvbei0rFmWk7zl2w5ovP+/eSe787zMvFOkGsJwIB/p4daaCB3NwkqHO+NCh ARpqwXIUuLpAW0wU4u3OtsAgIZjXC5qGBjA8lFKwM81Le9NOJvlzEcJbl8EQ+J81+DXT FurjMoEcMo01tgOnsDn0ykdVzyYQMjfE9AMrI0QBdoN2ruU/o/WreQMRI1PlCc375gud wbcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=1ipCGBV/n/pHhcn8EAX5sQ3aQO048zOofsMTQdAQzDs=; b=qGhwD4U7PfaQNa9V0SgXiUyKeVZBj7mpDFTI0gWWgLRBoSH0zZq4cqf3ad0LRhU/WP jK9xkaiO5ffapc85roa7JpCeBoVK/PA8Fh9N4cm6NOHT2RAAllAcfUumlT+4JzLDqUU9 EUK6ggfKXvH0O08aYEIuSbXJS63RMaERphzuuI3CVjgOW40eSOeQ7p9Gb2m4cJh1zNmN /EgDLZqiEZGvFd3R0ZJUzw69+ecm/avgGkrwpbMzWUZvoUKOKUDx/UiyS6mPgdzXxxnO wW77KnDGPp0G+rZryIktOMmgVFfK/kgARmBJ3W0cFkJZ04n2CIxoq+prY4NtkmQu78rE Gxrg== X-Gm-Message-State: ANhLgQ1UwM5r4P7evTC3gYGzL/wgjy7U1ZuLwaSwoFWAx2Dh0mDgW8ZH nKSRIt27ZdGWdn5FlGV+Dknl4EpKMO30r9GmCh1QelRx+LA= X-Google-Smtp-Source: ADFU+vv6e75aXrAg6h1CrncF3+EyUHFdyN6WIz+ZB4zaPT3cIlSr0lRTfdCXtSgOKEn1OlnJAdjGvpoAjJvrV/jieOc= X-Received: by 2002:a7b:ce85:: with SMTP id q5mr15600900wmj.83.1584776929901; Sat, 21 Mar 2020 00:48:49 -0700 (PDT) MIME-Version: 1.0 References: <20190205150522.635b9e34.ole@free.de> <20190926141353.55274583.ole@free.de> In-Reply-To: <20190926141353.55274583.ole@free.de> From: Tommy Pham Date: Sat, 21 Mar 2020 00:48:29 -0700 Message-ID: Subject: Re: Bhyve: virtio-net very slow with FreeBSD 12.0 Host To: freebsd-questions@freebsd.org X-Rspamd-Queue-Id: 48kt7R6T6Fz4BdN X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=PHCLcA8B; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of tommyhp2@gmail.com designates 2a00:1450:4864:20::333 as permitted sender) smtp.mailfrom=tommyhp2@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-8.53), ipnet: 2a00:1450::/32(-2.39), asn: 15169(-0.95), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; URIBL_BLOCKED(0.00)[pastebin.com.multi.uribl.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[3.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Mar 2020 07:48:52 -0000 Hi Ole, Is the slow performance in the guest? Are you using UEFI? If guest, have you tried the patches in PR 236922? I don't have bhyve in my environment so I don't know what chipset (legacy i440fx or recent Q35) bhyve can emulate. The man pages and wiki didn't specify. The original work for the patches in that PR was done by Brian Venteicher (@FreeBSd) but was not committed to the official source. I just made it into a patch for my personal use and others also. In reviewing the code to make the patch, I noticed there are some changes to network stack. Also, you post the "pciconf -lBbcevV" output of the guest at pastebin? HTH, Tommy On Thu, Sep 26, 2019 at 5:14 AM Ole wrote: > Hello, > > > I want to push this up again. Cause I ran into it again :-( > > I wonder why no one else has a similar problem. I think my setup is > very common. > > I made some tcpdumps during (very slow) package installation inside > the VM. > > tcpdump -i igb0 | grep pkg > tcpdump -i igb0.4030 | grep pkg > tcpdump -i bridge0 | grep pkg > tcpdump -i tap0 | grep pkg > > the very long output is here https://pastebin.com/vM1VFziJ . For me it > looks like very normal TCP traffic. seqs and acks ... > > thanks > Ole > > Tue, 5 Feb 2019 15:05:22 +0100 - Ole : > > > Hello, > > > > I discovered that virtio-net very slow with FreeBSD 12.0 Host. And > > with very slow I mean 16 kBit/s. > > > > I can't exclude that there is a problem in my setup. But I created two > > very similar setups. One with FreeBSD 11.2 and one with 12.0. The > > server with 11.2 has no problem. > > > > Either I do something really wrong or there is a Bug in the > > virtio-net. > > > > > > Both Servers are connected to a switch, the IPs of the VMs are in the > > vlan 4030. The Bhyve VMs are mangaed by vm-bhyve (master from > > github). > > > > Bridges are created by rc.conf: > > > > # Vlans > > vlans_em0="030" > > ifconfig_em0_4030="mtu 1400 up" > > > > # Bridges > > cloned_interfaces="bridge0" > > ifconfig_bridge0="addm em0.4030 mtu 1400 up" > > > > switch definition: > > > > switch_list="vm-net" > > type_vm-net="manual" > > bridge_vm-net="bridge0" > > > > VM config: > > > > loader="bhyveload" > > cpu=7 > > memory=40G > > network0_type="virtio-net" > > network0_switch="vm-net" > > disk0_type="virtio-blk" > > disk0_name="disk0" > > disk0_dev="sparse-zvol" > > > > > > vm creation: > > > > vm create -t test-host -i FreeBSD-12.0-RELEASE-amd64.raw > > test-example-com vm start test-example-com > > > > Network setup (on the VM): > > > > root@freebsd:~ # ifconfig vtnet0 xx.xx.xx.xx/28 mtu 1400 > > root@freebsd:~ # route add default yy.yy.yy.yy > > > > could someone help me to figure out if this is a Bug, and if yes > > where it is? > > > > Thanks > > Ole > > >