From owner-freebsd-jail@freebsd.org Wed Feb 6 21:01:07 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C21814DE607 for ; Wed, 6 Feb 2019 21:01:07 +0000 (UTC) (envelope-from khanzf@gmail.com) Received: from mail-lf1-x144.google.com (mail-lf1-x144.google.com [IPv6:2a00:1450:4864:20::144]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5933F85D0E; Wed, 6 Feb 2019 21:01:05 +0000 (UTC) (envelope-from khanzf@gmail.com) Received: by mail-lf1-x144.google.com with SMTP id a16so6500434lfg.3; Wed, 06 Feb 2019 13:01:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=u0kfS0geRNg4L8HzQQ+eJcojeMhwq6UJvPhZy8SQKkE=; b=DfXejZdBtV3NBbfluq4bEYZww0JCIKjhTckl/fQvRXSnjH6d8d7i2JUPLXNQeeO/Nl 3Y//T0qLBnpksEAk2buU1ElUpFUd+tYrZMopfwM3ZaIUo6Y5blOnnEYh9EiZ9jaTLIpS wz7iCavCihsuHTGNh3nEdhNB0Pz4StEWyRhWYp8Yco1bDeCjqec25DLGp1tifXITpTVs eLvwomglbNfALMiXuKVWDUub5vB9DVg0SQkdk5EcO/QYoUMQPlgACb4s1eoAyaOVayPQ PDS1HCp0RPCI4o4z/5xlXcQ37uKmPS9DqZDvZ8/5U+fcGbf9nFhWLqB/Fef/PmMHBYq2 1V3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=u0kfS0geRNg4L8HzQQ+eJcojeMhwq6UJvPhZy8SQKkE=; b=iWT649IXwhTK/JCGGEUu3hlG1l7Gvbfcf1Hf428qHsjtmexcdiie8gsHcPyREalcaA C3iWYOIKWhtbfA1IvPcjn3q71UeT446TdHU5SlMg+JbIzWw8tM+b/HqqNRsF2iTGrMMB BLZtIe7vMyrR/9lzfGhaI9+4NwufeeFIxnDUyQZ5uSVqnv5kuwE5pDVWLEqeXyv6AWGk 7VOLAuhWW+XXk3nkVwPg8AoIEa2ljBYBiqMgpe+JGX5J1HcBBlUjEUhTYn01Qk199+2n D9+FciuH1P+aEM/MaQZ9MWI2f6FPZ/vPpL2s4y0Le7Z7sCmz0SKFB7Sdsu2NpUmdXsEr 52MQ== X-Gm-Message-State: AHQUAuZVepPu8XLRLJuWm737w6L9iBJGoUzvv1wrZXk6/qyI7xiaB/Dp i0r/yKtmYtU/dwlAL8UYIqsR8yAxHOcipsixxTisWw== X-Google-Smtp-Source: AHgI3IYsjIJ5TqLfLrTL3z7rrZPqV90tecKosVnu5jb/6a4QlO5gtEHB+42UbKVtHZGKiWY7bDFpoUZHbVcGLVyzTus= X-Received: by 2002:ac2:42d2:: with SMTP id n18mr7483964lfl.37.1549486862617; Wed, 06 Feb 2019 13:01:02 -0800 (PST) MIME-Version: 1.0 References: <2B6B82BC-1105-4D3B-AD6C-E74109A76113@ellael.org> <20190205175845.GA86328@vega.codepro.be> In-Reply-To: <20190205175845.GA86328@vega.codepro.be> From: Farhan Khan Date: Wed, 6 Feb 2019 16:00:50 -0500 Message-ID: Subject: Re: vnet NAT'd jails extremely slow, connection dies To: Kristof Provost Cc: Michael Grimm , freebsd-jail@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 5933F85D0E X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=DfXejZdB; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of khanzf@gmail.com designates 2a00:1450:4864:20::144 as permitted sender) smtp.mailfrom=khanzf@gmail.com X-Spamd-Result: default: False [-2.70 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.979,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.39)[0.386,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; IP_SCORE(-0.10)[ip: (3.85), ipnet: 2a00:1450::/32(-2.30), asn: 15169(-1.97), country: US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Feb 2019 21:01:07 -0000 On Tue, Feb 5, 2019 at 12:58 PM Kristof Provost wrote: > > On 2019-02-05 18:47:23 (+0100), Michael Grimm wrote: > > Farhan Khan wrote: > > > On Mon, Feb 4, 2019 at 2:29 PM Farhan Khan wrote: > > > > >> I have a jail NAT'd to a base system, but the connection is extremely > > >> slow and frequently disconnects drops, whereas the base is fine has > > >> perfectly fine connectivity. > > >> > > >> My configuration is as follows: > > >> vtnet0: Has routeable IPv4 address and 172.16.0.1/16 > > >> Jail uses epair4b, base has epair4a. Jail's IP is 172.16.0.5/16. > > >> The base and jail can ping each other. > > >> bridge0: contains vtnet0 and epair4a. > > >> > > >> I have gateway_enable="YES" > > >> My pf.conf is as follows: > > >> nat pass from 172.16.0.0/16 to any -> (vtnet0) > > >> > > >> When I try to run clamav, the connectivity stalls after a few minutes > > >> and eventually disconnects. I ran tcpdump on the bridge and saw a lot > > >> of HTTP seq and ack packets but no actual data. I am not using IPv6 > > >> yet. > > > > > > Just to provide more context to my previous email, outside of the jail > > > I can download the FreeBSD ISO installer image at 3 MBps. Within the > > > jail it drops to 12KBps. > > > > This sounds familiar to me ;-) > > > > Please have a look at https://lists.freebsd.org/pipermail/freebsd-net/2017-December/049470.html > > Solution in https://lists.freebsd.org/pipermail/freebsd-net/2017-December/049484.html > > > > I ended up with the following additions to /boot/loader.conf (and a subsequent reboot): > > > > # needs to become turned off (LRO) in order to restore tcp performance within VNET jails: > > hw.vtnet.lro_disable="1" > > hw.vtnet.tso_disable="1" > > > Farhan has also solved his issue by turning off lro/tso. (We talked on > IRC). > > I've not seen this issue myself, but I'm interested in a couple of > points to hopefully pinpoint and maybe even fix the problem. > > These are questions for anyone who's running pf on top of a hypervisor > and has vnet or other jails, and has seen slowdowns. > > * What hypervisor are you running? > * Does the problem affect only the jails, or also the host system? > * Does it only happen with NAT, or with routed packets as well? > > If anyone is affected and not using pf that'd be interesting information > as well. > > Regards, > Kristof Michael, thank you very much. This appears to do the trick, as Kristof also directed me. A. This was on a Vultr instance. Per they are using KVM, per a support ticket. B. Just the Jail, not the post C. I will have to get back to you on that, as I do not have a publicly routeable IP to test on at the moment.