From owner-freebsd-jail@freebsd.org Tue Feb 5 17:58:54 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5906D14C2BB9 for ; Tue, 5 Feb 2019 17:58:54 +0000 (UTC) (envelope-from srs0=z7dd=qm=vega.codepro.be=kp@codepro.be) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB222868BB for ; Tue, 5 Feb 2019 17:58:53 +0000 (UTC) (envelope-from srs0=z7dd=qm=vega.codepro.be=kp@codepro.be) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id CB9747DED; Tue, 5 Feb 2019 18:58:45 +0100 (CET) Received: by vega.codepro.be (Postfix, from userid 1001) id C4F5780331; Tue, 5 Feb 2019 18:58:45 +0100 (CET) Date: Tue, 5 Feb 2019 18:58:45 +0100 From: Kristof Provost To: Michael Grimm Cc: Farhan Khan , freebsd-jail@freebsd.org Subject: Re: vnet NAT'd jails extremely slow, connection dies Message-ID: <20190205175845.GA86328@vega.codepro.be> References: <2B6B82BC-1105-4D3B-AD6C-E74109A76113@ellael.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <2B6B82BC-1105-4D3B-AD6C-E74109A76113@ellael.org> X-Checked-By-NSA: Probably User-Agent: Mutt/1.11.2 (2019-01-07) X-Rspamd-Queue-Id: EB222868BB X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Feb 2019 17:58:54 -0000 On 2019-02-05 18:47:23 (+0100), Michael Grimm wrote: > Farhan Khan wrote: > > On Mon, Feb 4, 2019 at 2:29 PM Farhan Khan wrote: > > >> I have a jail NAT'd to a base system, but the connection is extremely > >> slow and frequently disconnects drops, whereas the base is fine has > >> perfectly fine connectivity. > >> > >> My configuration is as follows: > >> vtnet0: Has routeable IPv4 address and 172.16.0.1/16 > >> Jail uses epair4b, base has epair4a. Jail's IP is 172.16.0.5/16. > >> The base and jail can ping each other. > >> bridge0: contains vtnet0 and epair4a. > >> > >> I have gateway_enable="YES" > >> My pf.conf is as follows: > >> nat pass from 172.16.0.0/16 to any -> (vtnet0) > >> > >> When I try to run clamav, the connectivity stalls after a few minutes > >> and eventually disconnects. I ran tcpdump on the bridge and saw a lot > >> of HTTP seq and ack packets but no actual data. I am not using IPv6 > >> yet. > > > > Just to provide more context to my previous email, outside of the jail > > I can download the FreeBSD ISO installer image at 3 MBps. Within the > > jail it drops to 12KBps. > > This sounds familiar to me ;-) > > Please have a look at https://lists.freebsd.org/pipermail/freebsd-net/2017-December/049470.html > Solution in https://lists.freebsd.org/pipermail/freebsd-net/2017-December/049484.html > > I ended up with the following additions to /boot/loader.conf (and a subsequent reboot): > > # needs to become turned off (LRO) in order to restore tcp performance within VNET jails: > hw.vtnet.lro_disable="1" > hw.vtnet.tso_disable="1" > Farhan has also solved his issue by turning off lro/tso. (We talked on IRC). I've not seen this issue myself, but I'm interested in a couple of points to hopefully pinpoint and maybe even fix the problem. These are questions for anyone who's running pf on top of a hypervisor and has vnet or other jails, and has seen slowdowns. * What hypervisor are you running? * Does the problem affect only the jails, or also the host system? * Does it only happen with NAT, or with routed packets as well? If anyone is affected and not using pf that'd be interesting information as well. Regards, Kristof