From owner-freebsd-hackers@freebsd.org Sat Apr 24 03:09:23 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A7F6C5EF07D for ; Sat, 24 Apr 2021 03:09:23 +0000 (UTC) (envelope-from dewayne@heuristicsystems.com.au) Received: from hermes.heuristicsystems.com.au (hermes.heuristicsystems.com.au [203.41.22.115]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2560 bits) client-digest SHA256) (Client CN "hermes.heuristicsystems.com.au", Issuer "Heuristic Systems Type 4 Host CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FRx2n6rPhz4Qv5 for ; Sat, 24 Apr 2021 03:09:21 +0000 (UTC) (envelope-from dewayne@heuristicsystems.com.au) Received: from [10.0.5.3] (noddy.hs [10.0.5.3]) (authenticated bits=0) by hermes.heuristicsystems.com.au (8.15.2/8.15.2) with ESMTPSA id 13O37q0m027934 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Sat, 24 Apr 2021 13:07:53 +1000 (AEST) (envelope-from dewayne@heuristicsystems.com.au) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=heuristicsystems.com.au; s=hsa; t=1619233673; x=1619838474; bh=TqsUvVZcONzkixxg1ylcjXUwSqHr/vc4T5Vz11tMz84=; h=Subject:To:From:Message-ID:Date; b=Ok1sj7eL8QpilGWBWg2batG5Es4Q2SdJb2P7aM5ks/KlWVa893c5SnKgtpMlybQiF LZ4AfoHTHCqDSGjAEWE+nWFoOkK/EPFGdU02ywfg1nwEPfKY/4+MaBmywwwQ8n7bSH gxUsO7s1ynX76o+e81NQ8SEyWnfnFiaAJUTbu+axsM0zxefxnY0qX X-Authentication-Warning: b3.hs: Host noddy.hs [10.0.5.3] claimed to be [10.0.5.3] Subject: Re: 32-bit jail on 64-bit host To: freebsd-hackers@freebsd.org References: From: Dewayne Geraghty Message-ID: <05b397f9-5bfc-1c19-bf8c-9429b7998113@heuristicsystems.com.au> Date: Sat, 24 Apr 2021 13:06:35 +1000 User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FRx2n6rPhz4Qv5 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=heuristicsystems.com.au header.s=hsa header.b=Ok1sj7eL; dmarc=none; spf=pass (mx1.freebsd.org: domain of dewayne@heuristicsystems.com.au designates 203.41.22.115 as permitted sender) smtp.mailfrom=dewayne@heuristicsystems.com.au X-Spamd-Result: default: False [-6.20 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_XAW(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCVD_IN_DNSWL_MED(-0.20)[203.41.22.115:from]; DKIM_TRACE(0.00)[heuristicsystems.com.au:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1221, ipnet:203.40.0.0/13, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[heuristicsystems.com.au:s=hsa]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_MED(-2.00)[heuristicsystems.com.au:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[heuristicsystems.com.au]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-Mailman-Approved-At: Sat, 24 Apr 2021 08:36:24 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Apr 2021 03:09:23 -0000 On 23/04/2021 1:12 am, Chris wrote: > I do it. I don't think vnet or a bridge is necessary or perhaps even > desirable > in this situation. All my 32/64bit jails access the hosts net out of > localhost > (127.0.0.2-N) and I use pf(4) to redirect the packets. > I have a static block of internet facing addresses. So change yours > accordingly > pf.conf(5) > EXT_ADDR="W.X.Y.Z" > ... > set skip on { lo0, lo1 } > ... > nat pass on re0 from { lo1 } to any -> $EXT_ADDR > rdr pass on re0 proto tcp from any to { lo1 } -> $EXT_ADDR > ... > block in > pass out > ... > I add an entry in the hosts hosts(5) file, and in the jails hosts(5) for > accounting purposes. The jails resolve.conf(5) file looks like this > nameserver 127.0.0.1 > nameserver 127.0.0.2 > options timeout:1 attempts:1 rotate > > And all gets it done for me. > > HTH > > --Chris >> >> Peter Peter, I use a similar setup to Chris, though with ipfw. ;) Jails have a few subtleties. They inherit much of the network of the base. So you only need to think about the IP's assigned to the jail and their assignment order. However one particular gotcha The jail will use the first IP address that's set in jail.conf effectively becoming your default route for the jail. And I recall that localhost will also latch onto that IP address, so if its internet facing, you'll need to think about the implications. I'm a little paranoid so I use: - /etc/hosts to define localhost to be something other than the default. Some applications/ports behave properly IF they use localhost for their unix sockets, rather than 127.0.0.1. (ie test what you need and become good friends with tcpdump) - consider carefully your firewall rules not just internet facing but also over lo0 :) And to reiterate what many have said, running i386 and amd64 on an amd64 platform is fun, as there are less machines to maintain when you need, as in our use-case, to test the operation of software for 32bit targets. (Though we just perform a buildworld with TARGET_ARCH=i386 CPU_TYPE=PRESCOTT with the appropriate destination.) I don't think your setup requires the complexity or additional processing from bridging or vnets.