From owner-freebsd-questions@freebsd.org Fri Aug 24 03:54:42 2018 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F2E110A02C6 for ; Fri, 24 Aug 2018 03:54:42 +0000 (UTC) (envelope-from freebsd.ed.lists@sumeritec.com) Received: from mx12-out5.antispamcloud.com (mx12-out5.antispamcloud.com [46.165.232.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0F0475220 for ; Fri, 24 Aug 2018 03:54:41 +0000 (UTC) (envelope-from freebsd.ed.lists@sumeritec.com) Received: from [153.92.8.106] (helo=srv31.niagahoster.com) by mx64.antispamcloud.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1ft39x-000Buu-HC; Fri, 24 Aug 2018 05:54:32 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sumeritec.com; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=bK5xlALm5DQ5GTYipqfI/UypPjRWY3SktHgpEaFnYt4=; b=SjZgy3ua/O9Wqe0Q1yHVAsH6Bb f+odcWnYF/WBnqFsSX7alOy6nHJiYKFQBFTDUJltKA+TaPRKNkwpvfhi9rfwg661RiSUwW9ZD8CHN FSat5FIVxzOdzw01NYVeKlUz3hOxkXk2lf++xHdutMjRH9C/q7R8lHxWVl0Hc9MSCT2befFkTxvbh +JPVycjbIstLmtKeYhmVK1VO6vzaF9wBe4Db5cBZCWIj9AkogSRXpHH54/2C90mhVnkpe/2GmacTu ZMKTxbNHwEt3ryPiyE7a+WpNoSePMh6v+pW6zG8o2OvR4L+LJC4mtiSgbwsZtm6L8NIkfqKYFzUYC +1ah8hXA==; Received: from [114.125.117.144] (port=14278 helo=X220.sumeritec.com) by srv31.niagahoster.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1ft38y-0003hH-5J; Fri, 24 Aug 2018 10:52:27 +0700 Date: Fri, 24 Aug 2018 11:52:14 +0800 From: Erich Dollansky To: Norman Gray Cc: FreeBSD Questions Subject: Re: Jails and networks Message-ID: <20180824115214.775c7464.freebsd.ed.lists@sumeritec.com> In-Reply-To: <6B17F10B-F3AE-45C5-8011-EBE52462230E@glasgow.ac.uk> References: <6B17F10B-F3AE-45C5-8011-EBE52462230E@glasgow.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-OutGoing-Spam-Status: No, score=-1.0 X-AuthUser: freebsd.ed.lists@sumeritec.com X-Originating-IP: 153.92.8.106 X-AntiSpamCloud-Domain: out.niagahoster.com X-AntiSpamCloud-Username: niaga Authentication-Results: antispamcloud.com; auth=pass (login) smtp.auth=niaga@out.niagahoster.com X-AntiSpamCloud-Outgoing-Class: ham X-AntiSpamCloud-Outgoing-Evidence: Combined (0.02) X-Recommended-Action: accept X-Filter-ID: EX5BVjFpneJeBchSMxfU5pavPtunc8Yj7lDeGWDtm0V602E9L7XzfQH6nu9C/Fh9KJzpNe6xgvOx q3u0UDjvOztukqweARitNSGTKzpDBfjZBtoFvjsKifOYu1ky/KMdmofmyFAiuF3KR7qyhMVStbt1 zPu5jyI5c1RdFy9oIkdMc1rc4c8gTdWHq8LmAzmxL3cGolkJqqMaoMXQLKIIaBI7+ctq55o3paXd 8fRhMeWV9mua2H42BgYi1KVTzBqaGM0/FQln02Joy/riMeAD717AhhPERPMiMEDgMLQ4ddEif6RK Y0jj73EtCvgIX8qELIa3UlBgamA9W9zuC4CANl39Ievnxxzd0BmBo43y1J2ncCFok92qPywea9RH NTLhb7xDqKYA4dWPeY3eejJf/kYLncxlFHqxAFR/7dEYjupTIr01Gcd5CvDMwJDeLpgcA7jjczEW i9yDCXHnlZrjGPhkOums4qAZ7fpXkOz/zyWuW5w6NuAxrbgdn+3FfbYnxySwKIPsYaBNUjbYktP7 HNOt8DSD/OhtTFTFIhsSLFsX270rIFeP4U0tTVK8nHn/2OUMeHyTpNN0eXybX/w7//sEFHS1KSfM kBZeDeLwWzxuXtcwcgyS54fDBel47YuQKKaT6y5i4LBNJV8bcQPZvurIrLmzq1cOLf+F+d8z176W eCLfSqgdhc7QeZLuiuMzRsHjMpUVGL2it/VpKpvQn5u3Hj9nzHFECLEV8EOCh/sqVOoQzwrCXZje TAZGYgNwO/IbvbY6zT46i9eJaYEqLX4joiM+oevRtVRATbWaFjo8CVsONrMJuGzuoGnKTKcy1eyD 7JcQilYK02SYmSiV1ucl21y2TwmMxJz+gU1wKb+BBkENQYujqKl8rFj50trSQA8PC5NBesxO/gc2 I+P8qev5+xNLX7J80SrUE4vW6MWZ0dEqwY4lelob0ErttooptwERZXSSDZL7eS4eAbBXy8hQdO02 pRv/nbK/cXX+6jRr9/Mn670rgEzRrsP4YRqdx41tn/0JOts9UlqtbDvD3A== X-Report-Abuse-To: spam@quarantine1.antispamcloud.com X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Aug 2018 03:54:42 -0000 Hi, I did not go through your e-mail. Just take my working settings to start with: In /etc/jails.conf Name { path = "/usr/home/whateverexists"; ip4.addr = 192.168.x.y; host.hostname = "jail.example.com"; allow.raw_sockets = 1; interface = yournetworkinterface; exec.start = "sh /etc/rc"; exec.stop = "sh /etc/rc.shutdown"; mount.devfs; } In /etc/rc.conf inside the jail: inetd_enable="YES" inetd_flags="-wW -a 192.168.x.y" dbus_enable="YES" hald_enable="YES" runlocalproxy_enable="YES" sshd_enable="YES" You can then start the jail with jail -c Name You can then add more of your settings until you have found the culprit. Erich On Thu, 23 Aug 2018 19:44:57 +0100 Norman Gray wrote: > Greetings. > > I'm having difficulty creating a jail which is able to see the outside > world. The various recipes I've found seem to be subtly > contradictory: I'm trying to understand what they're doing rather > than dumbly following them, and my lack of success here is telling me > that my mental model of jails+networking doesn't quite match > reality. I think I'm on the verge of a very educational > experience.... > > I'm using ezjail, on 11.2. > > Sources: > > * The manual [1] describes basic usage, but mentions release 9.3; I > get the impression that ezjail's procedure for starting and > configuring jails (using /etc/jail.conf rather than the old 4 > arguments) is slightly but significantly incompatible with 11.2. > > * The ezjail documentation [2] describes setting up a jail using > em0|10.0.0.2, very straightforwardly > > * A forum post [3] describes setting up a jail using ezjail and pf. > Now, I don't think I need pf in my situation, so I want to skip that > part of the instructions. But I now suspect I'm doing so naively. > > * Another forum post [4] describes setting up both a VIMAGE and a > non-VIMAGE jail, and is usefully explicit about the contents of the > /etc/jail.conf file. This is the one I've been following most > closely, but I realise that I don't understand why it configures a > bridge interface, but adds only a single real interface igb0 to it > (my model of a bridge interface is that it necessarily involves two > interfaces, or does the igb0 in the host and the one in the client > count as two?). > > My host is on a 172.16.0.0/12 private network, which is routable > locally, though it has to use a proxy to get to the web. I want to > set up a jail on (slightly at random) 192.168.11.128. > > I have: > > * net.inet.ip.forwarding: 1 > * igb0 configured with the correct IP address and mask, not aliased > at all > * I've created lo1 > > My /etc/jail.conf looks like > > exec.start = "/bin/sh /etc/rc"; > exec.stop = "/bin/sh /etc/rc.shutdown"; > exec.clean; > > path = "/local/jails/$name"; > > mount.fstab = "/etc/jail/fstab.${name}"; > mount.devfs; > mount.fdescfs; > mount.procfs; > > host.hostname = "${name}.local"; > > devfs_ruleset = "4"; > > norman { > # test jail > ip4.addr = "192.168.11.128"; > interface = "igb0"; > } > > and the non-comment lines in /usr/local/etc/ezjail.conf look like > > ezjail_jaildir=/local/jails > ezjail_ftphost=http://ftp.uk.freebsd.org > ezjail_use_zfs="YES" > ezjail_use_zfs_for_jails="YES" > ezjail_jailzfs=zroot/local/jails > > I've created a ezjail flavour called 'norman' (with the inevitable > solipsism). > > My _understanding_ is that this sets the jail to use the igb0 > interface in the host (a non-VIMAGE jail doesn't have a separate > networking stack). > > I create the jail > > ezjail-admin create -f norman -c zfs norman > 'lo1|127.0.1.1,igb0|192.168.11.128' > > lo1 first, as suggested in [1]. My impression is that that sets up > the loopback interface within the jail to be an alias of lo0 in the > host, and attaches 192.168.11.128 to igb0 in the jail. > > Then I start the jail > > jail -c norman > > it starts up sshd promptly, but takes a long time (presumably timing > out in fact) to start sendmail_submit and sendmail_msp_queue. Then > > jexec 4 /bin/sh > > lets me see > > # cat /etc/resolv.conf > search physics.gla.ac.uk > nameserver 130.209.4.16 > nameserver 130.209.4.18 > # ifconfig igb0 > igb0: flags=8843 metric 0 mtu > 1500 > options=6403bb > ether a4:bf:01:26:7d:b1 > hwaddr a4:bf:01:26:7d:b1 > inet 192.168.11.128 netmask 0xffffffff broadcast 192.168.11.128 > media: Ethernet autoselect (1000baseT ) > status: active > > ...which looks right. But > > # host www.gla.ac.uk > ;; connection timed out; no servers could be reached > # > > The routing table is very simple: > > # netstat -rn > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > 192.168.11.128 link#3 UHS lo0 > > > I don't think I've done anything at all exotic here, and the > resolv.conf contents and ifconfig output looks as I'd expect. The > routing table doesn't have a default route, but (a) if this interface > is just the same as the same-named one in the host, so ... *mumble*; > and (b) the various recipes I've quoted don't anywhere mention having > to add a default route, so I don't think that can be what I'm missing. > > I'm wondering if there's something to do with the private network the > host is on. But that can talk to the network without difficulty, and > in any case http_proxy is correctly set in the jail. > > I've seen a mention of epair(4), but I don't think that's relevant. > > So I'm clearly misunderstanding something terribly important (and > embarrassingly obvious in retrospect), which hasn't magically become > clear by my explaining the steps clearly to myself here. I suspect I > don't _actually_ understand the relationship between the jail's > interfaces and the host's -- they seem the same but not the same in > some very uncomfortable way. > > Any epiphanies gratefully received. > > Best wishes, > > Norman > > > > [1] https://www.freebsd.org/doc/handbook/jails-ezjail.html > [2] https://erdgeist.org/arts/software/ezjail/ > [3] https://forums.freebsd.org/threads/30063/ > [4] https://forums.freebsd.org/threads/49561/ > > -- > Norman Gray : https://nxg.me.uk > SUPA School of Physics and Astronomy, University of Glasgow, UK > > [University of Glasgow: The Times Scottish University of the Year > 2018] _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "freebsd-questions-unsubscribe@freebsd.org"