From owner-freebsd-jail@freebsd.org Sun Dec 11 15:05:07 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F18F0C721CB for ; Sun, 11 Dec 2016 15:05:07 +0000 (UTC) (envelope-from kayasaman@gmail.com) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 85A7A18BE for ; Sun, 11 Dec 2016 15:05:07 +0000 (UTC) (envelope-from kayasaman@gmail.com) Received: by mail-wm0-x231.google.com with SMTP id a197so28598194wmd.0 for ; Sun, 11 Dec 2016 07:05:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=7ndckuHq+08KBMotPHAn8NaEpHT2MBZ6jbsBjNXMrhU=; b=PuNoIpyy/+ugc0Si7srdeukOWb3RfHug9BuX3YSXg92Pz2ZU4qjEiRaxRXeFL8waJh 7K2Z8wQcAe0JWZ0YHNrIwyY7acWAoToqc3Tu98v9YukkrdC+jysUMashIFVgD9C1vRUB VXneb/M1cQVq/yJRl3bQ/Z/dJA9lmFIJxBN2pTxo1OiAQdaC8iD0UH1yYJFSya+yfRmV nmg6Lo4vAc9XMNFwx1Mc3dv+1oY0syBt3QVJSz+U0X9c1FHmmCZGACH2G4JyUa0mE+sw xPnDGrRKCLihNsrGbKTGraVv9wWtcIOmbdgHT48o9RGbOe0mjF0XK3xz18WfNTXyrf0l z9SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=7ndckuHq+08KBMotPHAn8NaEpHT2MBZ6jbsBjNXMrhU=; b=JfszayAwPwN/tXW9MSQliLgRQxl0wM37xXHxoM2nSaHHaDeNmm4+PWMmxWlsjMCA3v DcGLgu/OPT3fiwXmBHGSlq4qBE4cTLlMsDGgqypKDBIy4+uKbiM4TRdQU7hOSJQX7tgA NeenZOHECe4f3p3xnC8S8XRC4EAxBnCrNZ7FmShdxC/5r+u6JR65S4aTjXtW3wBYxc1J hX1bjjhw8PI+K1G1fUcrIeI7Etooi4ige0xGnr3ez+2YjsGtNC4vsIii3opXEFmloPaU L82Wv3LNCqyL7OcTPfdTAergfiX+XoYID2bTXzKNyCj14IaCto3I63aa2SPifru6LLZ5 JiFg== X-Gm-Message-State: AKaTC01aeix28w1Crs56y0uv+drVnYG2KKYPXrG4PIccmZo+3IEV7I2x9maeu0SMkN8TqA== X-Received: by 10.28.22.193 with SMTP id 184mr14641627wmw.100.1481468704768; Sun, 11 Dec 2016 07:05:04 -0800 (PST) Received: from x220.optiplex-networks.com (optiplexnetworks.plus.com. [212.159.80.17]) by smtp.googlemail.com with ESMTPSA id ei2sm52543810wjd.47.2016.12.11.07.05.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Dec 2016 07:05:04 -0800 (PST) Subject: Re: Getting "Permission Denied" issues after migrating jails To: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-jail@freebsd.org References: <584D6A13.9010502@quip.cz> From: Kaya Saman Message-ID: <78892b93-0a7d-b84a-6599-ce094735c9eb@gmail.com> Date: Sun, 11 Dec 2016 15:05:03 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <584D6A13.9010502@quip.cz> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 15:05:08 -0000 Thanks a lot Miroslav :-) I found the issue eventually..... For some reason it turns out the /var directory got set to 700 permissions after the rsync..... a simple chmod 755 /var fixed the issue, however, it looks like all 5 of my jails had the same problem? on vuln.xml the permissions were fine 444. Well took till 4am to sort out, I ended up rebuilding all my ports too just to be safe of sym link and permission issues. Regards, Kaya On 12/11/2016 03:00 PM, Miroslav Lachman wrote: > Kaya Saman wrote on 2016/12/10 15:33: > >> which suggests fixing the noexec flags. On the actual ZFS dataset the >> exec=on parameter is already set meaning that this must be a local issue >> and something to do with the "chflags" command but I can't recall or >> even find any clue on which files to run the command on and parameters >> to use in "man chflags". > > Run ls -lo /var/db/pkg/vuln.xml to view permissions. > > You can use something like this to check all files with specified flag > > find /path/to/jail/ -flags +schg -exec ls -lo {} + > > Then check what is your kern_securelevel settings in host and in a > jails rc.conf. You cannot modify files witch flags is securelevel is > higher than 0. > > Miroslav Lachman > From owner-freebsd-jail@freebsd.org Sun Dec 11 15:07:21 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A60CAC722E1 for ; Sun, 11 Dec 2016 15:07:21 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6BD991941 for ; Sun, 11 Dec 2016 15:07:20 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 164B228483; Sun, 11 Dec 2016 16:00:37 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 5D2E828482; Sun, 11 Dec 2016 16:00:36 +0100 (CET) Subject: Re: Getting "Permission Denied" issues after migrating jails To: Kaya Saman , freebsd-jail@freebsd.org References: From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <584D6A13.9010502@quip.cz> Date: Sun, 11 Dec 2016 16:00:35 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 15:07:21 -0000 Kaya Saman wrote on 2016/12/10 15:33: > which suggests fixing the noexec flags. On the actual ZFS dataset the > exec=on parameter is already set meaning that this must be a local issue > and something to do with the "chflags" command but I can't recall or > even find any clue on which files to run the command on and parameters > to use in "man chflags". Run ls -lo /var/db/pkg/vuln.xml to view permissions. You can use something like this to check all files with specified flag find /path/to/jail/ -flags +schg -exec ls -lo {} + Then check what is your kern_securelevel settings in host and in a jails rc.conf. You cannot modify files witch flags is securelevel is higher than 0. Miroslav Lachman From owner-freebsd-jail@freebsd.org Sun Dec 11 15:22:50 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A1CCC72703 for ; Sun, 11 Dec 2016 15:22:50 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C4CA9A6 for ; Sun, 11 Dec 2016 15:22:49 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id E73AC2847F; Sun, 11 Dec 2016 16:22:47 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 484162847B; Sun, 11 Dec 2016 16:22:47 +0100 (CET) Subject: Re: Getting "Permission Denied" issues after migrating jails To: Kaya Saman , freebsd-jail@freebsd.org References: <584D6A13.9010502@quip.cz> <78892b93-0a7d-b84a-6599-ce094735c9eb@gmail.com> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <584D6F47.8010106@quip.cz> Date: Sun, 11 Dec 2016 16:22:47 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: <78892b93-0a7d-b84a-6599-ce094735c9eb@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 15:22:50 -0000 rKaya Saman wrote on 2016/12/11 16:05: > Thanks a lot Miroslav :-) > > I found the issue eventually..... > > For some reason it turns out the /var directory got set to 700 > permissions after the rsync..... > > a simple chmod 755 /var fixed the issue, however, it looks like all 5 of > my jails had the same problem? rsync -avvcrt : you don't need to specify "r" and "t", they are included in "a" but you missed "H" for hardlinks (they are used for base and packages too) -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) I don't know why your /var had 0700, maybe you have too restrictive umask for user under you were running rsync. Miroslav Lachman From owner-freebsd-jail@freebsd.org Mon Dec 12 13:40:14 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B7F3C73CAD for ; Mon, 12 Dec 2016 13:40:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B545167C for ; Mon, 12 Dec 2016 13:40:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBCDeDqB067522 for ; Mon, 12 Dec 2016 13:40:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-jail@FreeBSD.org Subject: [Bug 215250] jail break under particular circumstance Date: Mon, 12 Dec 2016 13:40:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-jail@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 13:40:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215250 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-jail@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-jail@freebsd.org Mon Dec 12 14:40:51 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E495FC7308D for ; Mon, 12 Dec 2016 14:40:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD8051618 for ; Mon, 12 Dec 2016 14:40:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBCEepBa042726 for ; Mon, 12 Dec 2016 14:40:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-jail@FreeBSD.org Subject: [Bug 215250] jail break under particular circumstance Date: Mon, 12 Dec 2016 14:40:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jamie@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Works As Intended X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-jail@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 14:40:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215250 Jamie Gritton changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jamie@FreeBSD.org Resolution|--- |Works As Intended Status|New |Closed --- Comment #2 from Jamie Gritton --- Yes, it's expected behavior. It's not so much a "break" as being pulled ou= t of the jail by an administrator with proper permission who presumably knows wh= at he's doing. Preventing an assisted break like this would be doable, but would involve either tracing all .. traversals back to at least a prison root, or attachi= ng a prison reference to every directory in the vnode cache. Both of those seem= to be a bit of overkill. I have to admin I've done the very thing in the example: temporarily moving /usr/ports to a jail. Lately I've gone with nullfs instead, which doesn't = open this hole. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-jail@freebsd.org Mon Dec 12 17:14:05 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0E2FC73F90 for ; Mon, 12 Dec 2016 17:14:05 +0000 (UTC) (envelope-from fbstable@cps-intl.org) Received: from berkeley.cps-intl.org (websense.cps-intl.org [81.137.176.89]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ADB4E759 for ; Mon, 12 Dec 2016 17:14:05 +0000 (UTC) (envelope-from fbstable@cps-intl.org) Received: from [172.16.0.79] (helo=bdLL65j) by berkeley.cps-intl.org with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1cGUAm-000CFS-E3; Mon, 12 Dec 2016 17:13:56 +0000 To: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-jail , Alexander Leidinger References: <584986D0.3040109@quip.cz> <2b6346f8-ed02-0e6d-bd89-106098e7eb2d@cps-intl.org> <58499446.3050403@quip.cz> <5849C5BF.7020005@quip.cz> <584A9179.9060508@quip.cz> <584A9D89.4040003@quip.cz> <3851c5d9-7646-b670-357e-ae937fcc7e8f@cps-intl.org> <584AB345.4080307@quip.cz> From: SK Message-ID: <33473585-3cb9-10d3-acf9-0a917c5a0079@cps-intl.org> Date: Mon, 12 Dec 2016 17:13:27 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <584AB345.4080307@quip.cz> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 172.16.0.79 X-SA-Exim-Mail-From: fbstable@cps-intl.org X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on berkeley.lan.cps-intl.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=10.0 tests=ALL_TRUSTED autolearn=ham autolearn_force=no version=3.4.0 Subject: Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial] X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on berkeley.cps-intl.org) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 17:14:06 -0000 On 09/12/2016 13:36, Miroslav Lachman wrote: > > My last idea - put zfs_enable="YES" in jails /etc/rc.conf. > > Maybe the dataset is not mounted if has property jailed=on (I don't > know I didn't test it yet) Good evening Miroslav, good evening Alexander Thank you both for your support in this matter. I have completed (I think) my tests with the test box and have concluded as following a) Miroslav, you were correct, I could only see from the root of the dataset to the dataset itself, all other dataset that are not part of this branch is invisible from within the jail. This serves my purpose, so I am content (to some extent). The explanation about enforce_statfs was really helpful -- I think that was one thing I was missing (cannot confirm, but I believe that is what the error was on my part) b) Alexander, I am still not able to do snapshot or any other action from within my jail. My understanding is that you are using ezjail, which might be doing something that my regular jail creation is ommitting. If you do not mind sharing your configuration steps, I can try to reproduce it at this end. If it is exactly as it is on the site you pointed to earlier, please let me know, I will follow that verbatim (even though I do not remember seeing anything there that I have not tried already, but I might be mistaken). And now to everyone, I am still confused about zfs set jailed=on. As I mentioned on my previous emails, as soon as I do that, the dataset vanishes from the host system (as I understand, that is expected behaviour). Then the jail fails as it is unable to mount /dev, /proc and so on. I have to change jail.conf and comment out mount.devfs and mount.procfs -- but than in turn makes /dev/zfs unavaulable and I cannot do anything from inside the jail. I do not need it now, given that I am happy with the current situaion, but am curious to know how that zfs parameter works and how I can make it work, hence "solved" is "partial" in the subject line. Thanks to you both for your continuous support and suggestions, it is very much apprecaited. Best regards SK From owner-freebsd-jail@freebsd.org Mon Dec 12 23:04:00 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8CD5C74D1E for ; Mon, 12 Dec 2016 23:04:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D820BEDC for ; Mon, 12 Dec 2016 23:04:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBCN40xN095333 for ; Mon, 12 Dec 2016 23:04:00 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-jail@FreeBSD.org Subject: [Bug 215250] jail break under particular circumstance Date: Mon, 12 Dec 2016 23:04:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: simonp@email.it X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Works As Intended X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-jail@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 23:04:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215250 --- Comment #3 from simonp@email.it --- (In reply to Jamie Gritton from comment #2) Thank you very much you guys, As pointed by myself the case is absolutely particular, and indeed you have no need to "break" a jail=20 if you have admin permission for the host system ;-) ... However some unaware sysadm, or a buggy script may lead to the situation described, so, better to have had look ...=20 thank you again --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-jail@freebsd.org Tue Dec 13 20:14:30 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A8C5C76F1A for ; Tue, 13 Dec 2016 20:14:30 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 012921257 for ; Tue, 13 Dec 2016 20:14:26 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id A227C206C9 for ; Tue, 13 Dec 2016 15:14:25 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Tue, 13 Dec 2016 15:14:25 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=/wWBXDSnrEc1l4b a2St1skTkhY4=; b=h1+JkcAHYTZLh/yfllc1aSeLCR3+3Y5Rzoi41SOcPsr3rOR 6l8a5ZWsyTpeZtUMREKbnytytHtObCvbTBwOBZEutvmZjiuCcQNfA3zPFuk07re+ gDo/4ohs0ka7s+kl7L23drp0ttoinYsipwSbSGX0fWV7heXrOhWUps4Y4Zg0= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=smtpout; bh=/wWBXDSnrEc1l4 ba2St1skTkhY4=; b=uEv3iRdfc7/qaMRmtp15K3oiJG5p5v+LmK67DidKdQ5M1K vmPNfWf05l8OjgekbVNtByftvMCiUqHJQRGt7voidlx9XNOeyOrbuArS8C5Z4sna MQoKstuP1xu/9OJJIdS48PWJorq9RkgV3Ue9yRLtwZF2/VOx6PhyTLiGPy8GY= X-ME-Sender: X-Sasl-enc: MjFyJ+TOiHiU75BEw70OWyf9xhBtp/6i+//UscC797yv 1481660065 Received: from [192.168.0.11] (cpe-24-90-224-248.nyc.res.rr.com [24.90.224.248]) by mail.messagingengine.com (Postfix) with ESMTPA id 6153A7ED6D for ; Tue, 13 Dec 2016 15:14:25 -0500 (EST) From: "Isaac (.ike) Levy" Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: multiple interfaces for jail.conf(1) and jail_set(2) Message-Id: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> Date: Tue, 13 Dec 2016 15:14:24 -0500 To: freebsd-jail@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 20:14:30 -0000 Hi All, Can I specify multiple IP interfaces and assign IP=E2=80=99s to them = using jail.conf? I have jails with IPv4/IPv6 addresses on multiple physical interfaces, = as well as assigning a loopback. I have not found answers in the respective man pages or digging online. I=E2=80=99m finally starting to poke around to start using the = impressively simple jail.conf subsystem to manage jails. I have been = managing jails with simple custom start scripts since 99=E2=80=99, and = custom devfs rulesets since ~2006, so jail.conf(1) and jail_set(2) are a = big welcome change for me- really awesome and clean :) -- Additional detail to clarify my loopback use: In general, I always assign each jail it=E2=80=99s own a loopback IP = somewhere in the RFC5735 specified range, 127.0.0.0/8 - (simply saving = 127.0.0.1 for the jailing host), and then I simply set localhost to = point at it=E2=80=99s IP in /etc/hosts for the jail. On the host, I = simply add the IP alias to lo0 like any other interface. This is often overlooked in common jailing practice, but often = eliminates complexity and confusion for many userland daemons. For full = Virtual Server applications, loopback is simply dotting the i=E2=80=99s = and crossing the t=E2=80=99s. I can see how localhost would be challenging to automate for easy = jail.conf configuration, mostly, in picking a loopback IP for the jail = and not letting that get messy- etc=E2=80=A6 Thanks in advance for any info! Best, .ike From owner-freebsd-jail@freebsd.org Tue Dec 13 22:02:36 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CEF6CC76647 for ; Tue, 13 Dec 2016 22:02:36 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.20.71]) by mx1.freebsd.org (Postfix) with ESMTP id AFA013D8 for ; Tue, 13 Dec 2016 22:02:35 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id 8E6C9CB8C9F; Tue, 13 Dec 2016 16:03:26 -0600 (CST) Received: from 128.135.52.6 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Tue, 13 Dec 2016 16:03:26 -0600 (CST) Message-ID: <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> In-Reply-To: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> Date: Tue, 13 Dec 2016 16:03:26 -0600 (CST) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Valeri Galtsev" To: "Isaac (.ike) Levy" Cc: freebsd-jail@freebsd.org Reply-To: galtsev@kicp.uchicago.edu User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 22:02:36 -0000 On Tue, December 13, 2016 2:14 pm, Isaac (.ike) Levy wrote: > Hi All, > > Can I specify multiple IP interfaces and assign IP’s to them using > jail.conf? > I have jails with IPv4/IPv6 addresses on multiple physical interfaces, as > well as assigning a loopback. Last time I tried it which was about year and a half ago the answer was: no, this is not possible. Jail can only have one IP address (in addition to loopback addresses). Valeri > > I have not found answers in the respective man pages or digging online. > > I’m finally starting to poke around to start using the impressively > simple jail.conf subsystem to manage jails. I have been managing jails > with simple custom start scripts since 99’, and custom devfs rulesets > since ~2006, so jail.conf(1) and jail_set(2) are a big welcome change for > me- really awesome and clean :) > > -- > Additional detail to clarify my loopback use: > In general, I always assign each jail it’s own a loopback IP somewhere > in the RFC5735 specified range, 127.0.0.0/8 - (simply saving 127.0.0.1 for > the jailing host), and then I simply set localhost to point at it’s IP > in /etc/hosts for the jail. On the host, I simply add the IP alias to lo0 > like any other interface. > This is often overlooked in common jailing practice, but often eliminates > complexity and confusion for many userland daemons. For full Virtual > Server applications, loopback is simply dotting the i’s and crossing the > t’s. > > I can see how localhost would be challenging to automate for easy > jail.conf configuration, mostly, in picking a loopback IP for the jail and > not letting that get messy- etc… > > Thanks in advance for any info! > > Best, > .ike > > > _______________________________________________ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++ From owner-freebsd-jail@freebsd.org Wed Dec 14 02:09:43 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7AFF9C741F8 for ; Wed, 14 Dec 2016 02:09:43 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mail-pg0-x241.google.com (mail-pg0-x241.google.com [IPv6:2607:f8b0:400e:c05::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45EED1945 for ; Wed, 14 Dec 2016 02:09:43 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mail-pg0-x241.google.com with SMTP id x23so618422pgx.3 for ; Tue, 13 Dec 2016 18:09:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-transfer-encoding; bh=0dToFnEmbaDeZxAQKmCZ08oBVp8LXx49J1PFA/nnUJs=; b=tAFgmQQRlAm2tVt7sDqBRO3Xm48m7uF0iaeOhWd0nq/Adq5Mcj+GCUUPcGac22hVa2 LO7PKoETg4xvwyxMZBaEnUykxBCgPKPX6Hyuz/QFlhGilIOwRxr2M7gqEaeCmsKH3MFH DmkwCeqsmf7YBujfEZBB/QM6D+dLympAS6ysm/YY2hGOUotEEDxXsVfIv3NPlYbl0owP pHfV9bLooITa7k3oJrJgRFfapwVyOzp6/Of2+LdOlAIcPHi2+YdBvvhUY17Pc5swEHUl /AHpPJEFoAKOI9l5UEGW18F8SG8koDovEpBd2ujoO92La/0YdsdRjEF8l+164+64NMAn vs8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-transfer-encoding; bh=0dToFnEmbaDeZxAQKmCZ08oBVp8LXx49J1PFA/nnUJs=; b=VkuL/zsc4Av/NPHCGu0K08dJFTtgPuws0JQZRvYL70giSd3AX1aBowXzq5Wh10D4ow +EuFQW97jlcskZG/qsBdqUTt49hmfJqDtY1H1OMGt+XG1da+MCLmt2zQUV0ozafGpIet ArezwGN2LXiOo0CQty1QLMyd3XS1uyS6cXJ9XnN8W9WisaZm/EejiqzZ8wLxtu2Z+GgF +ien6dIHxnCtQfZBNfQkWtqXLTubSjUyWF1gymoy1H66oojTSTMxTZv4y9rH3hlHZ6sV tDiKd+0wJRDEwsrhsGDg1V4ObosTDL17jFERNIU7DX/pdtGLAKpFWi2wX5VVeT5Lw1Ka rHbg== X-Gm-Message-State: AKaTC00TLeNSRpfLW7EzFaMBrB7bzV0qv2yVP98nLSp0+jDY8owJ6iZRkK29jlIFKY7ttw== X-Received: by 10.99.127.72 with SMTP id p8mr181805270pgn.183.1481681382746; Tue, 13 Dec 2016 18:09:42 -0800 (PST) Received: from [192.168.1.103] ([120.29.76.197]) by smtp.googlemail.com with ESMTPSA id e5sm12567957pfd.77.2016.12.13.18.09.41 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Dec 2016 18:09:42 -0800 (PST) Message-ID: <5850A9F6.2090501@gmail.com> Date: Wed, 14 Dec 2016 10:09:58 +0800 From: Ernie Luzar User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: "Isaac (.ike) Levy" CC: freebsd-jail@FreeBSD.org Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> In-Reply-To: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 02:09:43 -0000 Isaac (.ike) Levy wrote: > Hi All, > > Can I specify multiple IP interfaces and assign IP’s to them using jail.conf? > I have jails with IPv4/IPv6 addresses on multiple physical interfaces, as well as assigning a loopback. > > I have not found answers in the respective man pages or digging online. > > I’m finally starting to poke around to start using the impressively simple jail.conf subsystem to manage jails. I have been managing jails with simple custom start scripts since 99’, and custom devfs rulesets since ~2006, so jail.conf(1) and jail_set(2) are a big welcome change for me- really awesome and clean :) > > -- > Additional detail to clarify my loopback use: > In general, I always assign each jail it’s own a loopback IP somewhere in the RFC5735 specified range, 127.0.0.0/8 - (simply saving 127.0.0.1 for the jailing host), and then I simply set localhost to point at it’s IP in /etc/hosts for the jail. On the host, I simply add the IP alias to lo0 like any other interface. > This is often overlooked in common jailing practice, but often eliminates complexity and confusion for many userland daemons. For full Virtual Server applications, loopback is simply dotting the i’s and crossing the t’s. > > I can see how localhost would be challenging to automate for easy jail.conf configuration, mostly, in picking a loopback IP for the jail and not letting that get messy- etc… > > Thanks in advance for any info! > > Best, > .ike > Using native jail.conf you can assign multiple NICs with both ipv4 & ipv6 ip address. By native I mean use the jail(8) command to start/stop your jails IE. not [service jail start] command. Use this format ip.addr = "rlo:10.0.10.02,xl0:10.20.10.07,lo0:127.10.0.02" This will also automatically create and remove the required aliases. A word about loopback. Just like on the host, most services do not use the loopback interface, this is also true for jailed services. Only services that default to using the loopback interface need one defined in the jail to work correctly. Take note, the services that use the loopback interface default to using 127.0.0.1 ip address. For a service in a jail that uses loopback MUST have it's configuration changed to use the 127.10.0.02 ip address assigned on the jails jail.conf ip.addr parameter. No service in a jail can be assigned the hosts 127.0.0.1 ip address. I recommend you check out these ports, jail-primer gives background on jails across Freebsd releases. qjail a utility that simplifies jail admin. From owner-freebsd-jail@freebsd.org Wed Dec 14 03:21:45 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE28DC76971 for ; Wed, 14 Dec 2016 03:21:45 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2840F120B for ; Wed, 14 Dec 2016 03:21:44 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id uBE3LNAA005091; Wed, 14 Dec 2016 14:21:23 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Wed, 14 Dec 2016 14:21:23 +1100 (EST) From: Ian Smith To: Ernie Luzar cc: "Isaac (.ike) Levy" , freebsd-jail@freebsd.org Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) In-Reply-To: <5850A9F6.2090501@gmail.com> Message-ID: <20161214141809.A26979@sola.nimnet.asn.au> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <5850A9F6.2090501@gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 03:21:45 -0000 On Wed, 14 Dec 2016 10:09:58 +0800, Ernie Luzar wrote: > I recommend you check out these ports, > jail-primer gives background on jails across Freebsd releases. > qjail a utility that simplifies jail admin. Joe Barbish, please stop using responses to people's issues to advertise your ports. Ian From owner-freebsd-jail@freebsd.org Wed Dec 14 04:28:15 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB850C73C7B for ; Wed, 14 Dec 2016 04:28:15 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 90C971D3C for ; Wed, 14 Dec 2016 04:28:14 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id A003C20977; Tue, 13 Dec 2016 23:28:13 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Tue, 13 Dec 2016 23:28:13 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=7glIgxIqjVrpSdko05S7ZHxIdwo=; b=K2+Pun 4s0VmxYmhJNfvHsYQVmFgZNofHd9TcpZwr+BteS4/yB6zuW6DSSnijsc5+HEtgHG zee4atYNXPKarqyDL6WLa0toieodvAWtPH8Lp7+8YyhfEv10/s1OsmAHKs8sL4Sd hpcs9ybQS8W9hnPtCidajRr7HHR9+n4+cTJ/o= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= smtpout; bh=7glIgxIqjVrpSdko05S7ZHxIdwo=; b=he868T/Ar4Ygr0afWlgg qlf7m71uTCjAv+TEP5jLgXSMIXarCYftmaruTMxnWMQAFpAuzmob2sX7SprLHnlj PZi54bKCZEnvFc2tKAso9QaNpdHDTPGnFEziq4rbASXrl69LiUIrlhFdUfgULKA6 aGpoaBCXEs6zX8+Nzzxr4Mk= X-ME-Sender: X-Sasl-enc: 6RjKpU4rVAi9rAocq5ZtrlcX3R+AQ3Dn7nXLe+T7J6yc 1481689693 Received: from [10.0.224.105] (cpe-24-90-119-105.nyc.res.rr.com [24.90.119.105]) by mail.messagingengine.com (Postfix) with ESMTPA id 55E497EA6B; Tue, 13 Dec 2016 23:28:13 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Isaac (.ike) Levy" In-Reply-To: <5850A9F6.2090501@gmail.com> Date: Tue, 13 Dec 2016 23:28:12 -0500 Cc: freebsd-jail@FreeBSD.org Content-Transfer-Encoding: quoted-printable Message-Id: <8E3BBF75-D2A2-4B42-A693-41D0B3F16D19@blackskyresearch.net> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <5850A9F6.2090501@gmail.com> To: Ernie Luzar X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 04:28:15 -0000 Thanks Ernie, But, that straight out did not work for me, > On Dec 13, 2016, at 9:09 PM, Ernie Luzar wrote: >=20 > Isaac (.ike) Levy wrote: >> Hi All, >> Can I specify multiple IP interfaces and assign IP=E2=80=99s to them = using jail.conf? >> I have jails with IPv4/IPv6 addresses on multiple physical = interfaces, as well as assigning a loopback. >> I have not found answers in the respective man pages or digging = online. >> I=E2=80=99m finally starting to poke around to start using the = impressively simple jail.conf subsystem to manage jails. I have been = managing jails with simple custom start scripts since 99=E2=80=99, and = custom devfs rulesets since ~2006, so jail.conf(1) and jail_set(2) are a = big welcome change for me- really awesome and clean :) >> -- >> Additional detail to clarify my loopback use: >> In general, I always assign each jail it=E2=80=99s own a loopback IP = somewhere in the RFC5735 specified range, 127.0.0.0/8 - (simply saving = 127.0.0.1 for the jailing host), and then I simply set localhost to = point at it=E2=80=99s IP in /etc/hosts for the jail. On the host, I = simply add the IP alias to lo0 like any other interface. >> This is often overlooked in common jailing practice, but often = eliminates complexity and confusion for many userland daemons. For full = Virtual Server applications, loopback is simply dotting the i=E2=80=99s = and crossing the t=E2=80=99s. >> I can see how localhost would be challenging to automate for easy = jail.conf configuration, mostly, in picking a loopback IP for the jail = and not letting that get messy- etc=E2=80=A6 >> Thanks in advance for any info! >> Best, >> .ike >=20 > Using native jail.conf you can assign multiple NICs with both ipv4 & = ipv6 ip address. By native I mean use the jail(8) command to start/stop = your jails IE. not [service jail start] command. Use this format > ip.addr =3D "rlo:10.0.10.02,xl0:10.20.10.07,lo0:127.10.0.02" This will = also automatically create and remove the required aliases. That does not appear to work- which is sad, I was excited by the syntax. I am getting the following error, r# jail -c myjail jail: medial: ip4.addr: not an IPv4 address: em0:10.0.0.22 jail: myjail: ip6.addr: not an IPv6 address: em0:2:2:2:2::22 # uname -r 11.0-RELEASE-p2 My jail.conf contains precisely the following, myjail { path =3D /foo/bar; mount.devfs; host.hostname =3D bar.blackskyresearch.net; ip4.addr =3D "em0:10.0.0.22,lo0:127.0.0.22"; ip6.addr =3D "em0:2:2:2:2::22"; exec.start =3D "/bin/sh /etc/rc"; exec.stop =3D "/bin/sh /etc/rc.shutdown"; } Noteworthy- the error thrown for ip4.addr does not even get to = mentioning the second listed IP on lo0. >=20 > A word about loopback. Just like on the host, most services do not use = the loopback interface, this is also true for jailed services. Only = services that default to using the loopback interface need one defined = in the jail to work correctly. Sure sometimes, but not always. While 127.0.0.1 is hardcoded into many = apps and configs, this is certainly more controllable in my world where = I can physically slap whomever wrote the daemon with hardcoded IP=E2=80=99= s- even for using local inet sockets :) >=20 > Take note, the services that use the loopback interface default to = using 127.0.0.1 ip address. For a service in a jail that uses loopback = MUST have it's configuration changed to use the 127.10.0.02 ip address = assigned on the jails jail.conf ip.addr parameter. No service in a jail = can be assigned the hosts 127.0.0.1 ip address. Certainly. Yet, I=E2=80=99ve found very few headaches after changing a = /etc/hosts to reflect the localhost IP for the jail. =E2=80=9Clocalhost=E2= =80=9D just resolves, as it should. >=20 > I recommend you check out these ports, > jail-primer gives background on jails across Freebsd releases. I believe I gave the author of that document extensive feedback when it = was originally authored- as a submission rewrite for the handbook. While this jail-primer doc was filled with many useful and practical = words of advice, it was also a document which I provided a great deal of = constructive feedback for the author, (pre 9.2 release). I was particularly worried about the way the =E2=80=9Cjail cell=E2=80=9D = vocabulary abstraction was introduced and used. I cited a relentless = =E2=80=9Cuse my port=E2=80=9D approach to jail administration. And = finally, in that doc, there was far too much of an overall fundamental = shift away from base UNIX ways of doing things- and even the FreeBSD way = of doing things. I find documentation like this to be frustrating for = oldschoolers because it is not concise or technically informative, and = detracts for new users- by presenting jail(8) in a manner which is = abstracted into something so from the FreeBSD operating system. On a quick skim, the jail-primer project you posted appears to be = roughly the same document- and it also does not have the information = about IP interfaces jail.conf syntax you mention above. > qjail a utility that simplifies jail admin. Thanks, but I=E2=80=99m not really interested in qjail or else I would = have asked about it wherever they run their list! While I do see tools like qjail, good ol=E2=80=99 ezjail, and iocage as = being very valuable, they have little to do with my question. -- Back to the original post- did I do something wrong or interpret your = instructions incorrectly? Thanks! Best, .ike >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 From owner-freebsd-jail@freebsd.org Wed Dec 14 04:32:20 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBD7FC73EDC for ; Wed, 14 Dec 2016 04:32:20 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B02351057 for ; Wed, 14 Dec 2016 04:32:20 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 66197206F7; Tue, 13 Dec 2016 23:32:19 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Tue, 13 Dec 2016 23:32:19 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=JCZzStNdq+uIO7/Lr2VRgSQK6r0=; b=c1hRDf Xu6wHTreSWw2LuanZFKV2A1qekqA6g4GdzWy24KZ32BmLc1fwEdkxTZ1BEWB343U 91nxy80Z/blMRHsdrB1KwaDoe8/O9CrPpoB8OLvr6Gn+Yf56r8sfjCUevlwh8A90 lZB5l1VJc6+zl+ecBplRBcKCO4mMG89+JCM9A= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= smtpout; bh=JCZzStNdq+uIO7/Lr2VRgSQK6r0=; b=hHxDHUNqn0N/j52kQYpJ 82RT3CRIgxXDFD9x/SL3sGHcNxlHM/AIdLdfgI7/aksNGArkR5ZiFM2whrz51afS YuwGo1dcPMIxo7KuyzdW7uLCLor5SLA2usLZlv6QKDyS1fDahEXnR1eTy8rin9DW x+kz7JvYAhp48tFeYZHy9rI= X-ME-Sender: X-Sasl-enc: Yy3Ill0LnLxx396hat9w+8hh556R3UcOkcBRLq9jEyVM 1481689939 Received: from [10.0.224.105] (cpe-24-90-119-105.nyc.res.rr.com [24.90.119.105]) by mail.messagingengine.com (Postfix) with ESMTPA id 261C37E8C1; Tue, 13 Dec 2016 23:32:19 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Isaac (.ike) Levy" In-Reply-To: <20161214141809.A26979@sola.nimnet.asn.au> Date: Tue, 13 Dec 2016 23:32:18 -0500 Cc: Ernie Luzar , freebsd-jail@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <5850A9F6.2090501@gmail.com> <20161214141809.A26979@sola.nimnet.asn.au> To: Ian Smith X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 04:32:21 -0000 Oh, > On Dec 13, 2016, at 10:21 PM, Ian Smith wrote: >=20 > On Wed, 14 Dec 2016 10:09:58 +0800, Ernie Luzar wrote: >=20 >> I recommend you check out these ports, >> jail-primer gives background on jails across Freebsd releases. >> qjail a utility that simplifies jail admin. >=20 > Joe Barbish, >=20 > please stop using responses to people's issues to advertise your = ports. >=20 > Ian Gah. Now I=E2=80=99m a bit sorry to have wasted everyone=E2=80=99s time = with such long description in my last post. If anyone has further clarity on my original post, I=E2=80=99m all ears- = thanks! Best, .ike From owner-freebsd-jail@freebsd.org Wed Dec 14 04:47:58 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E21AC7620D for ; Wed, 14 Dec 2016 04:47:58 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 226511506 for ; Wed, 14 Dec 2016 04:47:57 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 66A4E209BE; Tue, 13 Dec 2016 23:47:56 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Tue, 13 Dec 2016 23:47:56 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=GMPDOfe1Kexo7gUtHgCsyEN6V58=; b=dUcn7F zFjyQ2ql29TxitaUMdJFTYfJPsNn2rtZQVEECIKyjcDiRbY22IVx2YMQfdZ2q2sI yLixy78FmwSYhp0kgoEUQRznSwAYbhoW3poRMZjLSXfa0HalSdRLvV2Vampz6hiF M/Df1i1a6NYfKxyinjT7xVaBJolyNnkd1fQUo= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= smtpout; bh=GMPDOfe1Kexo7gUtHgCsyEN6V58=; b=S99CM1U9pKD9z7GMycMv xjuF/T/2w6erluRgp7o42URH0S92xN4UDfugjKeL8SXmvxCfoMSKJDoBo1gcUvR3 1NbaL/z3zaaOTi01JV2uIQ5EU+9mVP1RYy8GsY96EYfxl07V9UZ/AOA67jN6qL+F TDBM1zUvova8FWKB1eEokqQ= X-ME-Sender: X-Sasl-enc: eBhn+oRnYnZjJW8KODuuwJk7mF5A5HoKqIVzpg1LDBoL 1481690876 Received: from [10.0.224.105] (cpe-24-90-119-105.nyc.res.rr.com [24.90.119.105]) by mail.messagingengine.com (Postfix) with ESMTPA id 1B5627EE37; Tue, 13 Dec 2016 23:47:56 -0500 (EST) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 From: "Isaac (.ike) Levy" X-Priority: 3 (Normal) In-Reply-To: <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> Date: Tue, 13 Dec 2016 23:47:55 -0500 Cc: freebsd-jail@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> To: galtsev@kicp.uchicago.edu X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 04:47:58 -0000 Hi Valeri, > On Dec 13, 2016, at 5:03 PM, Valeri Galtsev = wrote: >=20 > On Tue, December 13, 2016 2:14 pm, Isaac (.ike) Levy wrote: >> Hi All, >>=20 >> Can I specify multiple IP interfaces and assign IP=C3=A2=C2=80=C2=99s = to them using >> jail.conf? >> I have jails with IPv4/IPv6 addresses on multiple physical = interfaces, as >> well as assigning a loopback. >=20 > Last time I tried it which was about year and a half ago the answer = was: Just to clarify your answer, > no, this is not possible. Jail can only have one IP address (in = addition > to loopback addresses). Do you mean this just for jail.conf configuration/usage? If so, from all I=E2=80=99ve read and tried, that makes complete sense, = and makes me sad as it prevents me from using the slickness of = jail.conf(1) and jail_set(2) - not yet :) -- However, to be very clear for the list archive, jails can most = definately have many IP addresses, (since between FreeBSD 7 and 8 I = believe?), including loopback, (which is just an IP address like any = other), For example, # ifconfig em0 inet alias 10.10.10.10/32 # ifconfig em1 inet alias 10.10.10.11/32 # ifconfig lo0 inet alias 127.0.0.11/32 # ifconfig em0 inet6 alias 2:2:2:2::10 prefixlen 64 # jail -c path=3D/some/place host.hostname=3Dmyjail = ip4.addr=3D=E2=80=9C10.10.10.10,10.10.10.11,127.0.0.11" = ip6.addr=3D"2:2:2:2::10" command=3D/bin/sh /etc/rc Best, .ike From owner-freebsd-jail@freebsd.org Wed Dec 14 05:43:30 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA078C7342F for ; Wed, 14 Dec 2016 05:43:30 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 707BA18C7 for ; Wed, 14 Dec 2016 05:43:30 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [10.1.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 5E5AFD6FC for ; Wed, 14 Dec 2016 05:43:28 +0000 (UTC) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) To: freebsd-jail@freebsd.org References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> From: Allan Jude Message-ID: <02b85a36-007b-605d-7ab0-c9e56495d86e@freebsd.org> Date: Wed, 14 Dec 2016 00:43:27 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="f9KsCC1sGLe5h9Vmo8r1N9DjxKTbLujju" X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 05:43:30 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --f9KsCC1sGLe5h9Vmo8r1N9DjxKTbLujju Content-Type: multipart/mixed; boundary="1pxRh7M0npfPIQwOWrx3dmJ2c5ug6dgoK"; protected-headers="v1" From: Allan Jude To: freebsd-jail@freebsd.org Message-ID: <02b85a36-007b-605d-7ab0-c9e56495d86e@freebsd.org> Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> In-Reply-To: --1pxRh7M0npfPIQwOWrx3dmJ2c5ug6dgoK Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2016-12-13 23:47, Isaac (.ike) Levy wrote: > Hi Valeri, >=20 >> On Dec 13, 2016, at 5:03 PM, Valeri Galtsev wrote: >> >> On Tue, December 13, 2016 2:14 pm, Isaac (.ike) Levy wrote: >>> Hi All, >>> >>> Can I specify multiple IP interfaces and assign IP=C3=A2=C2=80=C2=99s= to them using >>> jail.conf? >>> I have jails with IPv4/IPv6 addresses on multiple physical interfaces= , as >>> well as assigning a loopback. >> >> Last time I tried it which was about year and a half ago the answer wa= s: >=20 > Just to clarify your answer, >=20 >> no, this is not possible. Jail can only have one IP address (in additi= on >> to loopback addresses). >=20 > Do you mean this just for jail.conf configuration/usage? >=20 > If so, from all I=E2=80=99ve read and tried, that makes complete sense,= and makes me sad as it prevents me from using the slickness of jail.conf= (1) and jail_set(2) - not yet :) >=20 > -- > However, to be very clear for the list archive, jails can most definate= ly have many IP addresses, (since between FreeBSD 7 and 8 I believe?), in= cluding loopback, (which is just an IP address like any other), >=20 > For example, >=20 > # ifconfig em0 inet alias 10.10.10.10/32 > # ifconfig em1 inet alias 10.10.10.11/32 > # ifconfig lo0 inet alias 127.0.0.11/32 > # ifconfig em0 inet6 alias 2:2:2:2::10 prefixlen 64 > # jail -c path=3D/some/place host.hostname=3Dmyjail ip4.addr=3D=E2=80=9C= 10.10.10.10,10.10.10.11,127.0.0.11" ip6.addr=3D"2:2:2:2::10" command=3D/b= in/sh /etc/rc >=20 > Best, > .ike >=20 >=20 > _______________________________________________ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org"= >=20 In ezjail I can just do this: export jail_something_scaleengine_net_ip=3D"vlan43|10.0.0.17,vlan43|2001:470:1::= 1:6667,lo0|127.0.3.1" If you include the interface name like that, it will automatically add the alias when the jail starts, and remove it when the jail stops (simplifying the task of moving the jail to a different host) If the IP is already bound to the machine, just use the comma separated list of IPs. --=20 Allan Jude --1pxRh7M0npfPIQwOWrx3dmJ2c5ug6dgoK-- --f9KsCC1sGLe5h9Vmo8r1N9DjxKTbLujju Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJYUNv/AAoJEBmVNT4SmAt+94EP/AoMUo3sKE44gl7CrDKmKiv+ UOG4w8TOTLvtXlBWd2rZYGIe3qtfB1B2lyK877TZEbb1Y1j9KbyfYKu5aCV1Gk5f 2f7UKGxDDC45HoBoQ2/uNFEumb4RaLKSiqX2Bp5DxtFnaF6sEkM0eBubLCXsCaDR b9RerNfka/xnJ1BpmAORAb99e5IDjt4hSLhkNlMgQGQgCHQ5q40nWvEjd2MaaXa8 wzTf+fBRGItvnaRKR+SSGNCwb0QXuVaB+FTnCBWWif6dwIedkImWrm6vQ3IQ/Gxm kVRVhXWBYAgYdUvBifVEZ/oXtm5/Id/nNsR60wrjKZqE7vjagNfgXFbFig0lA2YL 5a4SeHsQT31YpGg5XyYyl2ULitN5RgMe70sDyliUuGqbcayHf9t8MuDqIXCOq5Jj dvHwa4lm4HHjpLPvpKEW2TuhjKCVPAHqs/o+AxZUO5n2pxllDClaA//PRwzWkG7J lzWAed4QY/nVMoV5xc5xck0brwVQ8+hTzrE7vAoWVUcsp9HkviKqU3e1IIsU/3Dt 7ZlziBsDi14oNzKyhct5RgkGSDS+HIKjy6tO8LCMsTshxkm3S4wcNmVsFMyIklm5 zYc0E7wkYcC79O6TkMtJT1EFZI97rUuxkZ1iZ8n7CrDK4xDOwVlP1x611w4yDS0U AJGBVyhAIW2lZOeeLjkl =HexK -----END PGP SIGNATURE----- --f9KsCC1sGLe5h9Vmo8r1N9DjxKTbLujju-- From owner-freebsd-jail@freebsd.org Wed Dec 14 09:28:35 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E38B0C76191 for ; Wed, 14 Dec 2016 09:28:35 +0000 (UTC) (envelope-from g8kbvdave@googlemail.com) Received: from mail-wj0-x22a.google.com (mail-wj0-x22a.google.com [IPv6:2a00:1450:400c:c01::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 99E3A1C87 for ; Wed, 14 Dec 2016 09:28:35 +0000 (UTC) (envelope-from g8kbvdave@googlemail.com) Received: by mail-wj0-x22a.google.com with SMTP id tg4so23166711wjb.1 for ; Wed, 14 Dec 2016 01:28:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=LFecoEmOTeq0mQbh0ZB2Xco+tptAsTbEtngaxtwYrps=; b=HCefIpBzUhNqts7n39z/kRBW9N4giCKLiiwMxfn7VQk90DUMnjBoafbHDXkWpy7Tke 7J4BYajkrrPkYBaV1DEkKEucM0U8x7/TrbvZtUqsxC1Tgvqbsm11qjk+rdyQPK5kWLu3 12ygGubZ+TkqnLYLp4FjG2u0b4yN0099QAF1mt63Y5UbOmvB+IEnukIGLEyQweHAmVJ6 LbTgVwGjZZrd0OGGB5u+a7/rOpZWbUcemDZNv1JaRSisjNXUbihOiuM40fdqOEalyN0Z o0ElArGuYWcT0QfcRFyyHOUhO1GWKvCQWFGJtYcyV4W4ZTSts7L3VnY3QAPW+xaOdcD1 +T7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=LFecoEmOTeq0mQbh0ZB2Xco+tptAsTbEtngaxtwYrps=; b=K2GaUXKDCkDij4qLBDBVuRg3fOo85K009meEwZl9NzL/9etTFRvjxT2W0ywjnyfQMx EC7GgWZ1pQjlQhLhBevH1Aos4gLxEoutrqAQVsUkNw0RgfyW/S9CWro1MQ+cYoCJbc/5 nzlZ48zUlFDJsXhIhpFwK1a1C5CURAyeI903TxtzDIRG7GjIZMtDZVZ53e3uUPg4VLnU NootS76NiClHteLQLLM+P7AkEgYozgAg6wQ8Wz6MZXp4MPN+b8tqlK+VjBOR3rgEznNj 8pDQDyl6NK3xfnH/AN4L//aszVeIog3HxeAFWNCfjZW5ypTgAvMtK8BmsMSMO1ODKVXe /m3g== X-Gm-Message-State: AKaTC02w9cEc2iFTXT0ryQLIcPC6/N6DLhek4nf+5LVVO7a5Sj21IEA61svV/X3t0ODzVQ== X-Received: by 10.28.214.84 with SMTP id n81mr5956020wmg.120.1481707713534; Wed, 14 Dec 2016 01:28:33 -0800 (PST) Received: from [192.168.2.55] ([217.41.35.220]) by smtp.gmail.com with ESMTPSA id r7sm66303658wjp.43.2016.12.14.01.28.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Dec 2016 01:28:32 -0800 (PST) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) To: freebsd-jail@freebsd.org References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <5850A9F6.2090501@gmail.com> <20161214141809.A26979@sola.nimnet.asn.au> From: Dave B Message-ID: <39cf9766-4a42-afb7-f488-6c4e335a4ac7@googlemail.com> Date: Wed, 14 Dec 2016 09:28:32 +0000 User-Agent: Mozilla/5.0 (X11; Linux i686; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161214141809.A26979@sola.nimnet.asn.au> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 09:28:36 -0000 On 14/12/16 03:21, Ian Smith wrote: > On Wed, 14 Dec 2016 10:09:58 +0800, Ernie Luzar wrote: > > > I recommend you check out these ports, > > jail-primer gives background on jails across Freebsd releases. > > qjail a utility that simplifies jail admin. > > Joe Barbish, > > please stop using responses to people's issues to advertise your ports. > > Ian > _______________________________________________ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" Why the hatred for QJail? It is very well documented, works, and the result is stable. Heck, it was even featured in the BSD Mag' (where I learnt about it) a couple of years ago, and have been happily using it since for a personal web server. Regards. Dave B. From owner-freebsd-jail@freebsd.org Wed Dec 14 10:43:39 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5758FC7781A for ; Wed, 14 Dec 2016 10:43:39 +0000 (UTC) (envelope-from marcel.plouf@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 3598D167E for ; Wed, 14 Dec 2016 10:43:39 +0000 (UTC) (envelope-from marcel.plouf@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 3226EC77819; Wed, 14 Dec 2016 10:43:39 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FF3EC77818 for ; Wed, 14 Dec 2016 10:43:39 +0000 (UTC) (envelope-from marcel.plouf@gmail.com) Received: from mail-wj0-x241.google.com (mail-wj0-x241.google.com [IPv6:2a00:1450:400c:c01::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B976A167D for ; Wed, 14 Dec 2016 10:43:38 +0000 (UTC) (envelope-from marcel.plouf@gmail.com) Received: by mail-wj0-x241.google.com with SMTP id j10so3723928wjb.3 for ; Wed, 14 Dec 2016 02:43:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bXsF7YXU95QDxuflsVeBnxHPW4ScmbnMImYsBEpMHxE=; b=Ch54r8nu1Se2weosHiC3lha4ZItZTTtdBj1g4Mo0xC+d9NdGl5DVicw4XdmOWrU7Eg JVUIe7liN1Q+WWc4sBJR7UwJhuM30UlW63ownHG+RNzvD8S9etPTZzpgypJkdGg0WJfN ZSiL1Mt1riuHDX8YLz0YcsfuKLisAGl63ZF+Bx3fYc0AB9PUa1+0H9GKtiHkG+KykuFo ov4SNLRTsCTm9nf3g2PYIxtaASERfFzZBTjcyvh67iLMEOf14U+MR42fQrJizdG0wLxh mCZJ7Yg+B7+Aj1ntDST2xvpkbfZkGI73kUlTr+khPFRkyfR1T/i6V9W5SZt7QUV3uxxZ /4Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bXsF7YXU95QDxuflsVeBnxHPW4ScmbnMImYsBEpMHxE=; b=EAfWW85Wh/XCWf18GM1ntDttNWqyIUkewcawRSxuvper99i9LCFbtaTHtqNeBOCuls LzecMEVv9gTSWSonX0zbh7dx9QfY/A4u4L3tRCxoHBdisbDXNKFJmuzoDNLRJhoPVCpG Y1r+94nABehDJ7MhWowNduOnY9barASFNk68xRxK1X5RwPjsGEN0VYvEvbN/IEea69iF 3Oty4tDHasLnfGLuv3VfE4nqaLnSajnNABFQfKMuiN96IMTWGhh/oQY5cK5lh3Tg+HEu vWnWjA0mMjiUc7PnrIVI3yrkWjEbN+mT3cuKYC37ydsggQk0Xs8hG84xHdZDeL7h18jo SSMA== X-Gm-Message-State: AKaTC02WnW/yO/PIIlshdNqHQqwfrJ0i6QGxFBsb3AmiKp6l5OTPc9yP8Wx4nNciBn+enQ== X-Received: by 10.28.211.72 with SMTP id k69mr6251019wmg.137.1481712217093; Wed, 14 Dec 2016 02:43:37 -0800 (PST) Received: from marcel-laptop.lan (85-171-136-71.rev.numericable.fr. [85.171.136.71]) by smtp.gmail.com with ESMTPSA id x5sm66550659wje.36.2016.12.14.02.43.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 14 Dec 2016 02:43:36 -0800 (PST) Date: Wed, 14 Dec 2016 11:42:39 +0100 From: marcel To: Ernie Luzar Cc: jail@freebsd.org Subject: Re: Closing ports in jail with ipfw Message-ID: <20161214114239.60b7fb48@marcel-laptop.lan> In-Reply-To: <5844B557.7050304@gmail.com> References: <20161117233607.3430afd4@marcel-laptop.lan> <5844B557.7050304@gmail.com> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 10:43:39 -0000 Le Mon, 05 Dec 2016 08:31:19 +0800, Ernie Luzar a =C3=A9crit : > marcel wrote: > > Hi there, > >=20 > > I've created a jail and when I do a nmap on his IP, I can see that > > port 25 and 22 are open but I don't want. So i've tried to create > > an IPFW rule by adding 'ipwf -q add 00290 deny all from router to > > jail' to my host ipfw conf file and applied it but ports jail are > > still open. How can I close or open the ports of my jail ? > >=20 > > Thanks ! =20 >=20 > You can not run nmap on the host targeting the jails ip. Doing so > only shows you open ports on the host. You have to run nmap from a > computer on a different public ip address targeting the public ip > address assigned to the jail. If jail is using a non-routeable ip > address, nmap is useless in looking for jail open ports. Hi ! Sorry for silence, I was not able to answer. Yeah I understand, maybe netstat -an in jail is more useful ? When I do that I see port 25 and 514 are open but if I haven't looked yet what is this port 514 I imagine both of these ports are not closable (or it's not advised)=20 isnt'it ?=20 From owner-freebsd-jail@freebsd.org Wed Dec 14 13:31:03 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 467D5C77453 for ; Wed, 14 Dec 2016 13:31:03 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.20.71]) by mx1.freebsd.org (Postfix) with ESMTP id 25CA4188D for ; Wed, 14 Dec 2016 13:31:02 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id 662FECB8CA2; Wed, 14 Dec 2016 07:31:54 -0600 (CST) Received: from 69.209.225.31 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Wed, 14 Dec 2016 07:31:54 -0600 (CST) Message-ID: <61526.69.209.225.31.1481722314.squirrel@cosmo.uchicago.edu> In-Reply-To: References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> Date: Wed, 14 Dec 2016 07:31:54 -0600 (CST) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Valeri Galtsev" To: "Isaac (.ike) Levy" Cc: galtsev@kicp.uchicago.edu, freebsd-jail@freebsd.org Reply-To: galtsev@kicp.uchicago.edu User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 13:31:03 -0000 On Tue, December 13, 2016 10:47 pm, Isaac (.ike) Levy wrote: > Hi Valeri, > >> On Dec 13, 2016, at 5:03 PM, Valeri Galtsev >> wrote: >> >> On Tue, December 13, 2016 2:14 pm, Isaac (.ike) Levy wrote: >>> Hi All, >>> >>> Can I specify multiple IP interfaces and assign IP’s to them using >>> jail.conf? >>> I have jails with IPv4/IPv6 addresses on multiple physical interfaces, >>> as >>> well as assigning a loopback. >> >> Last time I tried it which was about year and a half ago the answer was: > > Just to clarify your answer, > >> no, this is not possible. Jail can only have one IP address (in addition >> to loopback addresses). > > Do you mean this just for jail.conf configuration/usage? No, that was earlier, in /etc/rc.conf. Since then I've heard that it can be done on jail command line, though I never tried as I kind of prefer all my stuff done "generic" and uniform way, something done on command line what does not work in central configuration files be it /etc/rc.conf or /etc/jail.conf feels like hack to me, so I tend to avoid it. Valeri > > If so, from all I’ve read and tried, that makes complete sense, and > makes me sad as it prevents me from using the slickness of jail.conf(1) > and jail_set(2) - not yet :) > > -- > However, to be very clear for the list archive, jails can most definately > have many IP addresses, (since between FreeBSD 7 and 8 I believe?), > including loopback, (which is just an IP address like any other), > > For example, > > # ifconfig em0 inet alias 10.10.10.10/32 > # ifconfig em1 inet alias 10.10.10.11/32 > # ifconfig lo0 inet alias 127.0.0.11/32 > # ifconfig em0 inet6 alias 2:2:2:2::10 prefixlen 64 > # jail -c path=/some/place host.hostname=myjail > ip4.addr=“10.10.10.10,10.10.10.11,127.0.0.11" ip6.addr="2:2:2:2::10" > command=/bin/sh /etc/rc > > Best, > .ike > > ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++ From owner-freebsd-jail@freebsd.org Wed Dec 14 15:21:22 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B21BAC806B4 for ; Wed, 14 Dec 2016 15:21:22 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.20.71]) by mx1.freebsd.org (Postfix) with ESMTP id 72E2C1597; Wed, 14 Dec 2016 15:21:22 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id 7BCF8CB8CA1; Wed, 14 Dec 2016 09:22:14 -0600 (CST) Received: from 128.135.52.6 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Wed, 14 Dec 2016 09:22:14 -0600 (CST) Message-ID: <27934.128.135.52.6.1481728934.squirrel@cosmo.uchicago.edu> In-Reply-To: <02b85a36-007b-605d-7ab0-c9e56495d86e@freebsd.org> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> <02b85a36-007b-605d-7ab0-c9e56495d86e@freebsd.org> Date: Wed, 14 Dec 2016 09:22:14 -0600 (CST) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Valeri Galtsev" To: "Allan Jude" Cc: freebsd-jail@freebsd.org Reply-To: galtsev@kicp.uchicago.edu User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 15:21:22 -0000 On Tue, December 13, 2016 11:43 pm, Allan Jude wrote: > On 2016-12-13 23:47, Isaac (.ike) Levy wrote: >> Hi Valeri, >> >>> On Dec 13, 2016, at 5:03 PM, Valeri Galtsev >>> wrote: >>> >>> On Tue, December 13, 2016 2:14 pm, Isaac (.ike) Levy wrote: >>>> Hi All, >>>> >>>> Can I specify multiple IP interfaces and assign IP’s to them >>>> using >>>> jail.conf? >>>> I have jails with IPv4/IPv6 addresses on multiple physical interfaces, >>>> as >>>> well as assigning a loopback. >>> >>> Last time I tried it which was about year and a half ago the answer >>> was: >> >> Just to clarify your answer, >> >>> no, this is not possible. Jail can only have one IP address (in >>> addition >>> to loopback addresses). >> >> Do you mean this just for jail.conf configuration/usage? >> >> If so, from all I’ve read and tried, that makes complete sense, and >> makes me sad as it prevents me from using the slickness of jail.conf(1) >> and jail_set(2) - not yet :) >> >> -- >> However, to be very clear for the list archive, jails can most >> definately have many IP addresses, (since between FreeBSD 7 and 8 I >> believe?), including loopback, (which is just an IP address like any >> other), >> >> For example, >> >> # ifconfig em0 inet alias 10.10.10.10/32 >> # ifconfig em1 inet alias 10.10.10.11/32 >> # ifconfig lo0 inet alias 127.0.0.11/32 >> # ifconfig em0 inet6 alias 2:2:2:2::10 prefixlen 64 >> # jail -c path=/some/place host.hostname=myjail >> ip4.addr=“10.10.10.10,10.10.10.11,127.0.0.11" ip6.addr="2:2:2:2::10" >> command=/bin/sh /etc/rc >> >> Best, >> .ike >> >> >> _______________________________________________ >> freebsd-jail@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-jail >> To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" >> > > In ezjail I can just do this: > Of course, it is great to learn that some tools can do this or that. However, this only is helpful to those who are just choosing what to use for the future. Once your choice is made, you (at least I) kind of avoid jumping over to doing something using different tools, especially what is already done some specific way on your production machine. I guess, what I'm trying to say is: don't be surprised if OP finds your effort to help him ultimately useless. Incidentally, I for one set up jails "by the book", not by using some tool which does it all for me behind the scenes. So, reference to any tools are kind of set me off (hence this my reply ;-) Just my $0.02. Valeri > > export > jail_something_scaleengine_net_ip="vlan43|10.0.0.17,vlan43|2001:470:1::1:6667,lo0|127.0.3.1" > > If you include the interface name like that, it will automatically add > the alias when the jail starts, and remove it when the jail stops > (simplifying the task of moving the jail to a different host) > > If the IP is already bound to the machine, just use the comma separated > list of IPs. > > > -- > Allan Jude > > ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++ From owner-freebsd-jail@freebsd.org Wed Dec 14 17:45:39 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73C7EC80677 for ; Wed, 14 Dec 2016 17:45:39 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 449CF691 for ; Wed, 14 Dec 2016 17:45:38 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 5DE05209DC; Wed, 14 Dec 2016 12:45:37 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Wed, 14 Dec 2016 12:45:37 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=KiJ8a1xt+I8sfkcp5iiWj2J1I3M=; b=x440n0 YsGajASx9COKanZc7Bmt0vbGQk0ia7pCswgzB+r3qGv/n7ygJGyenisXfDhjojxq libNd6og+7Cm6CQhx1blcQCLTCX6akGTf1j/iCQomkoa0W1Vnz1HArP6AgBmL3or BwURsat3PkNtQbivafTxA/WaH4gDriieTcEz4= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= smtpout; bh=KiJ8a1xt+I8sfkcp5iiWj2J1I3M=; b=lAbR/ZFAPEu7iIs9QdWg TMMVOrvgr78ezBoQP2BZYf4OJSHGBO0YF6JTJzOH1pqJiHHgd+MzR5i1Rrh3RuEa LuZ82tEHtrPPm5WZ3v0mZWVPB7sbsOwclWlU8n41p9vRVrvcrkAT+dIaLZDFTFBp UkZuN4Z7Dd2H5mNwpqNrnLA= X-ME-Sender: X-Sasl-enc: UyLRZ+6YpdJ8ozhSh/yzGHplBgswLmr4mF5bIaK45xsg 1481737537 Received: from [192.168.0.11] (cpe-24-90-224-248.nyc.res.rr.com [24.90.224.248]) by mail.messagingengine.com (Postfix) with ESMTPA id 18E9F7EE98; Wed, 14 Dec 2016 12:45:37 -0500 (EST) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 From: "Isaac (.ike) Levy" X-Priority: 3 (Normal) In-Reply-To: <61526.69.209.225.31.1481722314.squirrel@cosmo.uchicago.edu> Date: Wed, 14 Dec 2016 12:45:35 -0500 Cc: freebsd-jail@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> <61526.69.209.225.31.1481722314.squirrel@cosmo.uchicago.edu> To: galtsev@kicp.uchicago.edu X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 17:45:39 -0000 > On Dec 14, 2016, at 8:31 AM, Valeri Galtsev = wrote: >=20 >>> no, this is not possible. Jail can only have one IP address (in = addition >>> to loopback addresses). >>=20 >> Do you mean this just for jail.conf configuration/usage? >=20 > No, that was earlier, in /etc/rc.conf. Since then I've heard that it = can > be done on jail command line, though I never tried as I kind of prefer = all > my stuff done "generic" and uniform way, something done on command = line > what does not work in central configuration files be it /etc/rc.conf = or > /etc/jail.conf feels like hack to me, so I tend to avoid it. >=20 > Valeri Thanks for clarifying Valeri- that makes complete sense. My first impression of jail.conf is simply that it=E2=80=99s not quite = complete in all the ways I need, yet it is certainly quite simple, = UNIX-ish, and clean! After hacking around with it, think that jail.conf = is only a few features away from being something I=E2=80=99d consider to = be an excellent base utility for starting/stopping my jailed systems. Best, .ike From owner-freebsd-jail@freebsd.org Wed Dec 14 17:51:10 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CFC6AC80850 for ; Wed, 14 Dec 2016 17:51:10 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A4006B9F; Wed, 14 Dec 2016 17:51:09 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id E9F5720729; Wed, 14 Dec 2016 12:41:14 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Wed, 14 Dec 2016 12:41:14 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=FGY5GW7n5Jt6dHtRvM0QMXvJ/TQ=; b=Ki5VPC maZjqZ9qzFKhpZJjLThO6cGQFkYd96G1Abr9eFLZBHEc8QfzeXUYxwSFwFhotLt5 j4uHdrmEDJV+MeayFEQKj9HJgIhRIhY2/Op0SV/IBeBodKc3N7n0b0GqP4ELPEAj ZE9tMWPg+Hzrsxn3K/u4Y8zK32GyY6up5Ia8E= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= smtpout; bh=FGY5GW7n5Jt6dHtRvM0QMXvJ/TQ=; b=q5NSvI/vUc7STm0NUhPH 6r5lPL/ogfSerBTUvVa5Fxpdqk/9yaGXQLeaqlOw15uZH5bcsa8/BcSKfE5q0OcS 0FT60OZ6dV/gLOy3dm7PliLNZBzC5eR8E/qIox0YkpSGbkjMTX8I3I92IPcu6B73 b6Rqh5TK2v8Mg5gLAFttI4Y= X-ME-Sender: X-Sasl-enc: 9BECq37bUM6KSguy17QmbHpgv9jend1hy7WhT4PKQDZT 1481737274 Received: from [192.168.0.11] (cpe-24-90-224-248.nyc.res.rr.com [24.90.224.248]) by mail.messagingengine.com (Postfix) with ESMTPA id AE1E97E8C1; Wed, 14 Dec 2016 12:41:14 -0500 (EST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Isaac (.ike) Levy" In-Reply-To: <02b85a36-007b-605d-7ab0-c9e56495d86e@freebsd.org> Date: Wed, 14 Dec 2016 12:41:13 -0500 Cc: freebsd-jail@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <127FAC22-A5F1-455A-8D3F-85084BF03C4C@blackskyresearch.net> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> <02b85a36-007b-605d-7ab0-c9e56495d86e@freebsd.org> To: Allan Jude X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 17:51:10 -0000 Hi Alan, > On Dec 14, 2016, at 12:43 AM, Allan Jude = wrote: >=20 > In ezjail I can just do this: >=20 >=20 > export > = jail_something_scaleengine_net_ip=3D"vlan43|10.0.0.17,vlan43|2001:470:1::1= :6667,lo0|127.0.3.1" >=20 > If you include the interface name like that, it will automatically add > the alias when the jail starts, and remove it when the jail stops > (simplifying the task of moving the jail to a different host) >=20 > If the IP is already bound to the machine, just use the comma = separated > list of IPs. >=20 >=20 > --=20 > Allan Jude While I appreciate the post, and the syntax is certainly cool to see, my = question is not about ezjail- I would ask questions about ezjail on = whatever respective list there is for that project. My question is about the relatively new and slick jail.conf(1) and = jail_set(2) subsystems in base. Best, .ike From owner-freebsd-jail@freebsd.org Wed Dec 14 18:33:17 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86CB6C769C0 for ; Wed, 14 Dec 2016 18:33:17 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 59D13AF2; Wed, 14 Dec 2016 18:33:16 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id EA9A520768; Wed, 14 Dec 2016 13:33:15 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Wed, 14 Dec 2016 13:33:15 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=5X80oSAcpTBoOa5LmHnCSq+Rub8=; b=k27mCN i5Y8xeP+oB7fNGHnTucFLGyvnHNe9cA7B1GgOgFcnTcIcXyK3a3A+ZZTirdglMT+ 1DCQrFmTh5nWCdEw0oB+FNSyE1tUn0z8+EPzlnUETnLGhGVgJHFTYDX/l5IgATbq 13FfbcjmRmHw3c+MivlElzxIDCnk3u2U5WxnM= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= smtpout; bh=5X80oSAcpTBoOa5LmHnCSq+Rub8=; b=RR0ZTbADQYlGgvB7o4Bz wVRcDHC/mferLBSbrFwq28lkc1lsjfifJlUOEECePv9ODzAKycfmIdE8o9HhbgTn yCKzutd55tqBPQJwlAVEQqZURNyH4HFsLz6EaNgCyoWG+NbBpM+OeiVoAGP7Fqzt KQ8GueVRoZ1VTr0vLEfXMAY= X-ME-Sender: X-Sasl-enc: EaCsxGOT+SFPg8mTYWLZP24Wj40PfVPt0KXHr3U3EEkD 1481740395 Received: from [192.168.0.11] (cpe-24-90-224-248.nyc.res.rr.com [24.90.224.248]) by mail.messagingengine.com (Postfix) with ESMTPA id 9B3AC7F02F; Wed, 14 Dec 2016 13:33:15 -0500 (EST) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 From: "Isaac (.ike) Levy" X-Priority: 3 (Normal) In-Reply-To: <27934.128.135.52.6.1481728934.squirrel@cosmo.uchicago.edu> Date: Wed, 14 Dec 2016 13:33:14 -0500 Cc: Allan Jude , freebsd-jail@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <11488.128.135.52.6.1481666606.squirrel@cosmo.uchicago.edu> <02b85a36-007b-605d-7ab0-c9e56495d86e@freebsd.org> <27934.128.135.52.6.1481728934.squirrel@cosmo.uchicago.edu> To: galtsev@kicp.uchicago.edu X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 18:33:17 -0000 >> In ezjail I can just do this: >>=20 >=20 > Of course, it is great to learn that some tools can do this or that. > However, this only is helpful to those who are just choosing what to = use > for the future. Once your choice is made, you (at least I) kind of = avoid > jumping over to doing something using different tools, especially what = is > already done some specific way on your production machine. >=20 > I guess, what I'm trying to say is: don't be surprised if OP finds = your > effort to help him ultimately useless. >=20 > Incidentally, I for one set up jails "by the book", not by using some = tool > which does it all for me behind the scenes. So, reference to any tools = are > kind of set me off (hence this my reply ;-) >=20 > Just my $0.02. >=20 > Valeri Sorry to drag this out further, but Valeri is spot on here. Sorry to indulge and repeat in my own words- after using jail(8) heavily = since 1999, and even helping run one of the earliest jail based ISP=E2=80=99= s, I am a bit taken back to see such a propensity toward suggesting 3rd = party tooling on this list- particularly as it does not answer my = original question. Has everyone been using so many 3rd party tools for jailing for so long = that we=E2=80=99ve forgotten how jail(8) works, to the point that my = original question can=E2=80=99t even be recognized? A question not = worth answering, but certainly worth pondering! I=E2=80=99m not arguing = against the use of nice 3rd party tools, but I do want to make it very = clear that they are not required for heavy or even light jailing. The strength of jail(8) and jail(2), even before important features like = multiple IP=E2=80=99s and per-jail securelevels etc, was always that = it=E2=80=99s just another small piece of the the UNIX ecosystem- jail(8) = was strong because the *entire* base system made it strong. For example: before multiple jail IP=E2=80=99s, we=E2=80=99d often = simply NAT addresses on the jailing host itself, a bit of scripting = ifconfig(8) made it simple for our environment. Before base provided = per-jail devfs rulesets, (and even before devfs), we=E2=80=99d simply = make and delete packs of =E2=80=98/dev=E2=80=99 tarballs for various = jails- removing the devices which were inappropriate for our applied = need. I could go on forever, but nearly everything one could need in a = jailed system can always be set up using other base tools- and the UNIX = philosophy. Even today, jail(8) is still trivially scriptable for starting/stopping = and managing many jails. For my use, just using the base system is = preferable over 3rd party tooling because I know exactly what I want to = do, and with common UNIX knowledge I can manage hundreds and thousands = of jails across multiple hardware hosts, with nothing but the base = system. 3rd party tools can be wonderful, but over the 17+ years I=E2=80=99= ve been using FreeBSD jail(8), many 3rd party tools have come and gone, = and changed a great deal- but the base UNIX system has not fundamentally = changed. I mean, even many jail related scripts I wrote in 1999 are = still completely functional and relevant. Best, .ike From owner-freebsd-jail@freebsd.org Wed Dec 14 20:30:20 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED7BAC77224 for ; Wed, 14 Dec 2016 20:30:20 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from mx1.enfer-du-nord.net (mx1.enfer-du-nord.net [IPv6:2001:41d0:1008:bcb:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BC619E4C for ; Wed, 14 Dec 2016 20:30:20 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from [IPv6:2003:8c:2e04:5401:2d94:6496:ed77:a6cf] (p2003008C2E0454012D946496ED77A6CF.dip0.t-ipconnect.de [IPv6:2003:8c:2e04:5401:2d94:6496:ed77:a6cf]) by mx1.enfer-du-nord.net (Postfix) with ESMTPSA id 3tf7VV46dMzjyp for ; Wed, 14 Dec 2016 21:30:10 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: Michael Grimm In-Reply-To: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> Date: Wed, 14 Dec 2016 21:30:09 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> To: freebsd-jail@FreeBSD.org X-Virus-Scanned: clamav-milter 0.99.2 at mail X-Virus-Status: Clean X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 20:30:21 -0000 Isaac (.ike) Levy wrote: > Can I specify multiple IP interfaces and assign IP=E2=80=99s to them = using jail.conf? Not sure if I understand your question correctly, but I do define the = following in my jail.conf for VNET jails: # # host dependent global settings # $ip6prefixLOCAL =3D "fd00:dead:beef:1234"; # # global jail settings # host.hostname =3D "${name}"; path =3D "/usr/home/jails/${name}"; mount.fstab =3D "/etc/fstab.${name}"; exec.consolelog =3D "/var/log/jail_${name}_console.log"; vnet =3D "new"; vnet.interface =3D "epair${jailID}b"; exec.clean; mount.devfs; persist; # # network settings to apply/destroy during start/stop of every jail # exec.prestart =3D "sleep 2"; exec.prestart +=3D "/sbin/ifconfig epair${jailID} create up"; exec.prestart +=3D "/sbin/ifconfig bridge0 addm = epair${jailID}a"; exec.start =3D "/sbin/sysctl net.inet6.ip6.dad_count=3D0"; exec.start +=3D "/sbin/ifconfig lo0 127.0.0.1 up"; exec.start +=3D "/sbin/ifconfig epair${jailID}b inet = ${ip4_addr}"; exec.start +=3D "/sbin/ifconfig epair${jailID}b inet6 = ${ip6_addr}"; exec.start +=3D "/sbin/route add default -gateway = 10.1.1.254"; exec.start +=3D "/sbin/route add -inet6 default -gateway = ${ip6prefixLOCAL}::254"; exec.stop =3D "/sbin/route del default"; exec.stop +=3D "/sbin/route del -inet6 default"; exec.stop +=3D "/bin/sh /etc/rc.shutdown"; exec.poststop =3D "/sbin/ifconfig epair${jailID}a destroy"; # # individual jail settings # dns { $jailID =3D 1; $ip4_addr =3D 10.1.1.1; $ip4_addr_2 =3D 10.1.1.2; $ip6_addr =3D ${ip6prefixLOCAL}::1/64; $ip6_addr_2 =3D ${ip6prefixLOCAL}::2/64; exec.start +=3D "/sbin/ifconfig epair${jailID}b inet = ${ip4_addr_2} alias"; exec.start +=3D "/sbin/ifconfig epair${jailID}b inet6 = ${ip6_addr_2} alias"; exec.start +=3D "/bin/sh /etc/rc"; } etc. Again, not sure if I do understand your issue correctly, but the shown = examples of exec.start, exec.stop, etc. are quite versatile to use. I do start/stop my jails by "service jail start/stop". Hope that helps, Michael From owner-freebsd-jail@freebsd.org Wed Dec 14 20:39:46 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A39BC77410 for ; Wed, 14 Dec 2016 20:39:46 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5FE0E1327 for ; Wed, 14 Dec 2016 20:39:45 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 3444320769; Wed, 14 Dec 2016 15:39:44 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Wed, 14 Dec 2016 15:39:44 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=ATv8w2s/T4aU+zH684H0eXikYEo=; b=pAy87x qVNQV00kAY/708VPs54InjtuZbb3t0rUDJeOD30shnD2CWquz9xJ1HJbWv1HeJCD /DSSy/PIQMMzT826GOoGyIzM0cFGFMEgieFyNwLrelk2ExcNDzmqCnjgy6Lb4yy4 SI7x7gtYzf/SAmCAI2wPaA23yVuQRFCvAlr5Y= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= smtpout; bh=ATv8w2s/T4aU+zH684H0eXikYEo=; b=c1SsDBSl0MXMFZICoTrz cnCzOPREcR9QtwKdTOK0Y7nlh5UghUueSEAYwokkJTnawuqOElUEMwL0UvH/d9K+ JFlAaCsXA+B3TFlLR5V0S5IIZCtC2hdAy0gZ/QLN2q21au/clPHpDu63MDZ7EmM2 j5uEXpS3i2eveILlZ1L1hkw= X-ME-Sender: X-Sasl-enc: D6sJu+gYvGSHw1R2bDPR9o37BqqOf4zafmLCNK51xfUz 1481747983 Received: from [192.168.0.11] (cpe-24-90-224-248.nyc.res.rr.com [24.90.224.248]) by mail.messagingengine.com (Postfix) with ESMTPA id DC1457E8C1; Wed, 14 Dec 2016 15:39:43 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Isaac (.ike) Levy" In-Reply-To: <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> Date: Wed, 14 Dec 2016 15:39:43 -0500 Cc: freebsd-jail@FreeBSD.org Content-Transfer-Encoding: quoted-printable Message-Id: <907B489D-899A-4204-96D8-ACF86EE829A7@blackskyresearch.net> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> To: Michael Grimm X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 20:39:46 -0000 Wow, that=E2=80=99s rad Michael, > On Dec 14, 2016, at 3:30 PM, Michael Grimm = wrote: >=20 > Isaac (.ike) Levy wrote: >=20 >> Can I specify multiple IP interfaces and assign IP=E2=80=99s to them = using jail.conf? >=20 > Not sure if I understand your question correctly, but I do define the = following in my jail.conf for VNET jails: >=20 > # > # host dependent global settings > # > $ip6prefixLOCAL =3D "fd00:dead:beef:1234"; >=20 > # > # global jail settings > # > host.hostname =3D "${name}"; > path =3D "/usr/home/jails/${name}"; > mount.fstab =3D "/etc/fstab.${name}"; > exec.consolelog =3D "/var/log/jail_${name}_console.log"; > vnet =3D "new"; > vnet.interface =3D "epair${jailID}b"; > exec.clean; > mount.devfs; > persist; >=20 > # > # network settings to apply/destroy during start/stop of every jail > # > exec.prestart =3D "sleep 2"; > exec.prestart +=3D "/sbin/ifconfig epair${jailID} create up"; > exec.prestart +=3D "/sbin/ifconfig bridge0 addm = epair${jailID}a"; > exec.start =3D "/sbin/sysctl net.inet6.ip6.dad_count=3D0"; > exec.start +=3D "/sbin/ifconfig lo0 127.0.0.1 up"; > exec.start +=3D "/sbin/ifconfig epair${jailID}b inet = ${ip4_addr}"; > exec.start +=3D "/sbin/ifconfig epair${jailID}b inet6 = ${ip6_addr}"; > exec.start +=3D "/sbin/route add default -gateway = 10.1.1.254"; > exec.start +=3D "/sbin/route add -inet6 default -gateway = ${ip6prefixLOCAL}::254"; > exec.stop =3D "/sbin/route del default"; > exec.stop +=3D "/sbin/route del -inet6 default"; > exec.stop +=3D "/bin/sh /etc/rc.shutdown"; > exec.poststop =3D "/sbin/ifconfig epair${jailID}a = destroy"; >=20 > # > # individual jail settings > # > dns { > $jailID =3D 1; > $ip4_addr =3D 10.1.1.1; > $ip4_addr_2 =3D 10.1.1.2; > $ip6_addr =3D ${ip6prefixLOCAL}::1/64; > $ip6_addr_2 =3D ${ip6prefixLOCAL}::2/64; > exec.start +=3D "/sbin/ifconfig epair${jailID}b inet = ${ip4_addr_2} alias"; > exec.start +=3D "/sbin/ifconfig epair${jailID}b inet6 = ${ip6_addr_2} alias"; > exec.start +=3D "/bin/sh /etc/rc"; > } >=20 > etc. I=E2=80=99ll need to study/look up some of that syntax, to fully grok = this, but that comprehensive example appears to hit the nail on the head = several times over with the exec.start/exec.stop action. Two questions though: - I=E2=80=99m confused how you define the shell style $ variables in = your individual jail settings above, e.g. =E2=80=98$ip4_addr_2 =3D = 10.1.1.2;=E2=80=99, why/how does that work? Is that a variable to be = expanded, or some other behavior? > Again, not sure if I do understand your issue correctly, but the shown = examples of exec.start, exec.stop, etc. are quite versatile to use. >=20 > I do start/stop my jails by "service jail start/stop=E2=80=9D. - Obviously you state you=E2=80=99re using service to start/stop jails, = but shouldn=E2=80=99t this work with =E2=80=98jail -c =E2=80=99,= or are these subsystems not interoperable? Thanks! Best, .ike >=20 > Hope that helps, > Michael >=20 >=20 >=20 > _______________________________________________ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to = "freebsd-jail-unsubscribe@freebsd.org" From owner-freebsd-jail@freebsd.org Wed Dec 14 20:53:43 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 147E6C77848 for ; Wed, 14 Dec 2016 20:53:43 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from mx2.enfer-du-nord.net (mx2.enfer-du-nord.net [IPv6:2001:41d0:d:3049:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A699C1CCE for ; Wed, 14 Dec 2016 20:53:42 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from [IPv6:2003:8c:2e04:5401:2d94:6496:ed77:a6cf] (p2003008C2E0454012D946496ED77A6CF.dip0.t-ipconnect.de [IPv6:2003:8c:2e04:5401:2d94:6496:ed77:a6cf]) by mx2.enfer-du-nord.net (Postfix) with ESMTPSA id 3tf81T1d8nzNDt for ; Wed, 14 Dec 2016 21:53:33 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: Michael Grimm In-Reply-To: <907B489D-899A-4204-96D8-ACF86EE829A7@blackskyresearch.net> Date: Wed, 14 Dec 2016 21:53:31 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <818391CE-7425-49DF-8794-B6E43C1389AD@ellael.org> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> <907B489D-899A-4204-96D8-ACF86EE829A7@blackskyresearch.net> To: freebsd-jail@FreeBSD.org X-Virus-Scanned: clamav-milter 0.99.2 at mail X-Virus-Status: Clean X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 20:53:43 -0000 Isaac (.ike) Levy wrote > Wow, that=E2=80=99s rad Michael, >=20 >> On Dec 14, 2016, at 3:30 PM, Michael Grimm = wrote: >>=20 >> Isaac (.ike) Levy wrote: >>=20 >>> Can I specify multiple IP interfaces and assign IP=E2=80=99s to them = using jail.conf? >>=20 >> Not sure if I understand your question correctly, but I do define the = following in my jail.conf for VNET jails: >>=20 >> # >> # host dependent global settings >> # >> $ip6prefixLOCAL =3D "fd00:dead:beef:1234"; >>=20 >> # >> # global jail settings >> # >> host.hostname =3D "${name}"; >> path =3D "/usr/home/jails/${name}"; >> mount.fstab =3D "/etc/fstab.${name}"; >> exec.consolelog =3D "/var/log/jail_${name}_console.log"; >> vnet =3D "new"; >> vnet.interface =3D "epair${jailID}b"; >> exec.clean; >> mount.devfs; >> persist; >>=20 >> # >> # network settings to apply/destroy during start/stop of every jail >> # >> exec.prestart =3D "sleep 2"; >> exec.prestart +=3D "/sbin/ifconfig epair${jailID} = create up"; >> exec.prestart +=3D "/sbin/ifconfig bridge0 addm = epair${jailID}a"; >> exec.start =3D "/sbin/sysctl net.inet6.ip6.dad_count=3D0"; >> exec.start +=3D "/sbin/ifconfig lo0 127.0.0.1 up"; >> exec.start +=3D "/sbin/ifconfig epair${jailID}b inet = ${ip4_addr}"; >> exec.start +=3D "/sbin/ifconfig epair${jailID}b inet6 = ${ip6_addr}"; >> exec.start +=3D "/sbin/route add default -gateway = 10.1.1.254"; >> exec.start +=3D "/sbin/route add -inet6 default -gateway = ${ip6prefixLOCAL}::254"; >> exec.stop =3D "/sbin/route del default"; >> exec.stop +=3D "/sbin/route del -inet6 default"; >> exec.stop +=3D "/bin/sh /etc/rc.shutdown"; >> exec.poststop =3D "/sbin/ifconfig epair${jailID}a = destroy"; >>=20 >> # >> # individual jail settings >> # >> dns { >> $jailID =3D 1; >> $ip4_addr =3D 10.1.1.1; >> $ip4_addr_2 =3D 10.1.1.2; >> $ip6_addr =3D ${ip6prefixLOCAL}::1/64; >> $ip6_addr_2 =3D ${ip6prefixLOCAL}::2/64; >> exec.start +=3D "/sbin/ifconfig epair${jailID}b inet = ${ip4_addr_2} alias"; >> exec.start +=3D "/sbin/ifconfig epair${jailID}b inet6 = ${ip6_addr_2} alias"; >> exec.start +=3D "/bin/sh /etc/rc"; >> } >>=20 >> etc. >=20 > I=E2=80=99ll need to study/look up some of that syntax, to fully grok = this, but that comprehensive example appears to hit the nail on the head = several times over with the exec.start/exec.stop action. >=20 > Two questions though: >=20 > - I=E2=80=99m confused how you define the shell style $ variables in = your individual jail settings above, e.g. =E2=80=98$ip4_addr_2 =3D = 10.1.1.2;=E2=80=99, why/how does that work? Is that a variable to be = expanded, or some other behavior? This is described in jail.conf(5) under the section "variables". I do = have 10 jails running, and those $ variables/parameters are very = helpful, indeed. >> Again, not sure if I do understand your issue correctly, but the = shown examples of exec.start, exec.stop, etc. are quite versatile to = use. >>=20 >> I do start/stop my jails by "service jail start/stop=E2=80=9D. >=20 > - Obviously you state you=E2=80=99re using service to start/stop = jails, but shouldn=E2=80=99t this work with =E2=80=98jail -c = =E2=80=99, or are these subsystems not interoperable? Hmm. I do have to admit that I never tried 'jail -c ', but I = just gave it a try, and yes, it works as well :-)=20 I do use "service jail start/stop" because that will obey my pre-defined = starting/stopping order of jails (which I do need to have, e.g. dns = before mail and such) in /etc/rc.conf jail_enable=3D"YES" jail_reverse_stop=3D"YES" jail_list=3D"dns mail ..." Regards, Michael From owner-freebsd-jail@freebsd.org Wed Dec 14 21:12:37 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4EB7C77F1D for ; Wed, 14 Dec 2016 21:12:37 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 79386F1F for ; Wed, 14 Dec 2016 21:12:36 +0000 (UTC) (envelope-from ike@blackskyresearch.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 047992068F; Wed, 14 Dec 2016 16:12:36 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Wed, 14 Dec 2016 16:12:36 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= blackskyresearch.net; h=cc:content-transfer-encoding :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=sXMu3DXDfET5fLS4vjYjziWX6VU=; b=kFDSiQ x7FPjFdv1INpDJN1+WrF24apIon4186lqyitA0QLNiEriF2c59xS0CmStNZcCYzF 1J2Ecr7/D8RJ3a+fJtvPMP109A8OI1ociCCu56oCZWCrvdZb2ViWKKlv01NgRaFm m+1/tw/s3Nuz2of5aqPW4FflAqXQOpEfg6Q+M= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= smtpout; bh=sXMu3DXDfET5fLS4vjYjziWX6VU=; b=Z+z3OcfLwfWBEi0l2+Ay 6jurG9Aeq1tqgBdK1jb7ctd1JjWHx8MSAA/rHAzsonFdiIgjuGkGXLYaqqlwoyk9 esBov2cDZNJYRF/bGqYzkJxAVN7cCeZOr8y8oHYtiFi/HyZpQFRecNYyLlU7bK0z RMzhqcoRUSPrqXi90OF9pJE= X-ME-Sender: X-Sasl-enc: RhT+x3qoOJ2puOgqcL6gHQuUrWkppaW/z43of0Yut5PX 1481749955 Received: from [192.168.0.11] (cpe-24-90-224-248.nyc.res.rr.com [24.90.224.248]) by mail.messagingengine.com (Postfix) with ESMTPA id BC8B67E8C1; Wed, 14 Dec 2016 16:12:35 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Isaac (.ike) Levy" In-Reply-To: <818391CE-7425-49DF-8794-B6E43C1389AD@ellael.org> Date: Wed, 14 Dec 2016 16:12:35 -0500 Cc: freebsd-jail@FreeBSD.org Content-Transfer-Encoding: quoted-printable Message-Id: <78BCD457-2007-4972-AAE7-0C008238FC4A@blackskyresearch.net> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> <907B489D-899A-4204-96D8-ACF86EE829A7@blackskyresearch.net> <818391CE-7425-49DF-8794-B6E43C1389AD@ellael.org> To: Michael Grimm X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 21:12:37 -0000 Wow, > On Dec 14, 2016, at 3:53 PM, Michael Grimm = wrote: >=20 >> Two questions though: >>=20 >> - I=E2=80=99m confused how you define the shell style $ variables in = your individual jail settings above, e.g. =E2=80=98$ip4_addr_2 =3D = 10.1.1.2;=E2=80=99, why/how does that work? Is that a variable to be = expanded, or some other behavior? >=20 > This is described in jail.conf(5) under the section "variables". I do = have 10 jails running, and those $ variables/parameters are very = helpful, indeed. I get it, the man page explained it well. >=20 >>> Again, not sure if I do understand your issue correctly, but the = shown examples of exec.start, exec.stop, etc. are quite versatile to = use. >>>=20 >>> I do start/stop my jails by "service jail start/stop=E2=80=9D. >>=20 >> - Obviously you state you=E2=80=99re using service to start/stop = jails, but shouldn=E2=80=99t this work with =E2=80=98jail -c = =E2=80=99, or are these subsystems not interoperable? >=20 > Hmm. I do have to admit that I never tried 'jail -c ', but I = just gave it a try, and yes, it works as well :-)=20 >=20 > I do use "service jail start/stop" because that will obey my = pre-defined starting/stopping order of jails (which I do need to have, = e.g. dns before mail and such) in /etc/rc.conf >=20 > jail_enable=3D"YES" > jail_reverse_stop=3D"YES" > jail_list=3D"dns mail =E2=80=A6=E2=80=9D Awesome! For my use, I=E2=80=99m averse to starting jails at host boot- = so I=E2=80=99m really excited this works. Thanks so much Michael- this totally answered my question, I=E2=80=99m = back on the right path to using jail.conf with my setup! Best, .ike From owner-freebsd-jail@freebsd.org Wed Dec 14 21:19:51 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AADE5C8014C for ; Wed, 14 Dec 2016 21:19:51 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from mx2.enfer-du-nord.net (mx2.enfer-du-nord.net [IPv6:2001:41d0:d:3049:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77C201532 for ; Wed, 14 Dec 2016 21:19:51 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from [IPv6:2003:8c:2e04:5401:2d94:6496:ed77:a6cf] (p2003008C2E0454012D946496ED77A6CF.dip0.t-ipconnect.de [IPv6:2003:8c:2e04:5401:2d94:6496:ed77:a6cf]) by mx2.enfer-du-nord.net (Postfix) with ESMTPSA id 3tf8bp0lqrzNFq for ; Wed, 14 Dec 2016 22:19:50 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: Michael Grimm In-Reply-To: <78BCD457-2007-4972-AAE7-0C008238FC4A@blackskyresearch.net> Date: Wed, 14 Dec 2016 22:19:49 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <8915C34D-7F6A-4D11-A5F9-9E31778400DA@ellael.org> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> <907B489D-899A-4204-96D8-ACF86EE829A7@blackskyresearch.net> <818391CE-7425-49DF-8794-B6E43C1389AD@ellael.org> <78BCD457-2007-4972-AAE7-0C008238FC4A@blackskyresearch.net> To: freebsd-jail@FreeBSD.org X-Virus-Scanned: clamav-milter 0.99.2 at mail X-Virus-Status: Clean X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 21:19:51 -0000 Isaac (.ike) Levy wrote: >> I do use "service jail start/stop" because that will obey my = pre-defined starting/stopping order of jails (which I do need to have, = e.g. dns before mail and such) in /etc/rc.conf >>=20 >> jail_enable=3D"YES" >> jail_reverse_stop=3D"YES" >> jail_list=3D"dns mail =E2=80=A6=E2=80=9D >=20 > Awesome! For my use, I=E2=80=99m averse to starting jails at host = boot- so I=E2=80=99m really excited this works. It works very well (for quite some years now). > Thanks so much Michael- this totally answered my question, I=E2=80=99m = back on the right path to using jail.conf with my setup! You are welcome, and I am glad having been able to help (as I have been = helped by others before). Regards, Michael From owner-freebsd-jail@freebsd.org Thu Dec 15 01:33:19 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94D26C7620C for ; Thu, 15 Dec 2016 01:33:19 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 730DEF28 for ; Thu, 15 Dec 2016 01:33:19 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 726A2C7620B; Thu, 15 Dec 2016 01:33:19 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 720F0C7620A for ; Thu, 15 Dec 2016 01:33:19 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mail-pg0-x244.google.com (mail-pg0-x244.google.com [IPv6:2607:f8b0:400e:c05::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 42D8FF27 for ; Thu, 15 Dec 2016 01:33:19 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mail-pg0-x244.google.com with SMTP id p66so4174602pga.2 for ; Wed, 14 Dec 2016 17:33:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-transfer-encoding; bh=y1B0p9yqmRTJjyfHMLmY/wUo32Lr7bYjQ0zkoJUjpbE=; b=W64lUNE0B1cK42Ygl8EZDm7NO0/l0MPrqmuEjaeqFZlvHvxx7bBIZ+JXkddSY/jY7B fkwt79U/COK1O4gHsvMemqpiCm87PIaVNLKwwZd0esOnnG1vvHcDhdBbQYp/UqUhY8nK tefh6/GAqOJg933IEWJBI/PB0mN5yUS+QptdAtrNKECVmfed+i5jit1tbKL8iyCQqB6k xU695lW5WcigyzYSUGYrGHj7xpASSKpfClAyNB9s6tPf/cgWgeK22jD+75aOceVnYpPT uAW5DwtCLU2KrwsWuL17ek68q0rZu6KG0xVuO6ghsMpKldd6C59GcYCdCIraaA99GESV 0vIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-transfer-encoding; bh=y1B0p9yqmRTJjyfHMLmY/wUo32Lr7bYjQ0zkoJUjpbE=; b=d0djl3ZwKRyYnglylprM9jPfmt7jsXbXqjS5B1eyYFOqCu42OuP1HjAOmL+eS7Ffjb uKoTQmmDVsMHt6X+i8RcU68zpnFF+UZLBU//bQM64hAsjuhr7u5p9e94AmH92iM0H/KZ PL25J2VUYvdHMpRhkGq8aBsN7drAM2UPn9HIZE2i/ZQdKWB7GxZfq+iMk2gOJ88F+dJp 7+I00jLOKgPU5UY4NjqY4gudTnkRXIaidEhySuNxr8XvKow/HqDXv+c5LaK1qOxzV/p0 BN+u+tUu6eo/utLX9QGVSnLNDE6gH9SETgF22qVcoOXFxVWjS3/Hl/SHukX/jbTzHU9n pLyQ== X-Gm-Message-State: AKaTC03/+NDavgzdZYVPbGJgFCJif91p3uDUWAbcQuBFcgG0/WuWqj1/i13JQTRCA2VMJA== X-Received: by 10.99.115.5 with SMTP id o5mr57140748pgc.165.1481765598896; Wed, 14 Dec 2016 17:33:18 -0800 (PST) Received: from [192.168.1.103] ([120.29.76.197]) by smtp.googlemail.com with ESMTPSA id s8sm89984130pfj.45.2016.12.14.17.33.17 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 14 Dec 2016 17:33:18 -0800 (PST) Message-ID: <5851F2ED.3070505@gmail.com> Date: Thu, 15 Dec 2016 09:33:33 +0800 From: Ernie Luzar User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: marcel CC: jail@freebsd.org Subject: Re: Closing ports in jail with ipfw References: <20161117233607.3430afd4@marcel-laptop.lan> <5844B557.7050304@gmail.com> <20161214114239.60b7fb48@marcel-laptop.lan> In-Reply-To: <20161214114239.60b7fb48@marcel-laptop.lan> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 01:33:19 -0000 marcel wrote: > Le Mon, 05 Dec 2016 08:31:19 +0800, > Ernie Luzar a écrit : > >> marcel wrote: >>> Hi there, >>> >>> I've created a jail and when I do a nmap on his IP, I can see that >>> port 25 and 22 are open but I don't want. So i've tried to create >>> an IPFW rule by adding 'ipwf -q add 00290 deny all from router to >>> jail' to my host ipfw conf file and applied it but ports jail are >>> still open. How can I close or open the ports of my jail ? >>> >>> Thanks ! >> You can not run nmap on the host targeting the jails ip. Doing so >> only shows you open ports on the host. You have to run nmap from a >> computer on a different public ip address targeting the public ip >> address assigned to the jail. If jail is using a non-routeable ip >> address, nmap is useless in looking for jail open ports. > > Hi ! Sorry for silence, I was not able to answer. Yeah I understand, > maybe netstat -an in jail is more useful ? When I do that I see port 25 > and 514 are open but if I haven't looked yet what is this port 514 I > imagine both of these ports are not closable (or it's not advised) > isnt'it ? > On the host port 25 is sendmail and port 514 is syslog. https://www.grc.com/port_514.htm The syslog server opens port 514 and listens for incoming syslog event notifications (carried by UDP protocol packets) generated by remote syslog clients. Any number of client devices can be programmed to send syslog event messages to whatever servers they choose. This defaults to off on clean install of Freebsd. You must have a statement in your /ect/rc.conf file that enables it. From owner-freebsd-jail@freebsd.org Thu Dec 15 18:36:31 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C5EE2C8183B; Thu, 15 Dec 2016 18:36:31 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from mx1.enfer-du-nord.net (mx1.enfer-du-nord.net [IPv6:2001:41d0:1008:bcb:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 86427D6E; Thu, 15 Dec 2016 18:36:31 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from [IPv6:2003:8c:2e04:6401:1146:4be5:5417:d762] (p2003008C2E04640111464BE55417D762.dip0.t-ipconnect.de [IPv6:2003:8c:2e04:6401:1146:4be5:5417:d762]) by mx1.enfer-du-nord.net (Postfix) with ESMTPSA id 3tfhws4GT6zkBp; Thu, 15 Dec 2016 19:36:29 +0100 (CET) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 From: Michael Grimm X-Priority: 3 (Normal) In-Reply-To: <56419.128.135.52.6.1481751332.squirrel@cosmo.uchicago.edu> Date: Thu, 15 Dec 2016 19:36:28 +0100 Cc: freebsd-jail@FreeBSD.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> <56419.128.135.52.6.1481751332.squirrel@cosmo.uchicago.edu> To: freebsd-questions@freebsd.org X-Virus-Scanned: clamav-milter 0.99.2 at mail X-Virus-Status: Clean X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 18:36:31 -0000 [cc'd to freebsd-jail@FreeBSD.org where that thread originated] Valeri Galtsev wrote: > On Wed, December 14, 2016 2:30 pm, Michael Grimm wrote: >> # >> # network settings to apply/destroy during start/stop of every jail >> # >> exec.prestart =3D "sleep 2"; >> exec.prestart +=3D "/sbin/ifconfig epair${jailID} = create up"; >> exec.prestart +=3D "/sbin/ifconfig bridge0 addm = epair${jailID}a"; >> exec.start =3D "/sbin/sysctl net.inet6.ip6.dad_count=3D0"; >> exec.start +=3D "/sbin/ifconfig lo0 127.0.0.1 up"; >> exec.start +=3D "/sbin/ifconfig epair${jailID}b inet = ${ip4_addr}"; >> exec.start +=3D "/sbin/ifconfig epair${jailID}b inet6 = ${ip6_addr}"; >> exec.start +=3D "/sbin/route add default -gateway = 10.1.1.254"; >> exec.start +=3D "/sbin/route add -inet6 default -gateway = ${ip6prefixLOCAL}::254"; >> exec.stop =3D "/sbin/route del default"; >> exec.stop +=3D "/sbin/route del -inet6 default"; >> exec.stop +=3D "/bin/sh /etc/rc.shutdown"; >> exec.poststop =3D "/sbin/ifconfig epair${jailID}a = destroy"; >>=20 >> # >> # individual jail settings >> # >> dns { >> $jailID =3D 1; >> $ip4_addr =3D 10.1.1.1; >> $ip4_addr_2 =3D 10.1.1.2; [=E2=80=A6] > Michael, is it possible to have two addresses belonging to two = different > networks (through two different network interfaces)? >=20 > Say, on host system: >=20 > ifconfig_igb0=3D"inet 172.20.9.22 ... > ifconfig_igb1=3D"inet 10.1.1.17 ... >=20 >=20 > and in some jail >=20 > $ip4_addr =3D 172.20.9.22; > $ip4_addr_2 =3D 10.1.1.17; >=20 > - will that work? This is what didn't work for me in the past when > configured jails old style in /etc/rc.conf I can't answer that because I have never tried it before. Those exec. will give you a very fine-grained control over which = commands are run at the host environment (exec.prestart, exec.poststop, = exec.poststart) or within the jail's environment (exec.start, exec.stop, = exec.prestop) in which order. Have a look at jail(8) for details (and = presumably more exec.). Therefore I am quite confident that whatever could be done with jails = might be "coded" into jail.conf. Regards, Michael From owner-freebsd-jail@freebsd.org Thu Dec 15 19:10:04 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B167C82570 for ; Thu, 15 Dec 2016 19:10:04 +0000 (UTC) (envelope-from marcel.plouf@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4824E399 for ; Thu, 15 Dec 2016 19:10:04 +0000 (UTC) (envelope-from marcel.plouf@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 47803C8256F; Thu, 15 Dec 2016 19:10:04 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47201C8256E for ; Thu, 15 Dec 2016 19:10:04 +0000 (UTC) (envelope-from marcel.plouf@gmail.com) Received: from mail-wj0-x241.google.com (mail-wj0-x241.google.com [IPv6:2a00:1450:400c:c01::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF581398 for ; Thu, 15 Dec 2016 19:10:03 +0000 (UTC) (envelope-from marcel.plouf@gmail.com) Received: by mail-wj0-x241.google.com with SMTP id j10so11325846wjb.3 for ; Thu, 15 Dec 2016 11:10:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Wp6NrScVEMf8v9q2ut1T5yvfnzqGUd4DWu7zB0BK8WE=; b=jDAZy9OwVxFAP+ZOpvPMOTqw229tRoNGIrW8Ke+IE5+HXh/IR+Cy4GjsQGbWAsNs43 v/oYUncHK9RTM49OP7viN3CJlE1g5xEgERa543DHL6Yt8ewPuxZNKjlNMKyen3DGcFuk Q1AXRln6AEcTSsqXy4+7XIa24+q6ni7o5HJS+2ytBxbGpN4FlUBxXjUOi1j04W6j9nFA 96mK1hSZ54v7mI7FWmbD00KAHLVQ5Hw4Ibu0e7d4G2t++GoZXGlAdcMwZ4dYJxbfhnQA nEp7oolWAb622jyeEFZjcaNDA/s34/OJVu8c6kR8kYZTQiRk4FcFRyD7AMIiewMsWwqX IO7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Wp6NrScVEMf8v9q2ut1T5yvfnzqGUd4DWu7zB0BK8WE=; b=b+txkh2Yfa4TFw4hzOPO+kFoD79JzLxbMHv/9gBLUnqflsIO1BTYGaDM7fss4XuHST wC66Y1kc6q7rxcpVtQAffVS7loHsX5Hgt0W9B77lV3lUruWZmm8xWnpO2IbpFw82rgBM FUgvY/NGZ9Ifs4fwWD/Of6GvD5bcn0a2G5c/YX/ltwpHv4L080rLAlErSN+UFJN8kejn wSjcAC+tmX15GoiGXlFDcgRG4GxcNmZpvppGz1Rf4fImm/31MQk2i9E/Os+d1v2iVlsT zGvOr/hwlrwqgv4PqslE6LyUWe9XiGGiE7V/+tjWZw6JbokKJJEoDUJYs2i3RF9OdOrP aHbg== X-Gm-Message-State: AKaTC03atLBYkIWlJg7qtupt9QX9/2UJ7BfQqVh+dNhPCT6125P+tV5thKzakNbGBucitA== X-Received: by 10.194.8.226 with SMTP id u2mr2899610wja.91.1481829000777; Thu, 15 Dec 2016 11:10:00 -0800 (PST) Received: from marcel-laptop.lan (85-171-136-71.rev.numericable.fr. [85.171.136.71]) by smtp.gmail.com with ESMTPSA id g184sm13734799wme.23.2016.12.15.11.10.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 15 Dec 2016 11:10:00 -0800 (PST) Date: Thu, 15 Dec 2016 20:09:05 +0100 From: marcel To: Ernie Luzar Cc: jail@freebsd.org Subject: Re: Closing ports in jail with ipfw Message-ID: <20161215200905.0f921a0a@marcel-laptop.lan> In-Reply-To: <5851F2ED.3070505@gmail.com> References: <20161117233607.3430afd4@marcel-laptop.lan> <5844B557.7050304@gmail.com> <20161214114239.60b7fb48@marcel-laptop.lan> <5851F2ED.3070505@gmail.com> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 19:10:04 -0000 Le Thu, 15 Dec 2016 09:33:33 +0800, Ernie Luzar a =C3=A9crit : > marcel wrote: > > Le Mon, 05 Dec 2016 08:31:19 +0800, > > Ernie Luzar a =C3=A9crit : > > =20 > >> marcel wrote: =20 > >>> Hi there, > >>> > >>> I've created a jail and when I do a nmap on his IP, I can see that > >>> port 25 and 22 are open but I don't want. So i've tried to create > >>> an IPFW rule by adding 'ipwf -q add 00290 deny all from router to > >>> jail' to my host ipfw conf file and applied it but ports jail are > >>> still open. How can I close or open the ports of my jail ? > >>> > >>> Thanks ! =20 > >> You can not run nmap on the host targeting the jails ip. Doing so > >> only shows you open ports on the host. You have to run nmap from a > >> computer on a different public ip address targeting the public ip > >> address assigned to the jail. If jail is using a non-routeable ip > >> address, nmap is useless in looking for jail open ports. =20 > >=20 > > Hi ! Sorry for silence, I was not able to answer. Yeah I understand, > > maybe netstat -an in jail is more useful ? When I do that I see > > port 25 and 514 are open but if I haven't looked yet what is this > > port 514 I imagine both of these ports are not closable (or it's > > not advised) isnt'it ?=20 > > =20 >=20 > On the host port 25 is sendmail and port 514 is syslog. >=20 > https://www.grc.com/port_514.htm >=20 > The syslog server opens port 514 and listens for incoming syslog > event notifications (carried by UDP protocol packets) generated by > remote syslog clients. Any number of client devices can be programmed > to send syslog event messages to whatever servers they choose. >=20 > This defaults to off on clean install of Freebsd. > You must have a statement in your /ect/rc.conf file that enables it. >=20 >=20 Okay thanks for clarifications for port 514. When you say "This defaults to off on clean install of Freebsd" you meant that this is the default on the default install but we can put it off on a clean modified freebsd install ? From owner-freebsd-jail@freebsd.org Thu Dec 15 20:09:31 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72778C81648; Thu, 15 Dec 2016 20:09:31 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 366449DE; Thu, 15 Dec 2016 20:09:30 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id CD5B62849B; Thu, 15 Dec 2016 21:09:27 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 80D5528429; Thu, 15 Dec 2016 21:09:26 +0100 (CET) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) To: Michael Grimm , freebsd-questions@freebsd.org Cc: freebsd-jail@FreeBSD.org References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> <56419.128.135.52.6.1481751332.squirrel@cosmo.uchicago.edu> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <5852F876.5070807@quip.cz> Date: Thu, 15 Dec 2016 21:09:26 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 20:09:31 -0000 Michael Grimm wrote on 2016/12/15 19:36: > [cc'd to freebsd-jail@FreeBSD.org where that thread originated] > > Valeri Galtsev wrote: > >> On Wed, December 14, 2016 2:30 pm, Michael Grimm wrote: > >>> # >>> # network settings to apply/destroy during start/stop of every jail >>> # >>> exec.prestart = "sleep 2"; >>> exec.prestart += "/sbin/ifconfig epair${jailID} create up"; >>> exec.prestart += "/sbin/ifconfig bridge0 addm epair${jailID}a"; >>> exec.start = "/sbin/sysctl net.inet6.ip6.dad_count=0"; >>> exec.start += "/sbin/ifconfig lo0 127.0.0.1 up"; >>> exec.start += "/sbin/ifconfig epair${jailID}b inet ${ip4_addr}"; >>> exec.start += "/sbin/ifconfig epair${jailID}b inet6 ${ip6_addr}"; >>> exec.start += "/sbin/route add default -gateway 10.1.1.254"; >>> exec.start += "/sbin/route add -inet6 default -gateway ${ip6prefixLOCAL}::254"; >>> exec.stop = "/sbin/route del default"; >>> exec.stop += "/sbin/route del -inet6 default"; >>> exec.stop += "/bin/sh /etc/rc.shutdown"; >>> exec.poststop = "/sbin/ifconfig epair${jailID}a destroy"; >>> >>> # >>> # individual jail settings >>> # >>> dns { >>> $jailID = 1; >>> $ip4_addr = 10.1.1.1; >>> $ip4_addr_2 = 10.1.1.2; > > […] > >> Michael, is it possible to have two addresses belonging to two different >> networks (through two different network interfaces)? >> >> Say, on host system: >> >> ifconfig_igb0="inet 172.20.9.22 ... >> ifconfig_igb1="inet 10.1.1.17 ... >> >> >> and in some jail >> >> $ip4_addr = 172.20.9.22; >> $ip4_addr_2 = 10.1.1.17; >> >> - will that work? This is what didn't work for me in the past when >> configured jails old style in /etc/rc.conf > > I can't answer that because I have never tried it before. More IP addresses on more interfaces works for me for many years even in old rc.conf style jails. Converted to new jail.conf is something like this costa { host.hostname = "costa.example.com"; ip4.addr = 94.104.135.21; ip4.addr += 192.168.222.57; } As you can see, IPs are from different networks. We are not using auto add / remove IP on interfaces. We don't want to have something else to manage IP addresses. All IPs are defined in rc.conf on their proper interfaces. In this case, first jail's IP is in bge1 and the second is on nfe0 (LAN interface) I already made jail using VPN assigned IP on tun0 OpenVPN interface. In another words - jail doesn't care about interfaces. If there is an IP in the system (on whatever interface) then you can assign it to jail and you can assign as many IPs as you want (up to some really high limit). Miroslav Lachman From owner-freebsd-jail@freebsd.org Thu Dec 15 20:32:47 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3AB9CC820D5; Thu, 15 Dec 2016 20:32:47 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.20.71]) by mx1.freebsd.org (Postfix) with ESMTP id EE15C1B33; Thu, 15 Dec 2016 20:32:46 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id 56D67CB8CA1; Thu, 15 Dec 2016 14:33:41 -0600 (CST) Received: from 128.135.52.6 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Thu, 15 Dec 2016 14:33:41 -0600 (CST) Message-ID: <14885.128.135.52.6.1481834021.squirrel@cosmo.uchicago.edu> In-Reply-To: <5852F876.5070807@quip.cz> References: <0ED7F403-F14E-4A72-8E54-AF74AAE15061@blackskyresearch.net> <45822529-2096-4B32-8515-F5875BEF7101@ellael.org> <56419.128.135.52.6.1481751332.squirrel@cosmo.uchicago.edu> <5852F876.5070807@quip.cz> Date: Thu, 15 Dec 2016 14:33:41 -0600 (CST) Subject: Re: multiple interfaces for jail.conf(1) and jail_set(2) From: "Valeri Galtsev" To: "Miroslav Lachman" <000.fbsd@quip.cz> Cc: "Michael Grimm" , freebsd-questions@freebsd.org, freebsd-jail@freebsd.org Reply-To: galtsev@kicp.uchicago.edu User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 20:32:47 -0000 On Thu, December 15, 2016 2:09 pm, Miroslav Lachman wrote: > Michael Grimm wrote on 2016/12/15 19:36: >> [cc'd to freebsd-jail@FreeBSD.org where that thread originated] >> >> Valeri Galtsev wrote: >> >>> On Wed, December 14, 2016 2:30 pm, Michael Grimm wrote: >> >>>> # >>>> # network settings to apply/destroy during start/stop of every jail >>>> # >>>> exec.prestart = "sleep 2"; >>>> exec.prestart += "/sbin/ifconfig epair${jailID} create up"; >>>> exec.prestart += "/sbin/ifconfig bridge0 addm epair${jailID}a"; >>>> exec.start = "/sbin/sysctl net.inet6.ip6.dad_count=0"; >>>> exec.start += "/sbin/ifconfig lo0 127.0.0.1 up"; >>>> exec.start += "/sbin/ifconfig epair${jailID}b inet ${ip4_addr}"; >>>> exec.start += "/sbin/ifconfig epair${jailID}b inet6 ${ip6_addr}"; >>>> exec.start += "/sbin/route add default -gateway 10.1.1.254"; >>>> exec.start += "/sbin/route add -inet6 default -gateway >>>> ${ip6prefixLOCAL}::254"; >>>> exec.stop = "/sbin/route del default"; >>>> exec.stop += "/sbin/route del -inet6 default"; >>>> exec.stop += "/bin/sh /etc/rc.shutdown"; >>>> exec.poststop = "/sbin/ifconfig epair${jailID}a destroy"; >>>> >>>> # >>>> # individual jail settings >>>> # >>>> dns { >>>> $jailID = 1; >>>> $ip4_addr = 10.1.1.1; >>>> $ip4_addr_2 = 10.1.1.2; >> >> […] >> >>> Michael, is it possible to have two addresses belonging to two >>> different >>> networks (through two different network interfaces)? >>> >>> Say, on host system: >>> >>> ifconfig_igb0="inet 172.20.9.22 ... >>> ifconfig_igb1="inet 10.1.1.17 ... >>> >>> >>> and in some jail >>> >>> $ip4_addr = 172.20.9.22; >>> $ip4_addr_2 = 10.1.1.17; >>> >>> - will that work? This is what didn't work for me in the past when >>> configured jails old style in /etc/rc.conf >> >> I can't answer that because I have never tried it before. > > > > More IP addresses on more interfaces works for me for many years even in > old rc.conf style jails. > > Converted to new jail.conf is something like this > > costa { > host.hostname = "costa.example.com"; > ip4.addr = 94.104.135.21; > ip4.addr += 192.168.222.57; > } Thanks, Miroslav. I do not recollect "ip4.addr += ..." that must have been my problem (though I asked on mail lists and wasn't directed towards that, got the answer "not possible", - I must have been unlucky then). Valeri > > As you can see, IPs are from different networks. > We are not using auto add / remove IP on interfaces. We don't want to > have something else to manage IP addresses. All IPs are defined in > rc.conf on their proper interfaces. > In this case, first jail's IP is in bge1 and the second is on nfe0 (LAN > interface) > > I already made jail using VPN assigned IP on tun0 OpenVPN interface. > > In another words - jail doesn't care about interfaces. If there is an IP > in the system (on whatever interface) then you can assign it to jail and > you can assign as many IPs as you want (up to some really high limit). > > Miroslav Lachman > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "freebsd-questions-unsubscribe@freebsd.org" ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++ From owner-freebsd-jail@freebsd.org Fri Dec 16 01:56:31 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96CA7C82E68 for ; Fri, 16 Dec 2016 01:56:31 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 740CB1FAA for ; Fri, 16 Dec 2016 01:56:31 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 7067EC82E67; Fri, 16 Dec 2016 01:56:31 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 700C4C82E66 for ; Fri, 16 Dec 2016 01:56:31 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mail-pg0-x243.google.com (mail-pg0-x243.google.com [IPv6:2607:f8b0:400e:c05::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49BE01FA7 for ; Fri, 16 Dec 2016 01:56:31 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mail-pg0-x243.google.com with SMTP id 3so7946767pgd.0 for ; Thu, 15 Dec 2016 17:56:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-transfer-encoding; bh=5T3s2bLtQO+SPXUe2Ph6ckQIkH3kjxziDeNoXYpGhLk=; b=rtlMCUeJL++/MhRYH/0pejd1f0dpq4TTWqcyttjTP+fVBjo2Nz8ZluHO23Oe5/HNyK hD20+xDDDW1NRd4GtTNjm9hrQCTI3ihGIjT9LqbF0jSRcMw6qHfXMvHbXOIxsD1OOxVj gxqOeR3xYk1JU6GSPXCXz5AoT5kum2Z4LSMQnHAJBlyM9P0jcYW97Nx3UuRf3gQdn+6c H/8CSGPEOMqdOy/gFdn0puDrl7tZpLHzP/I4gj7uJ6WarMl6Sn8JKmaBgLyjHO+fwau9 UPgQo4Ry4ky2XboKxQ7vH/k6RD7oKeJwIct1rlIAdw2WFB2D3ixyIvmF+9hsdrsEpMmL hrJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-transfer-encoding; bh=5T3s2bLtQO+SPXUe2Ph6ckQIkH3kjxziDeNoXYpGhLk=; b=ObyyFPhxGkffSJLY47URfxgaeQ7SiJhZFY4lHiuY5gIZ6c6MbcXwkDf6XPr4rT6O+4 j5nWsZBxm5s+a0zGu3F7lGya/PFEobegHsrfq1mhUgmz6HMba9iWC0vK9kMCj6o9C8x2 XUiYxbblUrRQhN6+z/HQFUkaaQrkEZkOUiPAegCyJO6b0Ju8GtHTajXSINa7xzA6fF+7 L6SV0zcnuUvfioxsTRF4pLsl2JhzAkg5vyCEDyzR+oAdDXpYv1hGV47FW4StbQGeeXqh tSE3idzDY2tn4ei59adk8r8PnxzgBOFVkrthdx3uptiMCmcx/6hYSQOHYn1+gDg+GOUE 3azg== X-Gm-Message-State: AKaTC002eKslkl2eBKbKhWEsT7VoqYtW4TI1CGe9roG1DkMrYUAjAH/Fpus1f0KTNbsZqA== X-Received: by 10.99.42.80 with SMTP id q77mr1062280pgq.170.1481853390777; Thu, 15 Dec 2016 17:56:30 -0800 (PST) Received: from [192.168.1.103] ([120.29.76.197]) by smtp.googlemail.com with ESMTPSA id f23sm7192382pff.59.2016.12.15.17.56.29 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 15 Dec 2016 17:56:30 -0800 (PST) Message-ID: <585349DF.40908@gmail.com> Date: Fri, 16 Dec 2016 09:56:47 +0800 From: Ernie Luzar User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: marcel CC: jail@freebsd.org Subject: Re: Closing ports in jail with ipfw References: <20161117233607.3430afd4@marcel-laptop.lan> <5844B557.7050304@gmail.com> <20161214114239.60b7fb48@marcel-laptop.lan> <5851F2ED.3070505@gmail.com> <20161215200905.0f921a0a@marcel-laptop.lan> In-Reply-To: <20161215200905.0f921a0a@marcel-laptop.lan> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 01:56:31 -0000 marcel wrote: > Le Thu, 15 Dec 2016 09:33:33 +0800, > Ernie Luzar a écrit : > >> marcel wrote: >>> Le Mon, 05 Dec 2016 08:31:19 +0800, >>> Ernie Luzar a écrit : >>> >>>> marcel wrote: >>>>> Hi there, >>>>> >>>>> I've created a jail and when I do a nmap on his IP, I can see that >>>>> port 25 and 22 are open but I don't want. So i've tried to create >>>>> an IPFW rule by adding 'ipwf -q add 00290 deny all from router to >>>>> jail' to my host ipfw conf file and applied it but ports jail are >>>>> still open. How can I close or open the ports of my jail ? >>>>> >>>>> Thanks ! >>>> You can not run nmap on the host targeting the jails ip. Doing so >>>> only shows you open ports on the host. You have to run nmap from a >>>> computer on a different public ip address targeting the public ip >>>> address assigned to the jail. If jail is using a non-routeable ip >>>> address, nmap is useless in looking for jail open ports. >>> Hi ! Sorry for silence, I was not able to answer. Yeah I understand, >>> maybe netstat -an in jail is more useful ? When I do that I see >>> port 25 and 514 are open but if I haven't looked yet what is this >>> port 514 I imagine both of these ports are not closable (or it's >>> not advised) isnt'it ? >>> >> On the host port 25 is sendmail and port 514 is syslog. >> >> https://www.grc.com/port_514.htm >> >> The syslog server opens port 514 and listens for incoming syslog >> event notifications (carried by UDP protocol packets) generated by >> remote syslog clients. Any number of client devices can be programmed >> to send syslog event messages to whatever servers they choose. >> >> This defaults to off on clean install of Freebsd. >> You must have a statement in your /ect/rc.conf file that enables it. >> >> > > Okay thanks for clarifications for port 514. > When you say "This defaults to off on clean install of Freebsd" you > meant that this is the default on the default install but we can put it > off on a clean modified freebsd install ? > yes In rc.conf syslogd_flags="-ss" From owner-freebsd-jail@freebsd.org Fri Dec 16 12:16:22 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F6E1C82E9F for ; Fri, 16 Dec 2016 12:16:22 +0000 (UTC) (envelope-from admin@x151.save85off.com) Received: from x151.save85off.com (x151.save85off.com [43.240.238.151]) by mx1.freebsd.org (Postfix) with ESMTP id DF5461D7E for ; Fri, 16 Dec 2016 12:16:21 +0000 (UTC) (envelope-from admin@x151.save85off.com) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=save85off; d=x151.save85off.com; h=MIME-Version:From:To:Date:Subject:Content-Type:Content-Transfer-Encoding; i=admin@x151.save85off.com; bh=0CF8fdUrp/mMt78Youl6eMyMNGU=; b=XOx8ltEOl10/2RasYg/C1KrzllJGjFyvczNnloXuEX1VerqDa710dMAfVKhqf8j4wsPZkW4TGClT 4kdAJRqkDdEPeiBTC/fH7s+Bfcr6oxDMR4vtBkvffOn38rgo/eCTrdByXxI8CflCRQ6lOoMBU/2C ANZSvGyM9avu3Cf6gIY= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=save85off; d=x151.save85off.com; b=HC3ojmD/bkg7IkEPF1a8GU0YCb2Jw41wT5n9r1Or52J4M25FeLN2XE9ho9oaKZJ0HLGgyMm3lq9D MGQEkc46+wg6WImDehMNyx+pN0ZzKlHtzyYkpSA1mJVyT8MlMnb4GoPNIN0vydbnwOc/f63cCKHr PSrtUG9wbJsiGo5JP4s=; From: "UGG Big Deals" To: freebsd-jail@freebsd.org Date: 16 Dec 2016 20:04:14 +0800 Subject: Products Almost Sold Out!Christmas Prices + Free Shipping For A Few More Hours! win 87$ MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 12:16:22 -0000 From owner-freebsd-jail@freebsd.org Fri Dec 16 13:16:03 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE8A1C82DBA for ; Fri, 16 Dec 2016 13:16:03 +0000 (UTC) (envelope-from Alexander@leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:375::1:5]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 466169BB for ; Fri, 16 Dec 2016 13:16:03 +0000 (UTC) (envelope-from Alexander@leidinger.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1481894142; bh=iZ61DbpVTiXcW5dINqF09uUGQw4SYP6yI6xVVJYk+gU=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=AiVmhRLqCHtlg/otOZQ1SPLh6XhKIYvNsOXXX8c4x1qEctWkzk9hG5VWfIVHg5EkP 8R6zGVd3+M8aCk9gapp7Bg4k+dGZfUQ3we+RQ0+MmmR6CsnZbGTpJ/EgMANt6psg+q +3GSLOHr0b50JNRNGrPq6JaP6nE7DXlNFMCIzwfB8cS/zSNCCVLRtAMupwhHrpPmXk +uO2hOnvmmhHyqaEFgYKwqnar+ROcCagvdaePYu9/8dZyfNMv0+MSBp89RZBfrn9d+ 3zyep3RIs4/wgfgn9CXzjLGxGMY1mRlI7Dx6Mpy+umSRh/Apq7wcC01+20IWtS450x +s+7alEG+SHUQ== DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1481894161; bh=iZ61DbpVTiXcW5dINqF09uUGQw4SYP6yI6xVVJYk+gU=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=nfP80H3BiCu1zBMcfIr2wSimZ1vZVb+4rmB/HCBmDQrQ9pqFmuI+w3bbU9kOFArPV Gy1IDJMlpvU9pXeqTnompVrR/TyPQplDnPn3V2kPXaor2iIxckrBYBS+DUVziFb/sh f0EP6OGtA8O7AGiB11LZmhF0n6PlunQV/a50I8Vj3NpfZYSUwcbT7R7njYe2eUZEky UXGDd6DUBYyYdtc57AhU7Jo0FgzAgYkYn/zMyh2QmxWn7En80kAldEVwp0Nep07x4/ uMNOD0obH5CS94tSw4IFL+zshT/boAjuw7XLA3cbiX1k4Wfn3d4A7dAFdUgGCWbE76 SV0ZGMsDMvPPA== Date: Fri, 16 Dec 2016 14:15:40 +0100 Message-ID: <20161216141540.Horde.zfu3fokeVx7FuFkk7_s-nbW@webmail.leidinger.net> From: Alexander Leidinger To: SK Cc: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-jail Subject: Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial] References: <584986D0.3040109@quip.cz> <2b6346f8-ed02-0e6d-bd89-106098e7eb2d@cps-intl.org> <58499446.3050403@quip.cz> <5849C5BF.7020005@quip.cz> <584A9179.9060508@quip.cz> <584A9D89.4040003@quip.cz> <3851c5d9-7646-b670-357e-ae937fcc7e8f@cps-intl.org> <584AB345.4080307@quip.cz> <33473585-3cb9-10d3-acf9-0a917c5a0079@cps-intl.org> In-Reply-To: <33473585-3cb9-10d3-acf9-0a917c5a0079@cps-intl.org> User-Agent: Horde Application Framework 5 Content-Type: multipart/signed; boundary="=_phyLGgUE4cHaL8DaVblK1rU"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 13:16:03 -0000 This message is in MIME format and has been PGP signed. --=_phyLGgUE4cHaL8DaVblK1rU Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Quoting SK (from Mon, 12 Dec 2016 17:13:27 +0000): > b) Alexander, I am still not able to do snapshot or any other action=20= =20 >=20from within my jail. My understanding is that you are using ezjail,=20= =20 >=20which might be doing something that my regular jail creation is=20=20 >=20ommitting. If you do not mind sharing your configuration steps, I=20=20 >=20can try to reproduce it at this end. If it is exactly as it is on=20=20 >=20the site you pointed to earlier, please let me know, I will follow=20= =20 >=20that verbatim (even though I do not remember seeing anything there=20= =20 >=20that I have not tried already, but I might be mistaken). Do you use quotas on the datasets you want to add to the jail? If yes,=20= =20 try=20without. The man-page of zfs tells that this value can not be=20=20 changed=20(but from the wording I would expect hat an already set value=20= =20 should=20work). ezjail is just a shell script which simplifies some things with jails.=20= =20 For=20a particular jail where I can manage the datasets which are handed=20= =20 over=20to the jail I have those settings in ezjail which correspond to=20= =20 the=20settings you can specify in jail.conf: ---snip--- export jail_xyz_leidinger_net_devfs_ruleset=3D"17" export jail_xyz_leidinger_net_zfs_datasets=3D"space/something" export jail_xyz_leidinger_net_parameters=3D"allow.mount allow.mount.zfs=20= =20 enforce_statfs=3D1" ---snip--- Check if you have allow.mount and allow.mount.zfs for the jails in question= . Note, "space/something" is not the root of the jail, it's a seperate=20=20 dataset.=20Do not add the root of the jail as a dataset. Example bellow. devfs.rules part: ---snip--- [devfsrules_unhide_zfs=3D12] add path zfs unhide [devfsrules_jail_withzfs=3D17] add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login add include $devfsrules_unhide_zfs ---snip--- The rc.conf inside this jail: ---snip--- zfs_enable=3D"YES" ---snip--- For one of the filesystems I have set "zfs allow" permissions, but=20=20 just=20that a specific user in the jail can do something on those FS=20=20 without=20the need to switch to root. So as long as you try to do a zfs=20= =20 create/snapshot=20with an user with UID 0 inside the jail, the "zfs=20=20 allow"=20part doesn't come into play. So assume space/jails/xyz.leidinger.net/ to be the dataset which=20=20 contains=20the root of the jail but is not attached/attributed to the=20=20 jail=20itself. space/jails/xyz.leidinger.net/data with mountpoint=3Dnone=20= =20 to=20be attributed ("zfs jail xyz space/jails/xyz.leidinger.net/data")=20= =20 to=20the jail (similar to the "space/something" in the ezjail config=20=20 above,=20I have some iocage-managed jails were this works). In this case=20= =20 you=20should be able to do from inside the jail "zfs create -o=20=20 mpuntpoint=3D/mnt space/jails/xyz.leidinger.net/data/test". > And now to everyone, I am still confused about zfs set jailed=3Don. As=20= =20 >=20I mentioned on my previous emails, as soon as I do that, the dataset=20= =20 >=20vanishes from the host system (as I understand, that is expected=20=20 >=20behaviour). Then the jail fails as it is unable to mount /dev, /proc From the zfs man page: ---snip--- After a dataset is attached to a jail and the jailed property is set,= a jailed file system cannot be mounted outside the jail, since the jail administrator might have set the mount point to an unacceptable value= . ---snip--- So yes, it is expected that it "vanishes", but it should be visible=20=20 from=20the parent host at the place inside the jail FS subtree were it=20= =20 is=20mounted there (after setting the mountpoint of the dataset). > and so on. I have to change jail.conf and comment out mount.devfs=20=20 >=20and mount.procfs -- but than in turn makes /dev/zfs unavaulable and=20= =20 >=20I cannot do anything from inside the jail. Could it be that you try to attribute the root of the jail as a=20=20 dataset=20into the jail? Bye, Alexander. --=20 http://www.Leidinger.net=20Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_phyLGgUE4cHaL8DaVblK1rU Content-Type: application/pgp-signature Content-Description: Digitale PGP-Signatur Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iQIcBAABAgAGBQJYU+j8AAoJEKrxQhqFIICEjbQQAJuO7/Xl8Pg7VtL+5iV67YeU 1tMBplCtPS4ep25gutG5065P6Ed0YVpo+mSFXgkEWxNqVjJwSd7E3Xfd1+x9d7mc QjtjfiHE98pycF97AdbMLkWwk7yOLEqM1Irfz7AB6CDK2wwVskMT7Lo0KwcH0zgy J+OcuZpVoSLxbFShWerpAJ3yGP6XCDERqNRDf+m64yLefk0kYXj2GE1StucAwsy7 nInboqxLcqwCthJ/hFx377f0Eo6W5Td/yTV0T8k5y8b4+vuZehSQihYYoiuDcXzP NNn/fDeVWYEerR+0qQMmirVFxDbCNfvQpVjzZxy6tPXrFGXlYcd7R5/a0GY0fkKQ eKjKqn3PNwF3IwgAi4fI53ekEd6gFLPfdQYM9iO73PxcVft5weUWXE2aBApAqUbh VEUInQbwE8JTlJ8w7cqi6x9sSi7Atr+iJ5qAKZ9zArGlx52i/ApwCOy692b57kaH b7G4zNFA0W5ssN6z7v2S4v3I7ED5bZqg+wywCHfZ+SFXkEK2lUhB0CvARE5ODDby hy8VbaKCou3gCv5QTyE1qns3kfILFRYTAO+UI0QcRYnaFZkBvi8bbe+gtaw8ML6e q0YQ3p4tJDnqwmK8uvXJvXm7muEbBa/A/6lZsvEzttgSrwVh/GMC5UKB/KD+9Hnl 7koYGyq1Q6kjQjvh7kBn =DQcR -----END PGP SIGNATURE----- --=_phyLGgUE4cHaL8DaVblK1rU-- From owner-freebsd-jail@freebsd.org Fri Dec 16 14:41:01 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBFE3C82946 for ; Fri, 16 Dec 2016 14:41:01 +0000 (UTC) (envelope-from fbstable@cps-intl.org) Received: from mailer.parlicentre.org (mailer.parlicentre.org [199.48.132.166]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91E111AAB for ; Fri, 16 Dec 2016 14:41:01 +0000 (UTC) (envelope-from fbstable@cps-intl.org) From: SK To: Alexander Leidinger , SK References: <584986D0.3040109@quip.cz> <2b6346f8-ed02-0e6d-bd89-106098e7eb2d@cps-intl.org> <58499446.3050403@quip.cz> <5849C5BF.7020005@quip.cz> <584A9179.9060508@quip.cz> <584A9D89.4040003@quip.cz> <3851c5d9-7646-b670-357e-ae937fcc7e8f@cps-intl.org> <584AB345.4080307@quip.cz> <33473585-3cb9-10d3-acf9-0a917c5a0079@cps-intl.org> <20161216141540.Horde.zfu3fokeVx7FuFkk7_s-nbW@webmail.leidinger.net> Cc: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-jail Message-ID: Date: Fri, 16 Dec 2016 14:02:20 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20161216141540.Horde.zfu3fokeVx7FuFkk7_s-nbW@webmail.leidinger.net> X-SA-Exim-Mail-From: fbstable@cps-intl.org X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mailer.parlicentre.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE, RP_MATCHES_RCVD autolearn=unavailable autolearn_force=no version=3.4.1 Subject: Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial] X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mailer.parlicentre.org) Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 14:41:02 -0000 On 16/12/2016 13:15, Alexander Leidinger wrote: > Quoting SK (from Mon, 12 Dec 2016 17:13:27 > +0000): > >> b) Alexander, I am still not able to do snapshot or any other action >> from within my jail. My understanding is that you are using ezjail, >> which might be doing something that my regular jail creation is >> ommitting. If you do not mind sharing your configuration steps, I can >> try to reproduce it at this end. If it is exactly as it is on the >> site you pointed to earlier, please let me know, I will follow that >> verbatim (even though I do not remember seeing anything there that I >> have not tried already, but I might be mistaken). > > Do you use quotas on the datasets you want to add to the jail? If yes, > try without. The man-page of zfs tells that this value can not be > changed (but from the wording I would expect hat an already set value > should work). > Hi Alexander, short answer, NO, I have not used quota yet > ezjail is just a shell script which simplifies some things with jails. > For a particular jail where I can manage the datasets which are handed > over to the jail I have those settings in ezjail which correspond to > the settings you can specify in jail.conf: > ---snip--- > export jail_xyz_leidinger_net_devfs_ruleset="17" > export jail_xyz_leidinger_net_zfs_datasets="space/something" > export jail_xyz_leidinger_net_parameters="allow.mount allow.mount.zfs > enforce_statfs=1" > ---snip--- > Check if you have allow.mount and allow.mount.zfs for the jails in > question. > Yes, those are in place. > Note, "space/something" is not the root of the jail, it's a seperate > dataset. Do not add the root of the jail as a dataset. Example bellow. > I think that might have been the cause for it to not work for me. I had space/something/jailRoot, and defined that as the dataset for the jail > devfs.rules part: > ---snip--- > [devfsrules_unhide_zfs=12] > add path zfs unhide > > [devfsrules_jail_withzfs=17] > add include $devfsrules_hide_all > add include $devfsrules_unhide_basic > add include $devfsrules_unhide_login > add include $devfsrules_unhide_zfs > ---snip--- > > The rc.conf inside this jail: > ---snip--- > zfs_enable="YES" > ---snip--- > These are all in place as expected. > For one of the filesystems I have set "zfs allow" permissions, but > just that a specific user in the jail can do something on those FS > without the need to switch to root. So as long as you try to do a zfs > create/snapshot with an user with UID 0 inside the jail, the "zfs > allow" part doesn't come into play. > > So assume space/jails/xyz.leidinger.net/ to be the dataset which > contains the root of the jail but is not attached/attributed to the > jail itself. space/jails/xyz.leidinger.net/data with mountpoint=none > to be attributed ("zfs jail xyz space/jails/xyz.leidinger.net/data") > to the jail (similar to the "space/something" in the ezjail config > above, I have some iocage-managed jails were this works). In this case > you should be able to do from inside the jail "zfs create -o > mpuntpoint=/mnt space/jails/xyz.leidinger.net/data/test". > hmmm, I'm slightly confused at this point. Let me see if I can clarify that in my brain If I understand you correctly, what you are suggesting is, the dataset used by the jail itself for its root/base cannot be "worked on" from within the jail, but if I define a different dataset (under the same branch below the jail dataset), and attribute it to the jail, then I can manipulate that "other" dataset. Could you please confirm if I understood it correctly? >> And now to everyone, I am still confused about zfs set jailed=on. As >> I mentioned on my previous emails, as soon as I do that, the dataset >> vanishes from the host system (as I understand, that is expected >> behaviour). Then the jail fails as it is unable to mount /dev, /proc > > From the zfs man page: > ---snip--- > After a dataset is attached to a jail and the jailed property is > set, a > jailed file system cannot be mounted outside the jail, since the > jail > administrator might have set the mount point to an unacceptable > value. > ---snip--- > > So yes, it is expected that it "vanishes", but it should be visible > from the parent host at the place inside the jail FS subtree were it > is mounted there (after setting the mountpoint of the dataset). > I think what you are trying to tell here is, unless and until that "vanished" dataset is put to use (mounted) from inside the jail, it will remain vanished/unusable from the host itself; however, once that dataset is put to use, the host system should be able to "see" and maybe even work on that dataset. Could you please confirm if I understood you correctly? >> and so on. I have to change jail.conf and comment out mount.devfs and >> mount.procfs -- but than in turn makes /dev/zfs unavaulable and I >> cannot do anything from inside the jail. > > Could it be that you try to attribute the root of the jail as a > dataset into the jail? > Yes, I did. But now that I have a few more slightly different explanation from you, once you confirm if my understanding is correct, I am willing to give this a try. Thanks again for your suggestions and guidlines. Have a lovely weekend. SK From owner-freebsd-jail@freebsd.org Sat Dec 17 19:00:20 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51F85C8564B for ; Sat, 17 Dec 2016 19:00:20 +0000 (UTC) (envelope-from Alexander@leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:375::1:5]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E192812F5 for ; Sat, 17 Dec 2016 19:00:19 +0000 (UTC) (envelope-from Alexander@leidinger.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1482001189; bh=Lr3/gP3/8z61i+jQcvgtIq0UYG6/mZV0omnRrr6nxiI=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=qIhSvgpy9xRGF/aON5+Grjt7VYaxg+jNkyoqVc2KgWZy23VeT6yHW5NG9kZ6jv1cz mOZl9jgSUl8T1XoNqNrNhNx81+plwiL85Om+CVHybYIJeHr/EdDO8qggY7fOrJglS5 R9KbaaeVnfYZiydnp1kQcwfHznhDjBvIxFh0uc00pmKlSn63PyjbZm0A2Fc+ajkOR1 ieQpEGJ9rd1qvdS4bM2CVh1J/Q0bkhk8tHmjLnbAFBy1AeLtbSCYu73BqlJBSHsiFI +O7CPoY6aDUdVk0ywL9JdgF+bFi6dP8sq5aETw+Yj1tsXc0dmWcJJCM6I7e+94TrP6 jtFbB9wtu+H5w== DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1482001208; bh=Lr3/gP3/8z61i+jQcvgtIq0UYG6/mZV0omnRrr6nxiI=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=Ic3iPhE3JLhKYW5ASy+IWrUrQu10uYQa0Hwdni2VpH4/qjV3Evxmz2zh0eD9IpBKS 6C55OzUWOXYcUAeDzv5csA14w9o0zWJYDTMOZszAPmLQMdMsNCPr309vkABY1v3WDH 5k1NSooPBNtWtBwKAnvopRZgUWu3R8i2SUbZ2i0WPDiVMYi+MqJB9bD237FwuJSMIe 1s7bmnU1OhgjUzlcn+q8wnQnre89bDs+rtwjGitVvWjQnR+r6ERlM9Mn88WSPjTJ/j F06BphvyV/dC+c29xiLjY6NWuV2uQoOB+7sxa5uenjD8jlnPTI87r6WLbBv5ueBRKV 9gF+gFhlPmHww== Date: Sat, 17 Dec 2016 19:59:49 +0100 Message-ID: <20161217195949.Horde.PTQ3AH5YpaT79dVSxM5UvNr@webmail.leidinger.net> From: Alexander Leidinger To: SK Cc: freebsd-jail Subject: Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial] References: <584986D0.3040109@quip.cz> <2b6346f8-ed02-0e6d-bd89-106098e7eb2d@cps-intl.org> <58499446.3050403@quip.cz> <5849C5BF.7020005@quip.cz> <584A9179.9060508@quip.cz> <584A9D89.4040003@quip.cz> <3851c5d9-7646-b670-357e-ae937fcc7e8f@cps-intl.org> <584AB345.4080307@quip.cz> <33473585-3cb9-10d3-acf9-0a917c5a0079@cps-intl.org> <20161216141540.Horde.zfu3fokeVx7FuFkk7_s-nbW@webmail.leidinger.net> In-Reply-To: User-Agent: Horde Application Framework 5 Content-Type: multipart/signed; boundary="=_K9AONlVuvw9Zy4MTTLpJPDH"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2016 19:00:20 -0000 This message is in MIME format and has been PGP signed. --=_K9AONlVuvw9Zy4MTTLpJPDH Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Quoting SK (from Fri, 16 Dec 2016 14:02:20 +0000): > On 16/12/2016 13:15, Alexander Leidinger wrote: >> For one of the filesystems I have set "zfs allow" permissions, but=20=20 >>=20just that a specific user in the jail can do something on those FS=20= =20 >>=20without the need to switch to root. So as long as you try to do a=20= =20 >>=20zfs create/snapshot with an user with UID 0 inside the jail, the=20=20 >>=20"zfs allow" part doesn't come into play. >> >> So assume space/jails/xyz.leidinger.net/ to be the dataset which=20=20 >>=20contains the root of the jail but is not attached/attributed to the=20= =20 >>=20jail itself. space/jails/xyz.leidinger.net/data with=20=20 >>=20mountpoint=3Dnone to be attributed ("zfs jail xyz=20=20 >>=20space/jails/xyz.leidinger.net/data") to the jail (similar to the=20=20 >>=20"space/something" in the ezjail config above, I have some=20=20 >>=20iocage-managed jails were this works). In this case you should be=20= =20 >>=20able to do from inside the jail "zfs create -o mpuntpoint=3D/mnt=20=20 >>=20space/jails/xyz.leidinger.net/data/test". >> > hmmm, I'm slightly confused at this point. Let me see if I can=20=20 >=20clarify that in my brain > > If I understand you correctly, what you are suggesting is, the=20=20 >=20dataset used by the jail itself for its root/base cannot be "worked=20= =20 >=20on" from within the jail, but if I define a different dataset (under=20= =20 >=20the same branch below the jail dataset), and attribute it to the=20=20 >=20jail, then I can manipulate that "other" dataset. Could you please=20= =20 >=20confirm if I understood it correctly? Correct. You need the data in the root of the jail to boot, if you then=20=20 attribute=20this dataset to the jail, it will vanish until "zfs mount=20=20 -a"=20is run (rc script inside the jail). As it will vanish during the=20= =20 boot=20of the jail (if added automatically), the rc script to mount all=20= =20 datasets=20can not be found. >>> And now to everyone, I am still confused about zfs set jailed=3Don.=20= =20 >>>=20As I mentioned on my previous emails, as soon as I do that, the=20=20 >>>=20dataset vanishes from the host system (as I understand, that is=20=20 >>>=20expected behaviour). Then the jail fails as it is unable to mount=20= =20 >>>=20/dev, /proc >> >> From the zfs man page: >> ---snip--- >> After a dataset is attached to a jail and the jailed property is set= , a >> jailed file system cannot be mounted outside the jail, since the jai= l >> administrator might have set the mount point to an unacceptable valu= e. >> ---snip--- >> >> So yes, it is expected that it "vanishes", but it should be visible=20= =20 >>=20from the parent host at the place inside the jail FS subtree were=20= =20 >>=20it is mounted there (after setting the mountpoint of the dataset). >> > I think what you are trying to tell here is, unless and until that=20=20 >=20"vanished" dataset is put to use (mounted) from inside the jail, it=20= =20 >=20will remain vanished/unusable from the host itself; however, once=20=20 >=20that dataset is put to use, the host system should be able to "see"=20= =20 >=20and maybe even work on that dataset. Could you please confirm if I=20= =20 >=20understood you correctly? Correct. A sub-dataset which is not needed to boot, or a dataset not within the=20= =20 subtree=20of the jail (and not needed to boot) can be used. Bye, Alexander. --=20 http://www.Leidinger.net=20Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_K9AONlVuvw9Zy4MTTLpJPDH Content-Type: application/pgp-signature Content-Description: Digitale PGP-Signatur Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iQIcBAABAgAGBQJYVYskAAoJEKrxQhqFIICErHwP/13/s8PzZMP0lLFi4hh4SjxA FHNVhEzdapZMtYMe62vz2W3UkSxZTRsGnehnYINvh6KOd0MZ6lKAfar8dIP20qrm eIsj4qh9sWcEpkSYOpM16uEdNJ1wIdniKhd4OmR/nyMshdHHVXVtdRIDVVA6OFav lscK45Evixfb5PKhReR1zpqa747jB8v8e6vbzmg8Or+I01gugb9Wa1j2g7BZhi+N wOzIYa2Y5jn20xcCtuWG3EMbKXQATyhhfPvz/fKnedLFag4I4uoMtnCOI1iREDMk FcZMs5YKXHvQRclfMKh+zTpc+Gszf8PQsHwNlU5veTmI7jsWP6p21HxTOjL/CbNy SpIn+YQkqfCvVRGEogKHinl/ltUQnV7nT0g2kl1Od8eSHxCtJQUf0sOy0/PSBf2q 50sxX0FMWUazRNWf00BhIHcxHbj4PmW0lgh014eptKNoB2o0DF23bhD6DigE2zaC TU0rnb3cDKeUIssu3gMrx0p3JlX1Ob9eyxVPWkSJwixlgIpazwAQRBFq6O8VuftW gSDsNE7XUw3YS59kCiKMu/Sswr3pshy3pDpIONQp2j4//uFckVDMgQsbHdghCf68 oLxcoulpl4iPEBR5l0IkvSKWUoE/CRTSRRvr1O2X7PYW552NRMdWAJbiWdfG+lTk JrPpii4sGtmpFoSjh5e4 =eVue -----END PGP SIGNATURE----- --=_K9AONlVuvw9Zy4MTTLpJPDH--