From owner-freebsd-jail@FreeBSD.ORG Mon Jan 16 11:07:05 2012 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68E4D106566B for ; Mon, 16 Jan 2012 11:07:05 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 569E08FC28 for ; Mon, 16 Jan 2012 11:07:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0GB75XO057675 for ; Mon, 16 Jan 2012 11:07:05 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0GB74E1057672 for freebsd-jail@FreeBSD.org; Mon, 16 Jan 2012 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 16 Jan 2012 11:07:04 GMT Message-Id: <201201161107.q0GB74E1057672@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-jail@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2012 11:07:05 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- p bin/161957 jail jls(8): jls -v doesn't show anything if system compile o kern/159918 jail [jail] inter-jail communication failure o kern/156111 jail [jail] procstat -b not supported in jail o misc/155765 jail [patch] `buildworld' does not honors WITHOUT_JAIL o conf/154246 jail [jail] [patch] Bad symlink created if devfs mount poin o conf/149050 jail [jail] rcorder ``nojail'' too coarse for Jail+VNET s conf/142972 jail [jail] [patch] Support JAILv2 and vnet in rc.d/jail o conf/141317 jail [patch] uncorrect jail stop in /etc/rc.d/jail o kern/133265 jail [jail] is there a solution how to run nfs client in ja o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid o bin/32828 jail [jail] w(1) incorrectly handles stale utmp slots with 12 problems total. From owner-freebsd-jail@FreeBSD.ORG Tue Jan 17 20:36:34 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 585931065689 for ; Tue, 17 Jan 2012 20:36:34 +0000 (UTC) (envelope-from prvs=1363d33761=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id D71B08FC1E for ; Tue, 17 Jan 2012 20:36:33 +0000 (UTC) X-Spam-Processed: mail1.multiplay.co.uk, Tue, 17 Jan 2012 16:36:55 +0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50017594316.msg for ; Tue, 17 Jan 2012 15:32:49 +0000 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1363d33761=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-jail@freebsd.org Message-ID: From: "Steven Hartland" To: Date: Tue, 17 Jan 2012 15:32:49 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Subject: mtr doesn't work in a jail even with security.jail.allow_raw_sockets: 1 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2012 20:36:34 -0000 Wanted to use mtr to diagnose an issue in a jail but it seems it totally fails even with security.jail.allow_raw_sockets: 1 Any ideas? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-jail@FreeBSD.ORG Tue Jan 17 21:11:11 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F0A49106577D for ; Tue, 17 Jan 2012 21:11:11 +0000 (UTC) (envelope-from michal.vanco@satro.sk) Received: from smtp1.satro.sk (smtp1.satro.sk [217.144.16.213]) by mx1.freebsd.org (Postfix) with ESMTP id 7755F8FC1E for ; Tue, 17 Jan 2012 21:11:11 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp1.satro.sk (Postfix) with ESMTP id 12D561BF962; Tue, 17 Jan 2012 21:41:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=satro.sk; h= x-mailer:references:message-id:content-transfer-encoding:date :date:in-reply-to:from:from:content-type:content-type :mime-version:subject:subject:received:received; s=smtp1; t= 1326832910; bh=xOEY1vhLwQgov0TlV/9ajSXI8+bkNylFCLQpSov52Dg=; b=M QYXrxCVRash1qHr2zKnY2AbrN04OXNRvo3QtDRqNxliU/h1O91GURgM1cki9rp21 NYogsHapxGZPoGTjGEv+CcVDnonZMvIoAXx5OXhf2yymEBT3JrmCJfLrWCwFIbsQ tB50mHEbYcOmPamXztUg5qFQEcgVfANhqVUmcP1FyI= X-Virus-Scanned: amavisd-new at satro.sk Received: from smtp1.satro.sk ([127.0.0.1]) by localhost (smtp1.satro.sk [127.0.0.1]) (amavisd-new, port 10026) with LMTP id P33IgYW5jswt; Tue, 17 Jan 2012 21:41:50 +0100 (CET) Received: from [192.168.0.11] (unknown [178.143.101.64]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp1.satro.sk (Postfix) with ESMTP id D9C021BF95B; Tue, 17 Jan 2012 21:41:50 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: =?utf-8?Q?Michal_Van=C4=8Do?= X-Priority: 3 In-Reply-To: Date: Tue, 17 Jan 2012 21:41:50 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Steven Hartland X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-jail@freebsd.org Subject: Re: mtr doesn't work in a jail even with security.jail.allow_raw_sockets: 1 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2012 21:11:12 -0000 Try with --address option. Address selection doesn't work when mtr is = run within jail. regards michal On 17.1.2012, at 16:32, Steven Hartland wrote: > Wanted to use mtr to diagnose an issue in a jail > but it seems it totally fails even with > security.jail.allow_raw_sockets: 1 >=20 > Any ideas? > Regards > Steve >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > This e.mail is private and confidential between Multiplay (UK) Ltd. = and the person or entity to whom it is addressed. In the event of = misdirection, the recipient is prohibited from using, copying, printing = or otherwise disseminating it or any information contained in it.=20 > In the event of misdirection, illegible or incomplete transmission = please telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. >=20 > _______________________________________________ > freebsd-jail@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to = "freebsd-jail-unsubscribe@freebsd.org" From owner-freebsd-jail@FreeBSD.ORG Tue Jan 17 21:30:58 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9F6B1065670 for ; Tue, 17 Jan 2012 21:30:58 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id 673588FC16 for ; Tue, 17 Jan 2012 21:30:58 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 33BC925D3893; Tue, 17 Jan 2012 21:30:57 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 617B4BD964C; Tue, 17 Jan 2012 21:30:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id FyR2w3d10Hie; Tue, 17 Jan 2012 21:30:55 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 4E3C3BD964B; Tue, 17 Jan 2012 21:30:55 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=utf-8 From: "Bjoern A. Zeeb" X-Priority: 3 In-Reply-To: Date: Tue, 17 Jan 2012 21:30:54 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: =?utf-8?Q?Michal_Van=C4=8Do?= X-Mailer: Apple Mail (2.1084) Cc: freebsd-jail@freebsd.org Subject: Re: mtr doesn't work in a jail even with security.jail.allow_raw_sockets: 1 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2012 21:30:58 -0000 On 17. Jan 2012, at 20:41 , Michal Van=C4=8Do wrote: > Try with --address option. Address selection doesn't work when mtr is = run within jail. I should or should be fixed. >> Wanted to use mtr to diagnose an issue in a jail >> but it seems it totally fails even with >> security.jail.allow_raw_sockets: 1 which version of freebsd? Anything newer than incl. 8.0 the systls are = not what you want anymore; it's per jail flags. /bz --=20 Bjoern A. Zeeb You have to have visions! It does not matter how good you are. It matters what good you do! From owner-freebsd-jail@FreeBSD.ORG Fri Jan 20 08:59:56 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22EAA1065678 for ; Fri, 20 Jan 2012 08:59:56 +0000 (UTC) (envelope-from linuxmail@4lin.net) Received: from mail.4lin.net (mail.4lin.net [IPv6:2a01:4f8:130:6021::50]) by mx1.freebsd.org (Postfix) with ESMTP id 5D3AB8FC13 for ; Fri, 20 Jan 2012 08:59:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.4lin.net (Postfix) with ESMTP id 7800B6904 for ; Fri, 20 Jan 2012 10:01:44 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.4lin.net Received: from mail.4lin.net ([127.0.0.1]) by localhost (mail.4lin.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7E82c0YZVv-W for ; Fri, 20 Jan 2012 10:01:37 +0100 (CET) Received: from pcdenny.rbg.informatik.tu-darmstadt.de (pcdenny.rbg.informatik.tu-darmstadt.de [130.83.160.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.4lin.net (Postfix) with ESMTPSA id BDF963E890 for ; Fri, 20 Jan 2012 10:01:36 +0100 (CET) From: Denny Schierz Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Fri, 20 Jan 2012 09:59:43 +0100 Message-Id: <07AF4A15-D2A2-4736-9596-8C41F93579B0@4lin.net> To: freebsd-jail@freebsd.org Mime-Version: 1.0 (Apple Message framework v1251.1) X-Mailer: Apple Mail (2.1251.1) Subject: Getting Jail v2 working with 9-stable X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2012 08:59:56 -0000 hi, I'm try to get jails with there own network stack working under 9-stable = and have "only" problems with removing / stopping the jail. I wrote most = on the stable list, so I just copy the relevant parts here: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D I've created a new patch (adapted the old freebsd-9RC2 patch) for = /etc/rc.d/jail: The original patch: http://wiki.polymorf.fr/files/jail_rc.patch My patch: http://pastebin.com/9LdLwaNA It works (was very happy) if you start the jail, but has problems with = stopping: it shows in jls still as active: # jls JID IP Address Hostname Path 1 - template.domain /jails/template If I try to remove with "jail -r 1" than first the process hang, second = after while, the whole machine needs a reset. There is no process from = the jail active, nor any epair* interfaces or mounts, which is quite = good, but ... If i try to create the jail again (after /etc/rc.d/jail stop), it tries = to create the epair0a (the last I can see) interface and than it hangs = again -> reset needed Also nice to know: # umount /jails/template=20 umount: unmount of /jails/template failed: Device busy Also not possible: a normal reboot after starting / stopping the jail. = -> reset needed =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D = http://lists.freebsd.org/pipermail/freebsd-stable/2012-January/065556.html= One more thing: If you wait / do nothing (5-15min) after a while the = machine hangs too -> Reset My plattform is a Sun SPARC64 Sunfire v245: 9.0-STABLE FreeBSD = 9.0-STABLE So, maybe there is something missing. My rc.conf: cloned_interfaces=3D"bridge0" ifconfig_bridge0=3D"addm bge0 up" ifconfig_bridge0_alias0=3D"inet CHANGEDnetmask 255.255.255.192 up" ifconfig_bge0=3D"up" defaultrouter=3D"CHANGED" gateway_enable=3D"YES" # Jails jail_enable=3D"NO" jail_v2_enable=3D"YES" jail_list=3D"" jail_sysvipc_allow=3D"YES" =20 for file in /etc/jails/*.conf; do . $file done=20 cat /etc/jails/template.conf=20 #JAIL template jail_list=3D"$jail_list template" jail_template_name=3D"template" jail_template_hostname=3D"template.CHANGED" jail_template_devfs_enable=3D"YES" jail_template_rootdir=3D"/jails/template" jail_template_mount_enable=3D"YES" jail_template_fstab=3D"/etc/jails/fstabs/template" jail_template_vnet_enable=3D"YES" jail_template_flags=3D"-c vnet persist" =20 #network jail_template_exec_prestart0=3D"ifconfig epair0 create" jail_template_exec_prestart1=3D"ifconfig bridge0 addm epair0a" jail_template_exec_prestart2=3D"ifconfig epair0a up" jail_template_exec_earlypoststart0=3D"ifconfig epair0b vnet template" jail_template_exec_afterstart0=3D"ifconfig lo0 127.0.0.1" jail_template_exec_afterstart1=3D"ifconfig epair0b CHANGED netmask = 255.255.255.192 up" jail_template_exec_afterstart2=3D"route add default CHANGED" jail_template_exec_afterstart3=3D"/bin/sh /etc/rc" jail_template_exec_prestop0=3D"/bin/sh /etc/rc.shutdown" jail_template_exec_poststop=3D"ifconfig epair0b destroy" jail_template_exec_poststop0=3D"ifconfig bridge0 deletem epair0a" jail_template_exec_poststop1=3D"ifconfig epair0a destroy" sysctl: security.jail.enforce_statfs: 2 security.jail.mount_allowed: 0 security.jail.chflags_allowed: 0 security.jail.allow_raw_sockets: 1 security.jail.sysvipc_allowed: 1 security.jail.socket_unixiproute_only: 0 security.jail.set_hostname_allowed: 1 security.jail.jail_max_af_ips: 255 security.jail.jailed: 0 Also nothing found with lsof/fuser ... any suggestions? cu denny= From owner-freebsd-jail@FreeBSD.ORG Fri Jan 20 14:06:16 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE2AA1065672 for ; Fri, 20 Jan 2012 14:06:16 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) by mx1.freebsd.org (Postfix) with ESMTP id 663F78FC0A for ; Fri, 20 Jan 2012 14:06:15 +0000 (UTC) Received: from pps.filterd (ltcfislmsgpa05 [127.0.0.1]) by ltcfislmsgpa05.fnfis.com (8.14.4/8.14.4) with SMTP id q0KDPlWV029184; Fri, 20 Jan 2012 08:06:11 -0600 Received: from smtp.fisglobal.com ([10.132.206.31]) by ltcfislmsgpa05.fnfis.com with ESMTP id 12f8yg05um-28 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Fri, 20 Jan 2012 08:06:11 -0600 Received: from [10.0.0.105] (10.14.152.28) by smtp.fisglobal.com (10.132.206.31) with Microsoft SMTP Server (TLS) id 14.1.323.3; Fri, 20 Jan 2012 08:05:38 -0600 References: <07AF4A15-D2A2-4736-9596-8C41F93579B0@4lin.net> In-Reply-To: <07AF4A15-D2A2-4736-9596-8C41F93579B0@4lin.net> MIME-Version: 1.0 (iPhone Mail 8C148) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="us-ascii" Message-ID: <40EF185D-8B72-4F30-90C6-B8DCA8EEDA29@fisglobal.com> X-Mailer: iPhone Mail (8C148) From: Devin Teske Date: Fri, 20 Jan 2012 06:05:28 -0800 To: Denny Schierz X-Originating-IP: [10.14.152.28] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.6.7361, 1.0.211, 0.0.0000 definitions=2012-01-20_03:2012-01-20, 2012-01-20, 1970-01-01 signatures=0 Cc: "" Subject: * Re: Getting Jail v2 working with 9-stable X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2012 14:06:16 -0000 Sent from my iPhone On Jan 20, 2012, at 12:59 AM, Denny Schierz wrote: > hi, >=20 > I'm try to get jails with there own network stack working under 9-stable = and have "only" problems with removing / stopping the jail. Try my vimage rc.d script for this. http://druidbsd.sf.net/vimage.html http://druidbsd.sourceforge.net/download/vimage-1.4.tbz --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-jail@FreeBSD.ORG Sat Jan 21 08:57:39 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37244106564A for ; Sat, 21 Jan 2012 08:57:39 +0000 (UTC) (envelope-from other@ahhyes.net) Received: from srv.ahhyes.net (srv.ahhyes.net [109.169.82.101]) by mx1.freebsd.org (Postfix) with ESMTP id 20DCB8FC0C for ; Sat, 21 Jan 2012 08:57:38 +0000 (UTC) Received: from [10.1.1.1] (helo=ahhyes.net) by srv.ahhyes.net with esmtpa (Exim 4.77 (FreeBSD)) (envelope-from ) id 1RoWOk-0002gv-5Q for freebsd-jail@freebsd.org; Sat, 21 Jan 2012 19:34:11 +1100 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Sat, 21 Jan 2012 19:34:06 +1100 From: other@ahhyes.net To: Message-ID: X-Sender: other@ahhyes.net User-Agent: Roundcube Webmail/0.7 X-SA-Exim-Connect-IP: 10.1.1.1 X-SA-Exim-Mail-From: other@ahhyes.net X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mail.ahhyes.net X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=ham version=3.3.2 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on srv.ahhyes.net) Subject: nat + pf, network weirdness X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2012 08:57:39 -0000 Hi Guys, I am running 9.0-RELEASE on my VPS. I decided to jail a bunch of services that are public facing in an effort to improve security. Firstly a breakdown of how things are setup: srv# ifconfig pflog0: flags=0<> metric 0 mtu 33152 pfsync0: flags=0<> metric 0 mtu 1500 syncpeer: 0.0.0.0 maxupd: 128 lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 xn0: flags=8843 metric 0 mtu 1500 options=503 ether 00:16:3e:85:8a:12 inet 109.IP.IP.IP netmask 0xffffff00 broadcast 109.169.82.255 media: Ethernet manual status: active lo1: flags=8049 metric 0 mtu 16384 options=3 inet 10.1.1.IP netmask 0xffffff00 inet 10.1.1.IP netmask 0xffffffff inet 10.1.1.IP netmask 0xffffffff inet 10.1.1.IP netmask 0xffffffff srv# jls JID IP Address Hostname Path 1 10.1.1.IP www.mydomain.net /somepath/jails/www 2 10.1.1.IP sql.mydomain.net /somepath/jails/db 3 10.1.1.IP ns.mydomain.net /somepath/jails/ns 5 10.1.1.IP mail.mydomain.net /somepath/jails/mail Interface xn0 is my public facing interface, with my public IP. Everything appears to work as it should, I have a PF running on the host with a default deny all policy. I have the following NAT rule in my pf.conf: nat on xn0 from 10.1.1.0/24 to any -> (xn0) This allows my jails to reach the outside world, and I have a bunch of port redirects to direct inbound traffic to the appropriate jail. The issue: There seems to be no ability to firewall the traffic between jails whilst the NAT rule is in place. For example, I can log into my jail for "ns" and telnet port 3306 on the jail for SQL and connect freely. If I remove the nat rule from PF, the jails cannot talk to each other over the network, which is what I expect because I have not specified any filter rules on my pf.conf to allow the traffic. According to the PF manual, the filter rules should still get run after the NAT translation takes place, but it does not. With NAT running, I can put a deny all from 10.1.1.0/24 as my very first filter rule and it will do nothing at all. I am unable to determine what the issue is as I cannot even run tcpdump -i lo1 on the host as it appears there is no traffic at all on that interface??? I have a suspicion that the NAT translation is causing traffic to hit a rule in PF that allows it to pass, but I cannot confirm this as I have no ability to see what's flowing over the lo1 interface, it's apparently silent (bull)... Any ideas? I really want to lock down the communication the jails have. From owner-freebsd-jail@FreeBSD.ORG Sat Jan 21 14:33:32 2012 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F0C47106566C for ; Sat, 21 Jan 2012 14:33:32 +0000 (UTC) (envelope-from artemrts@ukr.net) Received: from ffe8.ukr.net (ffe8.ukr.net [195.214.192.88]) by mx1.freebsd.org (Postfix) with ESMTP id 9D53E8FC08 for ; Sat, 21 Jan 2012 14:33:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Date:Message-Id:From:To:References:In-Reply-To:Subject:Cc:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=LWSwTBGCDzhc7bPmDYwv/LmlrNzg8N1H8ZNqUD/++ZM=; b=jwRi+8sY583wsLtf/TiQ1ra7jGgRq6Nws8rSSm0Iti5JhL5s+C1TdoKG3uEo26X2tn1AG3Pwx5kcMu16+nSNc1qaRsJzxBcENnf1tnBVec4vFJZAeYBnNCyajxyL7XpqC+wr7a+mwlqaYjqpM6ybKD+hW7XHvUhBYJ4Oz7FqBzs=; Received: from mail by ffe8.ukr.net with local ID 1Robhe-000612-Ce ; Sat, 21 Jan 2012 16:13:58 +0200 MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: binary Content-Type: text/plain; charset="windows-1251" In-Reply-To: References: To: other@ahhyes.net From: =?WINDOWS-1251?B?wujy4Ovo6SDC6+Dk6Ozo8O7i6Pc=?= X-Mailer: freemail.ukr.net 4.0 X-Originating-Ip: [195.200.251.83] Message-Id: <22966.1327155238.9808034899287998464@ffe8.ukr.net> X-Browser: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 Date: Sat, 21 Jan 2012 16:13:58 +0200 Cc: freebsd-jail@freebsd.org Subject: Re: nat + pf, network weirdness X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2012 14:33:33 -0000 --- Original message --- From: other@ahhyes.net To: freebsd-jail@freebsd.org Date: 21 January 2012, 10:57:48 Subject: nat + pf, network weirdness > Hi Guys, > > I am running 9.0-RELEASE on my VPS. I decided to jail a bunch of > services that are public facing in an effort to improve security. > > Firstly a breakdown of how things are setup: > > srv# ifconfig > pflog0: flags=0<> metric 0 mtu 33152 > pfsync0: flags=0<> metric 0 mtu 1500 > syncpeer: 0.0.0.0 maxupd: 128 > lo0: flags=8049 metric 0 mtu 16384 > options=3 > inet 127.0.0.1 netmask 0xff000000 > xn0: flags=8843 metric 0 mtu > 1500 > options=503 > ether 00:16:3e:85:8a:12 > inet 109.IP.IP.IP netmask 0xffffff00 broadcast 109.169.82.255 > media: Ethernet manual > status: active > lo1: flags=8049 metric 0 mtu 16384 > options=3 > inet 10.1.1.IP netmask 0xffffff00 > inet 10.1.1.IP netmask 0xffffffff > inet 10.1.1.IP netmask 0xffffffff > inet 10.1.1.IP netmask 0xffffffff > > srv# jls > JID IP Address Hostname Path > 1 10.1.1.IP www.mydomain.net > /somepath/jails/www > 2 10.1.1.IP sql.mydomain.net > /somepath/jails/db > 3 10.1.1.IP ns.mydomain.net > /somepath/jails/ns > 5 10.1.1.IP mail.mydomain.net > /somepath/jails/mail > > Interface xn0 is my public facing interface, with my public IP. > > Everything appears to work as it should, I have a PF running on the > host with a default deny all policy. I have the following NAT rule in my > pf.conf: > > nat on xn0 from 10.1.1.0/24 to any -> (xn0) > You should use Packet Tagging (Policy Filtering). Something like this: nat on $ext_if tag WWW tagged WWW -> ($ext_if) nat on $ext_if tag SQL tagged SQL -> ($ext_if) ...... block in block out pass in quick on lo1 inet from 10.1.1.1 to !(self) tag WWW <- mark traffic from jail to world ..... pass out quick on $ext_if inet from ($ext_if) tagged WWW <- dispatch only marked WWW PF is very well in situations like this. With PF it is possible to divide LAN traffic and router traffic easily.