From owner-freebsd-jail@FreeBSD.ORG Sun Apr 28 15:54:50 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6F2FCC40 for ; Sun, 28 Apr 2013 15:54:50 +0000 (UTC) (envelope-from fbsd8@a1poweruser.com) Received: from mail-03.name-services.com (mail-03.name-services.com [69.64.155.195]) by mx1.freebsd.org (Postfix) with ESMTP id 5D95F1ABB for ; Sun, 28 Apr 2013 15:54:50 +0000 (UTC) Received: from [10.0.10.1] ([173.88.202.176]) by mail-03.name-services.com with Microsoft SMTPSVC(6.0.3790.4675); Sun, 28 Apr 2013 08:54:51 -0700 Message-ID: <517D464A.7050101@a1poweruser.com> Date: Sun, 28 Apr 2013 11:54:50 -0400 From: Joe User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: zulu Subject: Re: state of the art ? References: <5177B1A4.6060502@free.fr> <1366868448.5178c1e04043f@gpo.cellcontainer.com> In-Reply-To: <1366868448.5178c1e04043f@gpo.cellcontainer.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 28 Apr 2013 15:54:51.0385 (UTC) FILETIME=[B7D45690:01CE4428] X-Sender: fbsd8@a1poweruser.com X-Authenticated-Sender: fbsd8@a1poweruser.com X-EchoSenderHash: [fbsd8]-[a1poweruser*com] Cc: Laurent Alebarde , "freebsd-jail@freebsd.org" X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2013 15:54:50 -0000 zulu wrote: > > Maybe this is what you need http://sourceforge.net/projects/zjails/ , > doesn't require any advanced ZFS or VNET knowledge (just a working ZFS > pool and VIMAGE kernel). > > VNET is supported and there is a "soft" jail restart option which > prevents the "kern/164763: Memory leak in VNET" issue from appearing. > > You can also run non VNET ZFS jails - you can turn on or off VNET by > simply executing "zjail set vnet=off/on myjailname" then restarting > the jail with "zjail restart -c myjailname". > > On FreeBSD 9.1 amd64, pf inside a jail will cause an immediate kernel > panic once you run pfctl in the jail - IPFW works as already stated by > others. > > You can have pf enabled on the host however and have IPFW firewall in > jails. > > Cheers, > > Peter > What exactly do you mean by ipfw will run in a vimage jail? Running a "open" ipfw rule set only proves the the ipfw program will run in a vimage jail. How about the "simple" or "client" types that need the outbound interface device name and use divert / nat? From owner-freebsd-jail@FreeBSD.ORG Mon Apr 29 11:06:47 2013 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 107E23B3 for ; Mon, 29 Apr 2013 11:06:47 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 026781920 for ; Mon, 29 Apr 2013 11:06:47 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3TB6kKg018172 for ; Mon, 29 Apr 2013 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3TB6kT2018170 for freebsd-jail@FreeBSD.org; Mon, 29 Apr 2013 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 29 Apr 2013 11:06:46 GMT Message-Id: <201304291106.r3TB6kT2018170@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-jail@FreeBSD.org Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 11:06:47 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/176112 jail [jail] [panic] kernel panic when starting jails o kern/176092 jail [jail] [panic] Starting a jail on my releng/9.1 kernel o kern/174902 jail [jail] jail should provide validator for jail names o kern/174436 jail [jail] Jails with numbers as names don't work o bin/173469 jail [jail] regression: security.jail.sysvipc_allowed=1 no o kern/169751 jail [jail] reading routing information does not work in ja o bin/167911 jail new jail(8) problem with removal, ifconfg -alias and k o kern/159918 jail [jail] inter-jail communication failure o kern/156111 jail [jail] procstat -b not supported in jail o misc/155765 jail [patch] `buildworld' does not honors WITHOUT_JAIL o conf/154246 jail [jail] [patch] Bad symlink created if devfs mount poin o conf/149050 jail [jail] rcorder ``nojail'' too coarse for Jail+VNET s conf/142972 jail [jail] [patch] Support JAILv2 and vnet in rc.d/jail o conf/141317 jail [patch] uncorrect jail stop in /etc/rc.d/jail o kern/133265 jail [jail] is there a solution how to run nfs client in ja o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid 17 problems total. From owner-freebsd-jail@FreeBSD.ORG Wed May 1 00:17:06 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A5E41F43 for ; Wed, 1 May 2013 00:17:06 +0000 (UTC) (envelope-from fbsd8@a1poweruser.com) Received: from mail-03.name-services.com (mail-03.name-services.com [69.64.155.195]) by mx1.freebsd.org (Postfix) with ESMTP id 929CF1037 for ; Wed, 1 May 2013 00:17:06 +0000 (UTC) Received: from [10.0.10.1] ([173.88.202.176]) by mail-03.name-services.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 30 Apr 2013 17:17:01 -0700 Message-ID: <51805EFB.6050806@a1poweruser.com> Date: Tue, 30 Apr 2013 20:16:59 -0400 From: Joe User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: freebsd-jail Subject: vnet jail with ipfw having logging problem References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> In-Reply-To: <517D3426.1090703@a1poweruser.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 May 2013 00:17:01.0818 (UTC) FILETIME=[33CB0DA0:01CE4601] X-Sender: fbsd8@a1poweruser.com X-Authenticated-Sender: fbsd8@a1poweruser.com X-EchoSenderHash: [fbsd8]-[a1poweruser*com] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 00:17:06 -0000 I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using the jail(8) definition statements for starting and stopping the vnet jail. As a side note non-vnet jails are working as expected. The host is running a custom kernel with modules and with options VIMAGE nooptions SCTP options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=10 options IPFIREWALL_DEFAULT_TO_ACCEPT options IPFIREWALL_IPDIVERT options IPFIREWALL_FORWARD compiled in. The host is also running ipfw from its rc.conf file. Both the vnet jail and the host, have ipfw rules, for logging everything. Host ipfw -q add 010 allow all from any to any via lo0 ipfw -q add 010 allow log all from any to any via rl0 vnet jail # pass the vnet epairXb to the jail rule. if [ -e /etc/epair ]; then pif=`cat "/etc/epair"` else pif="lo0" fi ipfw -q add 010 allow all from any to any via lo0 ipfw -q add 010 allow log all from any to any via $pif I work around the (nojail keyword problem with the rc.d startup scripts) by manually issuing service netif start, service routing start, service ipfw start commands, after the exec.start="/bin/sh /etc/rc" is run and issue them in reverse order before the exec.stop="/bin/sh /etc/rc.shutdown" is executed. After booting the system I can ping the internet from the host and see the rule counter increase using this command "ipfw -a list". I also see the ping packets logged in the hosts /var/log/security file. After the vnet jail is started I see a empty /var/log/security file inside of the vnet jail that never gets populated. But in the hosts /var/log/security file I see log messages from the vnet jail. I would expect to see the vnet jail log message interspersed with the host logging messages as the jails packets pass through the hosts ipfw firewall. But I only see the vnet jails ipfw logging messages in the hosts /var/log/security file from that point on. I can stop the vnet jail and restart it and the ipfw logged messages continue to populate the hosts security file. With the vnet jail stopped, I issue ping from host to internet and get 100% packets replied message. The ipfw -a list command from the host shows the rule count has increased, but there are no host packets logged to the hosts security file. Rebooting the host is the only way to get the host to log ipfw packets and this only works until the vent jail starts after which time the host no longer logs packets. I can comment out the firewall statements in the hosts rc.conf and reboot the host without ipfw, but since ipfw is compiled into the kernel ipfw is really running on the host with the allow all default and the started vmet/ipfw jail still populates the hosts security file. I'm thinking this is a bug. I would like conformation of this problem. Or maybe someone has some other (nojail keyword work around method) that results in the vnet/ipfw jail logging to the jails security file and the hosts ipfw logging to it's security file that they would share with me. Thanks Joe From owner-freebsd-jail@FreeBSD.ORG Wed May 1 16:51:52 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BAB7AE4A for ; Wed, 1 May 2013 16:51:52 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id 370721904 for ; Wed, 1 May 2013 16:51:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id r41GUJ6n031788; Thu, 2 May 2013 02:30:19 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Thu, 2 May 2013 02:30:19 +1000 (EST) From: Ian Smith To: Joe Subject: Re: vnet jail with ipfw having logging problem In-Reply-To: <51805EFB.6050806@a1poweruser.com> Message-ID: <20130502021830.O30818@sola.nimnet.asn.au> References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-jail X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 16:51:52 -0000 On Tue, 30 Apr 2013 20:16:59 -0400, Joe wrote: > I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using the > jail(8) definition statements for starting and stopping the vnet jail. As a > side note non-vnet jails are working as expected. > > The host is running a custom kernel with modules and with > options VIMAGE > nooptions SCTP > options IPFIREWALL > options IPFIREWALL_VERBOSE > options IPFIREWALL_VERBOSE_LIMIT=10 What steps have you taken during testing to override this ridiculously low limit on logging? Otherwise, after e.g. just 5 pings and 5 ping responses are logged, all logging ceases until issuing 'ipfw resetlog'. > options IPFIREWALL_DEFAULT_TO_ACCEPT > options IPFIREWALL_IPDIVERT You'd likely do better using in-kernel NAT; natd doesn't get much love. > options IPFIREWALL_FORWARD > > compiled in. Ian From owner-freebsd-jail@FreeBSD.ORG Wed May 1 21:43:06 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 54DC03C1 for ; Wed, 1 May 2013 21:43:06 +0000 (UTC) (envelope-from fbsd8@a1poweruser.com) Received: from mail-03.name-services.com (mail-03.name-services.com [69.64.155.195]) by mx1.freebsd.org (Postfix) with ESMTP id 40AED1513 for ; Wed, 1 May 2013 21:43:06 +0000 (UTC) Received: from [10.0.10.1] ([173.88.202.176]) by mail-03.name-services.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 1 May 2013 14:43:07 -0700 Message-ID: <51818C67.7070708@a1poweruser.com> Date: Wed, 01 May 2013 17:43:03 -0400 From: Joe User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Ian Smith Subject: Re: vnet jail with ipfw having logging problem References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> In-Reply-To: <20130502021830.O30818@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 May 2013 21:43:07.0582 (UTC) FILETIME=[DE2C35E0:01CE46B4] X-Sender: fbsd8@a1poweruser.com X-Authenticated-Sender: fbsd8@a1poweruser.com X-EchoSenderHash: [fbsd8]-[a1poweruser*com] Cc: freebsd-jail X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 21:43:06 -0000 > > I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using the > > jail(8) definition statements for starting and stopping the vnet jail. As a > > side note non-vnet jails are working as expected. > > > > The host is running a custom kernel with modules and with > > options VIMAGE > > nooptions SCTP > > options IPFIREWALL > > options IPFIREWALL_VERBOSE > > options IPFIREWALL_VERBOSE_LIMIT=10 > > What steps have you taken during testing to override this ridiculously > low limit on logging? Otherwise, after e.g. just 5 pings and 5 ping > responses are logged, all logging ceases until issuing 'ipfw resetlog'. /usr/src/sys/conf/NOTES says IPFIREWALL_VERBOSE_LIMIT; limits the number of times a matching entry can be logged. Says nothing about this limit being the maximum number of log records allowed after which the log file is closed for business. Are you saying the /usr/src/sys/conf/NOTES info is no longer true? Without IPFIREWALL_VERBOSE and IPFIREWALL_VERBOSE_LIMIT where does the logged packets get written to? /var/log/security I have not used ipfw since it's ipfw2 rewrite so my knowledge is dated. > > > options IPFIREWALL_DEFAULT_TO_ACCEPT > > options IPFIREWALL_IPDIVERT > > You'd likely do better using in-kernel NAT; natd doesn't get much love. > I kept getting kernel compile errors using "options IPFIREWALL_NAT". I thought the error was caused by vimage. Now I know "options LIBALIAS" is required. Could not find info on internet search for IPFIREWALL_NAT with vimage kernel. Do you have first hand experience getting "ipfw kernel nat" to work in a vimage jail or having logging work on the host and within the vnet jail? From owner-freebsd-jail@FreeBSD.ORG Wed May 1 22:15:25 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 55B6C322 for ; Wed, 1 May 2013 22:15:25 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) by mx1.freebsd.org (Postfix) with ESMTP id E749B1619 for ; Wed, 1 May 2013 22:15:24 +0000 (UTC) Received: from mailout-eu.gmx.com ([10.1.101.212]) by mrigmx.server.lan (mrigmx001) with ESMTP (Nemesis) id 0M2Jbe-1UH6KA26mS-00s5VN for ; Thu, 02 May 2013 00:15:18 +0200 Received: (qmail invoked by alias); 01 May 2013 22:15:18 -0000 Received: from vpn11.hotsplots.net (EHLO [192.168.44.194]) [80.237.194.100] by mail.gmx.com (mp-eu012) with SMTP; 02 May 2013 00:15:18 +0200 X-Authenticated: #46156728 X-Provags-ID: V01U2FsdGVkX19ov1pYzWxor1YijCRCuCmrrAO+gW8WLS/bzSeQ3/ 2MnkQ9Jdu/EM2S Message-ID: <518193F1.5080501@gmx.com> Date: Thu, 02 May 2013 00:15:13 +0200 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130329 Thunderbird/17.0.5 MIME-Version: 1.0 To: Joe Subject: Re: vnet jail with ipfw having logging problem References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> In-Reply-To: <51818C67.7070708@a1poweruser.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: freebsd-jail , Ian Smith X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2013 22:15:25 -0000 On 05/01/2013 11:43 PM, Joe wrote: > Do you have first hand experience getting "ipfw kernel nat" to work in a > vimage jail ipfw nat should work in a vnet jail. HTH, Nikos From owner-freebsd-jail@FreeBSD.ORG Thu May 2 05:42:06 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8D3731F6 for ; Thu, 2 May 2013 05:42:06 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id D4AE31554 for ; Thu, 2 May 2013 05:42:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id r425g0vP059747; Thu, 2 May 2013 15:42:01 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Thu, 2 May 2013 15:42:00 +1000 (EST) From: Ian Smith To: Joe Subject: Re: vnet jail with ipfw having logging problem In-Reply-To: <51818C67.7070708@a1poweruser.com> Message-ID: <20130502142443.V30818@sola.nimnet.asn.au> References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-jail X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 05:42:06 -0000 On Wed, 1 May 2013 17:43:03 -0400, Joe wrote: > > > I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using > > the > > > jail(8) definition statements for starting and stopping the vnet jail. > > As a > > > side note non-vnet jails are working as expected. > > > > The host is running a custom kernel with modules and with > > > options VIMAGE > > > nooptions SCTP > > > options IPFIREWALL > > > options IPFIREWALL_VERBOSE > > > options IPFIREWALL_VERBOSE_LIMIT=10 Please maintain attributions for the archives. I wrote: > > What steps have you taken during testing to override this ridiculously low > > limit on logging? Otherwise, after e.g. just 5 pings and 5 ping responses > > are logged, all logging ceases until issuing 'ipfw resetlog'. > > /usr/src/sys/conf/NOTES says IPFIREWALL_VERBOSE_LIMIT; limits the number of > times a matching entry can be logged. Says nothing about this limit being the > maximum number of log records allowed after which the log file is closed for > business. Are you saying the /usr/src/sys/conf/NOTES info is no longer true? You showed one (1) 'log' rule for each of the host's and jail's ruleset. Once that one rule has been logged 'logamount' times (default as per NOTES is 100, but in your case is 10) then logging for THAT rule stops, therefore with only one 'log' rule, ALL logging stops. Understand? If you take the time to properly study the correct reference, ipfw(8), all of this will become clear. See especially section SYSCTL VARIABLES, and read thoroughly 'log [logamount number]', at the very least. Ignore the Handbook section on ipfw, it's full of errors and misunderstandings. > Without IPFIREWALL_VERBOSE and IPFIREWALL_VERBOSE_LIMIT where does the logged > packets get written to? /var/log/security See above. Both of these options merely set defaults for the sysctls. > I have not used ipfw since it's ipfw2 rewrite so my knowledge is dated. Indeed it is; that's a very long time ago. > > > options IPFIREWALL_DEFAULT_TO_ACCEPT > > > options IPFIREWALL_IPDIVERT > > > > You'd likely do better using in-kernel NAT; natd doesn't get much love. > > > > I kept getting kernel compile errors using "options IPFIREWALL_NAT". I > thought the error was caused by vimage. Now I know "options LIBALIAS" is > required. Could not find info on internet search for IPFIREWALL_NAT with > vimage kernel. Apart from FIREWALL_FORWARD (not even that in 10.x), none of that needs to be in the kernel, it's all loadable as modules; see /etc/rc.d/ipfw. If you're doing NAT in the vimage jail, you must have at least two interfaces assigned to the jail. Care to show your config for that? > Do you have first hand experience getting "ipfw kernel nat" to work in a > vimage jail or having logging work on the host and within the vnet jail? No, but I have just on 15 years experience managing ipfw firewalls :) Ian From owner-freebsd-jail@FreeBSD.ORG Thu May 2 10:09:18 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1128051D for ; Thu, 2 May 2013 10:09:18 +0000 (UTC) (envelope-from anders.hagman@netplex.se) Received: from smtp-out11.han.skanova.net (smtp-out11.han.skanova.net [195.67.226.200]) by mx1.freebsd.org (Postfix) with ESMTP id A63A7128D for ; Thu, 2 May 2013 10:09:17 +0000 (UTC) Received: from macen.halleforshunden.org (31.210.252.116) by smtp-out11.han.skanova.net (8.5.133) (authenticated as u48002568) id 516D088C006B7E16 for freebsd-jail@freebsd.org; Thu, 2 May 2013 12:09:09 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: vnet jail with ipfw having logging problem From: Anders Hagman In-Reply-To: <20130502142443.V30818@sola.nimnet.asn.au> Date: Thu, 2 May 2013 12:09:08 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> <20130502142443.V30818@sola.nimnet.asn.au> To: freebsd-jail X-Mailer: Apple Mail (2.1503) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 10:09:18 -0000 Hi 2 maj 2013 kl. 07:42 skrev Ian Smith : > On Wed, 1 May 2013 17:43:03 -0400, Joe wrote: >>>> I have ipfw running inside of a vnet jail on a 9.1-RELEASE host = using >>> the >>>> jail(8) definition statements for starting and stopping the vnet = jail. >>> As a >>>> side note non-vnet jails are working as expected. >>>>> The host is running a custom kernel with modules and with >>>> options VIMAGE >>>> nooptions SCTP >>>> options IPFIREWALL >>>> options IPFIREWALL_VERBOSE >>>> options IPFIREWALL_VERBOSE_LIMIT=3D10 >=20 > Please maintain attributions for the archives. I wrote: >=20 >>> What steps have you taken during testing to override this = ridiculously low >>> limit on logging? Otherwise, after e.g. just 5 pings and 5 ping = responses >>> are logged, all logging ceases until issuing 'ipfw resetlog'. >>=20 >> /usr/src/sys/conf/NOTES says IPFIREWALL_VERBOSE_LIMIT; limits the = number of >> times a matching entry can be logged. Says nothing about this limit = being the >> maximum number of log records allowed after which the log file is = closed for >> business. Are you saying the /usr/src/sys/conf/NOTES info is no = longer true? >=20 > You showed one (1) 'log' rule for each of the host's and jail's = ruleset.=20 > Once that one rule has been logged 'logamount' times (default as per=20= > NOTES is 100, but in your case is 10) then logging for THAT rule = stops,=20 > therefore with only one 'log' rule, ALL logging stops. Understand? >=20 > If you take the time to properly study the correct reference, ipfw(8),=20= > all of this will become clear. See especially section SYSCTL = VARIABLES,=20 > and read thoroughly 'log [logamount number]', at the very least. = Ignore=20 > the Handbook section on ipfw, it's full of errors and = misunderstandings. >=20 >> Without IPFIREWALL_VERBOSE and IPFIREWALL_VERBOSE_LIMIT where does = the logged >> packets get written to? /var/log/security >=20 > See above. Both of these options merely set defaults for the sysctls. >=20 >> I have not used ipfw since it's ipfw2 rewrite so my knowledge is = dated. >=20 > Indeed it is; that's a very long time ago. >=20 >>>> options IPFIREWALL_DEFAULT_TO_ACCEPT >>>> options IPFIREWALL_IPDIVERT >>>=20 >>> You'd likely do better using in-kernel NAT; natd doesn't get much = love. >>>=20 >>=20 >> I kept getting kernel compile errors using "options IPFIREWALL_NAT". = I >> thought the error was caused by vimage. Now I know "options LIBALIAS" = is >> required. Could not find info on internet search for IPFIREWALL_NAT = with >> vimage kernel. >=20 > Apart from FIREWALL_FORWARD (not even that in 10.x), none of that = needs=20 > to be in the kernel, it's all loadable as modules; see /etc/rc.d/ipfw. >=20 > If you're doing NAT in the vimage jail, you must have at least two=20 > interfaces assigned to the jail. Care to show your config for that? >=20 >> Do you have first hand experience getting "ipfw kernel nat" to work = in a >> vimage jail or having logging work on the host and within the vnet = jail? >=20 > No, but I have just on 15 years experience managing ipfw firewalls :) When you are new at things you do mistakes, remember. To try to answer Joes question: You don't need to compile anything into the kernel regarding ipfw. Just load the ipfw module in the host system with: kldload ipfw By default a deny all rule is added, so add a allow rule to the host = system. ipfw add 10 allow ip from any to any To log things you change the sysctl value net.inet.ip.fw.verbose to 1 sysctl net.inet.ip.fw.verbose=3D1 If you keep net.inet.ip.fw.verbose_limit=3D0 you don't have a log limit, = and for tests thats fine. log in to the jail system. Change the sysctl value = net.inet.ip.fw.verbose to 1 sysctl net.inet.ip.fw.verbose=3D1 Add a logging firewall rule ipfw add 10 allow log ip from any to any Do a ping to an external system. Look inside /var/log/security in the jail system and its empty. Go to the main host and look at the /var/log/security file and you will = find log entries. I can confirm Joes bug. I don't have a log rule in the main host but = still get log messages. All log messages are from the log rule in the jail system. System used: 9.1-RELEASE-p2 BR /Anders= From owner-freebsd-jail@FreeBSD.ORG Thu May 2 13:49:59 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 158DDEBA for ; Thu, 2 May 2013 13:49:59 +0000 (UTC) (envelope-from fbsd8@a1poweruser.com) Received: from mail-03.name-services.com (mail-03.name-services.com [69.64.155.195]) by mx1.freebsd.org (Postfix) with ESMTP id E4A941DCD for ; Thu, 2 May 2013 13:49:58 +0000 (UTC) Received: from [10.0.10.1] ([173.88.202.176]) by mail-03.name-services.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 2 May 2013 06:49:51 -0700 Message-ID: <51826EF7.30302@a1poweruser.com> Date: Thu, 02 May 2013 09:49:43 -0400 From: Joe User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Anders Hagman Subject: Re: vnet jail with ipfw having logging problem References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> <20130502142443.V30818@sola.nimnet.asn.au> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 May 2013 13:49:51.0366 (UTC) FILETIME=[EB1F4260:01CE473B] X-Sender: fbsd8@a1poweruser.com X-Authenticated-Sender: fbsd8@a1poweruser.com X-EchoSenderHash: [fbsd8]-[a1poweruser*com] Cc: freebsd-jail X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 13:49:59 -0000 Anders Hagman wrote: > Hi > > 2 maj 2013 kl. 07:42 skrev Ian Smith : > >> On Wed, 1 May 2013 17:43:03 -0400, Joe wrote: >>>>> I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using >>>> the >>>>> jail(8) definition statements for starting and stopping the vnet jail. >>>> As a >>>>> side note non-vnet jails are working as expected. >>>>>> The host is running a custom kernel with modules and with >>>>> options VIMAGE >>>>> nooptions SCTP >>>>> options IPFIREWALL >>>>> options IPFIREWALL_VERBOSE >>>>> options IPFIREWALL_VERBOSE_LIMIT=10 >> Please maintain attributions for the archives. I wrote: >> >>>> What steps have you taken during testing to override this ridiculously low >>>> limit on logging? Otherwise, after e.g. just 5 pings and 5 ping responses >>>> are logged, all logging ceases until issuing 'ipfw resetlog'. >>> /usr/src/sys/conf/NOTES says IPFIREWALL_VERBOSE_LIMIT; limits the number of >>> times a matching entry can be logged. Says nothing about this limit being the >>> maximum number of log records allowed after which the log file is closed for >>> business. Are you saying the /usr/src/sys/conf/NOTES info is no longer true? >> You showed one (1) 'log' rule for each of the host's and jail's ruleset. >> Once that one rule has been logged 'logamount' times (default as per >> NOTES is 100, but in your case is 10) then logging for THAT rule stops, >> therefore with only one 'log' rule, ALL logging stops. Understand? >> >> If you take the time to properly study the correct reference, ipfw(8), >> all of this will become clear. See especially section SYSCTL VARIABLES, >> and read thoroughly 'log [logamount number]', at the very least. Ignore >> the Handbook section on ipfw, it's full of errors and misunderstandings. >> >>> Without IPFIREWALL_VERBOSE and IPFIREWALL_VERBOSE_LIMIT where does the logged >>> packets get written to? /var/log/security >> See above. Both of these options merely set defaults for the sysctls. >> >>> I have not used ipfw since it's ipfw2 rewrite so my knowledge is dated. >> Indeed it is; that's a very long time ago. >> >>>>> options IPFIREWALL_DEFAULT_TO_ACCEPT >>>>> options IPFIREWALL_IPDIVERT >>>> You'd likely do better using in-kernel NAT; natd doesn't get much love. >>>> >>> I kept getting kernel compile errors using "options IPFIREWALL_NAT". I >>> thought the error was caused by vimage. Now I know "options LIBALIAS" is >>> required. Could not find info on internet search for IPFIREWALL_NAT with >>> vimage kernel. >> Apart from FIREWALL_FORWARD (not even that in 10.x), none of that needs >> to be in the kernel, it's all loadable as modules; see /etc/rc.d/ipfw. >> >> If you're doing NAT in the vimage jail, you must have at least two >> interfaces assigned to the jail. Care to show your config for that? >> >>> Do you have first hand experience getting "ipfw kernel nat" to work in a >>> vimage jail or having logging work on the host and within the vnet jail? >> No, but I have just on 15 years experience managing ipfw firewalls :) > > When you are new at things you do mistakes, remember. > > To try to answer Joes question: > > You don't need to compile anything into the kernel regarding ipfw. > > Just load the ipfw module in the host system with: > > kldload ipfw > > By default a deny all rule is added, so add a allow rule to the host system. > > ipfw add 10 allow ip from any to any > > To log things you change the sysctl value net.inet.ip.fw.verbose to 1 > > sysctl net.inet.ip.fw.verbose=1 > > If you keep net.inet.ip.fw.verbose_limit=0 you don't have a log limit, and for tests thats fine. > > log in to the jail system. Change the sysctl value net.inet.ip.fw.verbose to 1 > > sysctl net.inet.ip.fw.verbose=1 > > Add a logging firewall rule > > ipfw add 10 allow log ip from any to any > > Do a ping to an external system. > Look inside /var/log/security in the jail system and its empty. > Go to the main host and look at the /var/log/security file and you will find log entries. > > I can confirm Joes bug. I don't have a log rule in the main host but still get log messages. > All log messages are from the log rule in the jail system. > > System used: 9.1-RELEASE-p2 > > BR > /Anders Thank you Anders, your reply was direct and to the point. Lets talk about this bug. The console.log parameter creates a log file in the hosts /var/log directory for each jail. I would think the ipfw log file should behave the same way. IE: the host ipfw log should be going to the hosts /var/log/security file with each ipfw jail creating it's own /var/log/jailname.security file on the host and not create a /var/log/security file in the jails filesystem. I searched the PR database for any PR's with vnet or vimage and ipfw logging and came up with no hits. Should I submit a PR about this problem? I tested doing a kldload ipfw and fall into the default deny problem. Is there a sysctl to flip the default deny to default accept? Thanks Joe From owner-freebsd-jail@FreeBSD.ORG Thu May 2 14:30:01 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1491B557 for ; Thu, 2 May 2013 14:30:01 +0000 (UTC) (envelope-from feld@feld.me) Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) by mx1.freebsd.org (Postfix) with ESMTP id D9B611FFA for ; Thu, 2 May 2013 14:30:00 +0000 (UTC) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 73B322088 for ; Thu, 2 May 2013 10:29:59 -0400 (EDT) Received: from frontend2.nyi.mail.srv.osa ([10.202.2.161]) by compute5.internal (MEProxy); Thu, 02 May 2013 10:29:59 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=feld.me; h= content-type:to:subject:references:date:mime-version :content-transfer-encoding:from:message-id:in-reply-to; s= mesmtp; bh=gYjrriCbi+be06udOo4RJpdRILk=; b=Gg7pVuTf/baadbyVOGJ7Y rv5KFBF08h6AcopndnMhF8cP1FYHxz/CMRn97cxigH4mdR0Qv8WwczTYkpJbRZ2r uxn53JIXQCgpBpeI3sidykv8ZqvZVN0LZLuY2evzXDb23HpCSgvgBBWLVkYQvYYz 8FNNVxJG1uo+F1YQN/PWq0= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:to:subject:references:date :mime-version:content-transfer-encoding:from:message-id :in-reply-to; s=smtpout; bh=gYjrriCbi+be06udOo4RJpdRILk=; b=rNXg BuV4Ixo6v4k8muviKHsUR8V4lq2xdzYqyc3/XSC9m2ua44v0qoh4esiK1KFDfVsT Q0k1h4EefRhawsaON4gHQJX/cT4psfp+ijkTT7HbOl7AF+E1TBwnX74UI+dvpNFK 9+cpS09Gqz08yS00cbcL79jnjw/lw9jBDKiO3hA= X-Sasl-enc: nKhojKzNBlySKZgiSpgoEAOJkGNqilHmrQEUTvFHi6xT 1367504999 Received: from tech304.office.supranet.net (unknown [66.170.8.18]) by mail.messagingengine.com (Postfix) with ESMTPA id 1324220016C for ; Thu, 2 May 2013 10:29:59 -0400 (EDT) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-jail@freebsd.org Subject: Re: vnet jail with ipfw having logging problem References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> <20130502142443.V30818@sola.nimnet.asn.au> <51826EF7.30302@a1poweruser.com> Date: Thu, 02 May 2013 09:29:58 -0500 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Mark Felder" Message-ID: In-Reply-To: <51826EF7.30302@a1poweruser.com> User-Agent: Opera Mail/12.14 (FreeBSD) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 14:30:01 -0000 On Thu, 02 May 2013 08:49:43 -0500, Joe wrote: > > I tested doing a kldload ipfw and fall into the default deny problem. > Is there a sysctl to flip the default deny to default accept? > options IPFIREWALL_DEFAULT_TO_ACCEPT when you build the kernel is the only way AFAIK From owner-freebsd-jail@FreeBSD.ORG Thu May 2 14:59:47 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AE0A5FCC for ; Thu, 2 May 2013 14:59:47 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id 24E011186 for ; Thu, 2 May 2013 14:59:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id r42ExcaA078398; Fri, 3 May 2013 00:59:39 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Fri, 3 May 2013 00:59:38 +1000 (EST) From: Ian Smith To: Mark Felder Subject: Re: vnet jail with ipfw having logging problem In-Reply-To: Message-ID: <20130503004508.L30818@sola.nimnet.asn.au> References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> <20130502142443.V30818@sola.nimnet.asn.au> <51826EF7.30302@a1poweruser.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-jail@freebsd.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 14:59:47 -0000 On Thu, 2 May 2013 09:29:58 -0500, Mark Felder wrote: > On Thu, 02 May 2013 08:49:43 -0500, Joe wrote: > > > > I tested doing a kldload ipfw and fall into the default deny problem. > > Is there a sysctl to flip the default deny to default accept? > > > > options IPFIREWALL_DEFAULT_TO_ACCEPT when you build the kernel is the only > way AFAIK % man ipfw /FINE POINTS [..] o If you are logged in over a network, loading the kld(4) version of ipfw is probably not as straightforward as you would think. The fol- lowing command line is recommended: kldload ipfw && \ ipfw add 32000 allow ip from any to any Along the same lines, doing an ipfw flush in similar surroundings is also a bad idea. I expect running jexec(8) qualifies as 'logged in over a network' here? cheers, Ian From owner-freebsd-jail@FreeBSD.ORG Thu May 2 16:46:20 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 521936DD for ; Thu, 2 May 2013 16:46:20 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id 691DA18BF for ; Thu, 2 May 2013 16:46:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id r42GkBpv081883; Fri, 3 May 2013 02:46:11 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Fri, 3 May 2013 02:46:10 +1000 (EST) From: Ian Smith To: Anders Hagman Subject: Re: vnet jail with ipfw having logging problem In-Reply-To: Message-ID: <20130503010007.C30818@sola.nimnet.asn.au> References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> <20130502142443.V30818@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-jail X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 16:46:20 -0000 On Thu, 2 May 2013 12:09:08 +0200, Anders Hagman wrote: > Hi Yo > 2 maj 2013 kl. 07:42 skrev Ian Smith : > > > On Wed, 1 May 2013 17:43:03 -0400, Joe wrote: > >>>> I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using > >>> the > >>>> jail(8) definition statements for starting and stopping the vnet jail. > >>> As a > >>>> side note non-vnet jails are working as expected. > >>>>> The host is running a custom kernel with modules and with > >>>> options VIMAGE > >>>> nooptions SCTP > >>>> options IPFIREWALL > >>>> options IPFIREWALL_VERBOSE > >>>> options IPFIREWALL_VERBOSE_LIMIT=10 > > > > Please maintain attributions for the archives. I wrote: > > > >>> What steps have you taken during testing to override this ridiculously low > >>> limit on logging? Otherwise, after e.g. just 5 pings and 5 ping responses > >>> are logged, all logging ceases until issuing 'ipfw resetlog'. > >> > >> /usr/src/sys/conf/NOTES says IPFIREWALL_VERBOSE_LIMIT; limits the number of > >> times a matching entry can be logged. Says nothing about this limit being the > >> maximum number of log records allowed after which the log file is closed for > >> business. Are you saying the /usr/src/sys/conf/NOTES info is no longer true? > > > > You showed one (1) 'log' rule for each of the host's and jail's ruleset. > > Once that one rule has been logged 'logamount' times (default as per > > NOTES is 100, but in your case is 10) then logging for THAT rule stops, > > therefore with only one 'log' rule, ALL logging stops. Understand? > > > > If you take the time to properly study the correct reference, ipfw(8), > > all of this will become clear. See especially section SYSCTL VARIABLES, > > and read thoroughly 'log [logamount number]', at the very least. Ignore > > the Handbook section on ipfw, it's full of errors and misunderstandings. > > > >> Without IPFIREWALL_VERBOSE and IPFIREWALL_VERBOSE_LIMIT where does the logged > >> packets get written to? /var/log/security > > > > See above. Both of these options merely set defaults for the sysctls. > > > >> I have not used ipfw since it's ipfw2 rewrite so my knowledge is dated. > > > > Indeed it is; that's a very long time ago. > > > >>>> options IPFIREWALL_DEFAULT_TO_ACCEPT > >>>> options IPFIREWALL_IPDIVERT > >>> > >>> You'd likely do better using in-kernel NAT; natd doesn't get much love. > >>> > >> > >> I kept getting kernel compile errors using "options IPFIREWALL_NAT". I > >> thought the error was caused by vimage. Now I know "options LIBALIAS" is > >> required. Could not find info on internet search for IPFIREWALL_NAT with > >> vimage kernel. > > > > Apart from FIREWALL_FORWARD (not even that in 10.x), none of that needs > > to be in the kernel, it's all loadable as modules; see /etc/rc.d/ipfw. > > > > If you're doing NAT in the vimage jail, you must have at least two > > interfaces assigned to the jail. Care to show your config for that? > > > >> Do you have first hand experience getting "ipfw kernel nat" to work in a > >> vimage jail or having logging work on the host and within the vnet jail? > > > > No, but I have just on 15 years experience managing ipfw firewalls :) > > When you are new at things you do mistakes, remember. I still do mistakes. Trying to teach fishing rather than just tossing another fish is often one of mine :) I'm glad you had some to spare. > To try to answer Joes question: > > You don't need to compile anything into the kernel regarding ipfw. > > Just load the ipfw module in the host system with: > > kldload ipfw > > By default a deny all rule is added, so add a allow rule to the host system. > > ipfw add 10 allow ip from any to any > > To log things you change the sysctl value net.inet.ip.fw.verbose to 1 > > sysctl net.inet.ip.fw.verbose=1 > > If you keep net.inet.ip.fw.verbose_limit=0 you don't have a log limit, and for tests thats fine. Sure, though the default of 100 is plenty for such tests; it's surprisingly easy to DoS syslogd with e.g. a logged flood ping .. > log in to the jail system. Change the sysctl value net.inet.ip.fw.verbose to 1 > > sysctl net.inet.ip.fw.verbose=1 > > Add a logging firewall rule > > ipfw add 10 allow log ip from any to any > > Do a ping to an external system. > Look inside /var/log/security in the jail system and its empty. But it does exist, rw for root, with 0 or more bytes, right? And does the vimage jail's /etc/syslog.conf contain: security.* /var/log/security That is, I'm checking that the jail's syslogd should be handling these. What happens if you run in the jail, say: # logger -p security.info Syslog, wherefore art thou, Syslog? Does that go to the jail's /var/log/security? or the host's? > Go to the main host and look at the /var/log/security file and you will find log entries. Showing the host's hostname, or the jail's? Can you post some examples? > I can confirm Joes bug. I don't have a log rule in the main host but still get log messages. > All log messages are from the log rule in the jail system. > > System used: 9.1-RELEASE-p2 > > BR > /Anders Ok, before determining that this is an ipfw-only issue - in which case we need to move it over to freebsd-ipfw@ - can you confirm that normal syslogging in the jail to /var/log/messages and such is working? In particular I'm wondering what happens when you do set (say) net.inet.ip.fw.verbose_limit=10 and then ping from the jail until logging stops .. you should then see a message such as Apr 23 23:42:05 sola kernel: ipfw: limit 500 reached on entry 26400 both in /var/log/security and in /var/log/messages since it's logged as security.notice and default syslog.conf is for *.notice to log to /var/log/messages .. see the tail of /sys/netpfil/ipfw/ip_fw_log.c Yes sure, I'm flying blind, don't have a system with jails here yet, and am making assumptions about how syslogd(8) should work in jails that I really don't have time to properly research currently, nor am I properly across all the security implications of (particularly vimage) jails. cheers, Ian From owner-freebsd-jail@FreeBSD.ORG Thu May 2 20:06:19 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0CCDBA13 for ; Thu, 2 May 2013 20:06:19 +0000 (UTC) (envelope-from anders.hagman@netplex.se) Received: from smtp-out11.han.skanova.net (smtp-out11.han.skanova.net [195.67.226.200]) by mx1.freebsd.org (Postfix) with ESMTP id 681FD133E for ; Thu, 2 May 2013 20:06:18 +0000 (UTC) Received: from macen.halleforshunden.org (31.210.252.116) by smtp-out11.han.skanova.net (8.5.133) (authenticated as u48002568) id 516D088C006FF277; Thu, 2 May 2013 22:05:51 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: vnet jail with ipfw having logging problem From: Anders Hagman In-Reply-To: <20130503010007.C30818@sola.nimnet.asn.au> Date: Thu, 2 May 2013 22:05:49 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> <20130502142443.V30818@sola.nimnet.asn.au> <20130503010007.C30818@sola.nimnet.asn.au> To: Ian Smith X-Mailer: Apple Mail (2.1503) Cc: freebsd-jail X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 20:06:19 -0000 2 maj 2013 kl. 18:46 skrev Ian Smith : > On Thu, 2 May 2013 12:09:08 +0200, Anders Hagman wrote: >> Hi > Yo >> 2 maj 2013 kl. 07:42 skrev Ian Smith : >>=20 >>> On Wed, 1 May 2013 17:43:03 -0400, Joe wrote: >>>>>> I have ipfw running inside of a vnet jail on a 9.1-RELEASE host = using >>>>> the >>>>>> jail(8) definition statements for starting and stopping the vnet = jail. >>>>> As a >>>>>> side note non-vnet jails are working as expected. >>>>>>> The host is running a custom kernel with modules and with >>>>>> options VIMAGE >>>>>> nooptions SCTP >>>>>> options IPFIREWALL >>>>>> options IPFIREWALL_VERBOSE >>>>>> options IPFIREWALL_VERBOSE_LIMIT=3D10 >>>=20 >>> Please maintain attributions for the archives. I wrote: >>>=20 >>>>> What steps have you taken during testing to override this = ridiculously low >>>>> limit on logging? Otherwise, after e.g. just 5 pings and 5 ping = responses >>>>> are logged, all logging ceases until issuing 'ipfw resetlog'. >>>>=20 >>>> /usr/src/sys/conf/NOTES says IPFIREWALL_VERBOSE_LIMIT; limits the = number of >>>> times a matching entry can be logged. Says nothing about this limit = being the >>>> maximum number of log records allowed after which the log file is = closed for >>>> business. Are you saying the /usr/src/sys/conf/NOTES info is no = longer true? >>>=20 >>> You showed one (1) 'log' rule for each of the host's and jail's = ruleset.=20 >>> Once that one rule has been logged 'logamount' times (default as per=20= >>> NOTES is 100, but in your case is 10) then logging for THAT rule = stops,=20 >>> therefore with only one 'log' rule, ALL logging stops. Understand? >>>=20 >>> If you take the time to properly study the correct reference, = ipfw(8),=20 >>> all of this will become clear. See especially section SYSCTL = VARIABLES,=20 >>> and read thoroughly 'log [logamount number]', at the very least. = Ignore=20 >>> the Handbook section on ipfw, it's full of errors and = misunderstandings. >>>=20 >>>> Without IPFIREWALL_VERBOSE and IPFIREWALL_VERBOSE_LIMIT where does = the logged >>>> packets get written to? /var/log/security >>>=20 >>> See above. Both of these options merely set defaults for the = sysctls. >>>=20 >>>> I have not used ipfw since it's ipfw2 rewrite so my knowledge is = dated. >>>=20 >>> Indeed it is; that's a very long time ago. >>>=20 >>>>>> options IPFIREWALL_DEFAULT_TO_ACCEPT >>>>>> options IPFIREWALL_IPDIVERT >>>>>=20 >>>>> You'd likely do better using in-kernel NAT; natd doesn't get much = love. >>>>>=20 >>>>=20 >>>> I kept getting kernel compile errors using "options = IPFIREWALL_NAT". I >>>> thought the error was caused by vimage. Now I know "options = LIBALIAS" is >>>> required. Could not find info on internet search for IPFIREWALL_NAT = with >>>> vimage kernel. >>>=20 >>> Apart from FIREWALL_FORWARD (not even that in 10.x), none of that = needs=20 >>> to be in the kernel, it's all loadable as modules; see = /etc/rc.d/ipfw. >>>=20 >>> If you're doing NAT in the vimage jail, you must have at least two=20= >>> interfaces assigned to the jail. Care to show your config for that? >>>=20 >>>> Do you have first hand experience getting "ipfw kernel nat" to work = in a >>>> vimage jail or having logging work on the host and within the vnet = jail? >>>=20 >>> No, but I have just on 15 years experience managing ipfw firewalls = :) >>=20 >> When you are new at things you do mistakes, remember. >=20 > I still do mistakes. Trying to teach fishing rather than just tossing=20= > another fish is often one of mine :) I'm glad you had some to spare. I know the game. ;-> >=20 >> To try to answer Joes question: >>=20 >> You don't need to compile anything into the kernel regarding ipfw. >>=20 >> Just load the ipfw module in the host system with: >>=20 >> kldload ipfw >>=20 >> By default a deny all rule is added, so add a allow rule to the host = system. >>=20 >> ipfw add 10 allow ip from any to any >>=20 >> To log things you change the sysctl value net.inet.ip.fw.verbose to 1 >>=20 >> sysctl net.inet.ip.fw.verbose=3D1 >>=20 >> If you keep net.inet.ip.fw.verbose_limit=3D0 you don't have a log = limit, and for tests thats fine. >=20 > Sure, though the default of 100 is plenty for such tests; it's=20 > surprisingly easy to DoS syslogd with e.g. a logged flood ping .. >=20 >> log in to the jail system. Change the sysctl value = net.inet.ip.fw.verbose to 1 >>=20 >> sysctl net.inet.ip.fw.verbose=3D1 >>=20 >> Add a logging firewall rule >>=20 >> ipfw add 10 allow log ip from any to any >>=20 >> Do a ping to an external system. >> Look inside /var/log/security in the jail system and its empty. >=20 > But it does exist, rw for root, with 0 or more bytes, right? And does=20= > the vimage jail's /etc/syslog.conf contain: > security.* /var/log/security >=20 Yes > That is, I'm checking that the jail's syslogd should be handling = these. =20 > What happens if you run in the jail, say: > # logger -p security.info Syslog, wherefore art thou, Syslog? > Does that go to the jail's /var/log/security? or the host's? In jail system webben: logger -p security.info Syslog, wherefore art thou, Syslog? tail /var/log/security May 2 21:24:48 webben root: Syslog, wherefore art thou, Syslog? >=20 >> Go to the main host and look at the /var/log/security file and you = will find log entries. >=20 > Showing the host's hostname, or the jail's? Can you post some = examples? In host system dator5: tail /var/log/security May 2 21:29:15 dator5 kernel: ipfw: 10 Accept TCP 10.2.0.101:80 = 94.153.64.32:3085 out via vlan101 May 2 21:29:15 dator5 kernel: ipfw: 10 Accept TCP 94.153.64.32:3085 = 10.2.0.101:80 in via vlan101 >=20 >> I can confirm Joes bug. I don't have a log rule in the main host but = still get log messages. >> All log messages are from the log rule in the jail system. >>=20 >> System used: 9.1-RELEASE-p2 >>=20 >> BR >> /Anders >=20 > Ok, before determining that this is an ipfw-only issue - in which case=20= > we need to move it over to freebsd-ipfw@ - can you confirm that normal=20= > syslogging in the jail to /var/log/messages and such is working? >=20 In jail system login anders password ***** tail /var/log/messages May 2 21:41:57 webben login: login_getclass: unknown class 'svensk' May 2 21:42:00 webben last message repeated 3 times > In particular I'm wondering what happens when you do set (say)=20 > net.inet.ip.fw.verbose_limit=3D10 and then ping from the jail until=20 > logging stops .. you should then see a message such as >=20 > Apr 23 23:42:05 sola kernel: ipfw: limit 500 reached on entry 26400 >=20 > both in /var/log/security and in /var/log/messages since it's logged=20= > as security.notice and default syslog.conf is for *.notice to log to > /var/log/messages .. see the tail of /sys/netpfil/ipfw/ip_fw_log.c >=20 > Yes sure, I'm flying blind, don't have a system with jails here yet, = and=20 > am making assumptions about how syslogd(8) should work in jails that I=20= > really don't have time to properly research currently, nor am I = properly > across all the security implications of (particularly vimage) jails. >=20 On jail system: =20 sysctl net.inet.ip.fw.verbose_limit=3D10 Pinging repeatedly. Just continue to log to host system. Add new ipfw log role will use the new limit: ipfw add 5 allow log ip from any to any 00005 allow log logamount 10 ip from any to any New ping test. /var/log/security in host system : May 2 21:52:28 dator5 kernel: ipfw: 5 Accept ICMP:8.0 10.2.0.101 = 195.49.241.132 out via vlan101 May 2 21:52:28 dator5 kernel: ipfw: 5 Accept ICMP:0.0 195.49.241.132 = 10.2.0.101 in via vlan101 May 2 21:52:28 dator5 kernel: ipfw: limit 10 reached on entry 5 /var/log/messages in host system : May 2 21:52:28 dator5 kernel: ipfw: limit 10 reached on entry 5 Nothing at all is logged to the jail syslog. BR /Anders From owner-freebsd-jail@FreeBSD.ORG Fri May 3 00:54:36 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EB7B1C55 for ; Fri, 3 May 2013 00:54:36 +0000 (UTC) (envelope-from fbsd8@a1poweruser.com) Received: from mail-03.name-services.com (mail-03.name-services.com [69.64.155.195]) by mx1.freebsd.org (Postfix) with ESMTP id D528D1EE5 for ; Fri, 3 May 2013 00:54:36 +0000 (UTC) Received: from [10.0.10.1] ([173.88.202.176]) by mail-03.name-services.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 2 May 2013 17:54:37 -0700 Message-ID: <51830AC9.9080708@a1poweruser.com> Date: Thu, 02 May 2013 20:54:33 -0400 From: Joe User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Anders Hagman Subject: Re: vnet jail with ipfw having logging problem References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> <20130502142443.V30818@sola.nimnet.asn.au> <20130503010007.C30818@sola.nimnet.asn.au> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 03 May 2013 00:54:37.0370 (UTC) FILETIME=[C90879A0:01CE4798] X-Sender: fbsd8@a1poweruser.com X-Authenticated-Sender: fbsd8@a1poweruser.com X-EchoSenderHash: [fbsd8]-[a1poweruser*com] Cc: freebsd-jail , Ian Smith X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2013 00:54:37 -0000 I am posting 2 console logs created using the script command. The main differences between the 2 is, log 1 is a 9.1 kernel with modules and vimage compiled in. This shows the first problem being that dynamically loaded ipfw with a vimage kernel don't work. Log 2 is a 9.1 kernel with modules and vimage plus ipfw compiled in. This shows the second problem with vnet jails running ipfw log to host security file and don't log any ipfw log messages to the hosts message file. Secondly the vnet jails security and messages files never get populated with ipfw log messages. Console log 1. 9.1-RELEASE ipfw dynamically loaded by firewall statements in hosts rc.conf with modules and only vimage compiled into kernel. logger cmd on host did not work until after vnet jail was started and stopped. vnet jail pings passed through vnet jail but was not handed to host ipfw. vnet jail pings got logged to hosts security file but not messages. After vnet jail stopped, host logger cmd works and host pings work and logged correctly to security and messages. # /root >sysctl net.inet.ip.fw.verbose net.inet.ip.fw.verbose: 1 # /root >sysctl net.inet.ip.fw.verbose_limit net.inet.ip.fw.verbose_limit: 0 # /root >cat /etc/rc.comf # snip firewall_enable="YES" firewall_logging="YES" firewall_script="/etc/ipfw.rules" # /root >logger security.notice this msg is from logger cmd on host # /root >cat /var/log/security empty file # /root >cat /var/log/messages empty file # /root >ping -c 4 freebsd.org PING freebsd.org (8.8.178.135): 56 data bytes 64 bytes from 8.8.178.135: icmp_seq=0 ttl=51 time=102.814 ms 64 bytes from 8.8.178.135: icmp_seq=1 ttl=51 time=84.625 ms 64 bytes from 8.8.178.135: icmp_seq=2 ttl=51 time=101.332 ms 64 bytes from 8.8.178.135: icmp_seq=3 ttl=51 time=120.662 ms --- freebsd.org ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 84.625/102.358/120.662/12.755 ms # /root >cat /var/log/messages empty file # /root >cat /var/log/security May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept UDP 10.0.10.5:42524 209.18.47.61:53 out via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.0.10.5:42524 in via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:51 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:51 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:52 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:52 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:53 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:53 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 # /root >logger security.notice this msg is from logger cmd on host # /root >cat /var/log/security May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept UDP 10.0.10.5:42524 209.18.47.61:53 out via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.0.10.5:42524 in via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:51 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:51 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:52 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:52 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:53 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:53 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 vnet jail gets started # /root >jls JID IP Address Hostname Path 2 - vdir2 /usr/jails/vdir2 # /root >jexec vdir2 tcsh vdir2 / >logger -p security.notice logger cmd msg from within the host vdir2 / >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 0 0 allow log ip from any to any via epair2b 65535 5 368 deny ip from any to any vdir2 / >ping -c 4 freebsd.org ping: cannot resolve freebsd.org: Host name lookup failure vdir2 / >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 8 480 allow log ip from any to any via epair2b 65535 5 368 deny ip from any to any vdir2 / >exit exit # back on the host # /root >cat /var/log/security May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept UDP 10.0.10.5:42524 209.18.47.61:53 out via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.0.10.5:42524 in via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:50 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:51 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:51 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:52 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:52 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:05:53 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:05:53 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:10:50 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:32606 209.18.47.61:53 out via epair2b May 2 19:10:55 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:29810 209.18.47.62:53 out via epair2b May 2 19:10:57 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:32606 209.18.47.61:53 out via epair2b May 2 19:11:00 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:35933 209.18.47.61:53 out via epair2b May 2 19:11:05 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:56823 209.18.47.62:53 out via epair2b May 2 19:11:07 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:35933 209.18.47.61:53 out via epair2b May 2 19:11:07 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:29810 209.18.47.62:53 out via epair2b May 2 19:11:17 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:56823 209.18.47.62:53 out via epair2b May 2 19:11:22 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:37981 209.18.47.61:53 out via epair2b May 2 19:11:27 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:24567 209.18.47.62:53 out via epair2b May 2 19:11:29 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:37981 209.18.47.61:53 out via epair2b May 2 19:11:39 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:24567 209.18.47.62:53 out via epair2b May 2 19:11:44 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:54854 209.18.47.61:53 out via epair2b May 2 19:11:49 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:33964 209.18.47.62:53 out via epair2b May 2 19:11:51 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:54854 209.18.47.61:53 out via epair2b # /root >logger -p security.notice host logger msg # /root >cat /var/log/security May 2 19:11:39 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:24567 209.18.47.62:53 out via epair2b May 2 19:11:44 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:54854 209.18.47.61:53 out via epair2b May 2 19:11:49 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:33964 209.18.47.62:53 out via epair2b May 2 19:11:51 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:54854 209.18.47.61:53 out via epair2b May 2 19:12:01 fbsdjones kernel: ipfw: 11 Accept UDP 10.2.0.2:33964 209.18.47.62:53 out via epair2b May 2 19:12:50 fbsdjones root: host logger msg # /root >cat /var/log/messages May 2 19:08:10 fbsdjones kernel: bridge0: Ethernet address: 02:8f:94:84:0c:00 May 2 19:08:10 fbsdjones kernel: bridge0: link state changed to UP May 2 19:08:10 fbsdjones kernel: epair2a: Ethernet address: 02:c0:a4:00:0a:0a May 2 19:08:10 fbsdjones kernel: epair2b: Ethernet address: 02:c0:a4:00:0b:0b May 2 19:08:10 fbsdjones kernel: epair2a: link state changed to UP May 2 19:08:10 fbsdjones kernel: epair2b: link state changed to UP May 2 19:12:50 fbsdjones root: host logger msg Console log 2. This test run is using 9.1-RELEASE with modules plus vimage and ipfw compiled in. options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT logger command works. logged msg in both security and messages on host vnet jail can ping the public internet. Hosts security file has log messages from both jail and host. ipfw log messages are not being put into the hosts messages file. # ran on the host # /root >sysctl net.inet.ip.fw.verbose net.inet.ip.fw.verbose: 1 # /root >sysctl net.inet.ip.fw.verbose_limit net.inet.ip.fw.verbose_limit: 0 # /root >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 0 0 allow log ip from any to any via rl0 65535 1 328 allow ip from any to any # /root >/var/log/security empty file # /root >cat /var/log/messages empty file # /root >logger -p security.notice host logger cmd 1 # /root >cat /var/log/security May 2 19:45:51 fbsdjones root: host logger cmd 1 # /root >cat /var/log/messages May 2 19:45:51 fbsdjones root: host logger cmd 1 # /root >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 0 0 allow log ip from any to any via rl0 65535 1 328 allow ip from any to any # /root >ping -c 3 freebsd.org PING freebsd.org (8.8.178.135): 56 data bytes 64 bytes from 8.8.178.135: icmp_seq=0 ttl=51 time=85.032 ms 64 bytes from 8.8.178.135: icmp_seq=1 ttl=51 time=84.381 ms 64 bytes from 8.8.178.135: icmp_seq=2 ttl=51 time=84.647 ms --- freebsd.org ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 84.381/84.687/85.032/0.267 ms # /root >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 9 869 allow log ip from any to any via rl0 65535 1 328 allow ip from any to any vnet jail started # /root >jls JID IP Address Hostname Path 1 - vdir2 /usr/jails/vdir2 # /root >jexec vdir2 tcsh vdir2 / >cat /etc/ipfw.rules # Flush out the list before we begin. ipfw -q -f flush cmd="ipfw -q add" if [ -e /etc/epair ]; then pif=`cat "/etc/epair"` rm /etc/epair else pif="lo0" fi $cmd 010 allow all from any to any via lo0 $cmd 011 allow log all from any to any via $pif vdir2 / >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 0 0 allow log ip from any to any via epair1b 65535 8 624 allow ip from any to any vdir2 / >ping -c 3 freebsd.org PING freebsd.org (8.8.178.135): 56 data bytes 64 bytes from 8.8.178.135: icmp_seq=0 ttl=51 time=84.342 ms 64 bytes from 8.8.178.135: icmp_seq=1 ttl=51 time=84.195 ms 64 bytes from 8.8.178.135: icmp_seq=2 ttl=51 time=84.015 ms --- freebsd.org ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 84.015/84.184/84.342/0.134 ms vdir2 / >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 8 634 allow log ip from any to any via epair1b 65535 8 624 allow ip from any to any vdir2 / >cat /var/log/security May 1 21:56:27 vdir2 newsyslog[5202]: logfile first created vdir2 / >cat /var/log/messages May 1 21:56:27 vdir2 newsyslog[5202]: logfile first created vdir2 / >exit exit Back on the host # /root >cat /var/log/security May 2 19:45:51 fbsdjones root: host logger cmd 1 May 2 19:46:53 fbsdjones kernel: ipfw: 11 Accept UDP 10.0.10.1:138 10.0.10.7:138 in via rl0 May 2 19:46:58 fbsdjones kernel: ipfw: 11 Accept UDP 10.0.10.5:64721 209.18.47.61:53 out via rl0 May 2 19:46:58 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.0.10.5:64721 in via rl0 May 2 19:46:58 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:46:58 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:46:59 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:46:59 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:47:00 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:47:00 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:47:38 fbsdjones kernel: ipfw: 11 Accept ICMPv6:143.0 [::] [ff02::16] out via rl0 May 2 19:47:38 fbsdjones kernel: ipfw: 11 Accept ICMPv6:143.0 [::] [ff02::16] out via rl0 May 2 19:47:39 fbsdjones kernel: ipfw: 11 Accept ICMPv6:135.0 [::] [ff02::1:ff00:b0b] out via rl0 May 2 19:47:39 fbsdjones kernel: ipfw: 11 Accept ICMPv6:143.0 [::] [ff02::16] out via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 10.1.0.2:13101 209.18.47.61:53 out via epair1b May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 10.1.0.2:13101 209.18.47.61:53 out via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.1.0.2:13101 in via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.1.0.2:13101 in via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.1.0.2:13101 in via epair1b May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via epair1b May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via epair1b May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via epair1b May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via rl0 May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via epair1b May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via epair1b May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via rl0 May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via epair1b # /root >cat /var/log/messages May 2 19:45:51 fbsdjones root: host logger cmd 1 May 2 19:47:38 fbsdjones kernel: bridge0: Ethernet address: 02:8f:94:84:0c:00 May 2 19:47:38 fbsdjones kernel: bridge0: link state changed to UP May 2 19:47:38 fbsdjones kernel: epair1a: Ethernet address: 02:c0:24:00:0a:0a May 2 19:47:38 fbsdjones kernel: epair1b: Ethernet address: 02:c0:24:00:0b:0b May 2 19:47:38 fbsdjones kernel: epair1a: link state changed to UP May 2 19:47:38 fbsdjones kernel: epair1b: link state changed to UP May 2 19:50:59 fbsdjones kernel: epair1a: link state changed to DOWN May 2 19:50:59 fbsdjones kernel: epair1b: link state changed to DOWN May 2 19:50:59 fbsdjones kernel: bridge0: link state changed to DOWN May 2 19:51:02 fbsdjones kernel: Freed UMA keg was not empty (30 items). Lost 2 pages of memory. May 2 19:51:02 fbsdjones kernel: Freed UMA keg was not empty (203 items). Lost 1 pages of memory. May 2 19:51:02 fbsdjones kernel: Freed UMA keg was not empty (30 items). Lost 2 pages of memory. May 2 19:51:02 fbsdjones kernel: hhook_vnet_uninit: hhook_head type=1, id=1 cleanup required May 2 19:51:02 fbsdjones kernel: hhook_vnet_uninit: hhook_head type=1, id=0 cleanup required # /root >exit exit From owner-freebsd-jail@FreeBSD.ORG Sat May 4 21:27:18 2013 Return-Path: Delivered-To: freebsd-jail@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3FF37464; Sat, 4 May 2013 21:27:18 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 18E4EA2; Sat, 4 May 2013 21:27:18 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r44LRHpl078364; Sat, 4 May 2013 21:27:17 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r44LRHSL078363; Sat, 4 May 2013 21:27:17 GMT (envelope-from linimon) Date: Sat, 4 May 2013 21:27:17 GMT Message-Id: <201305042127.r44LRHSL078363@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-jail@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: bin/178302: jail(8): unknown parameter: ip6.addr when kernel compiled without ipv6 support X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2013 21:27:18 -0000 Old Synopsis: jail: unknown parameter: ip6.addr when kernel compiled without ipv6 support New Synopsis: jail(8): unknown parameter: ip6.addr when kernel compiled without ipv6 support Responsible-Changed-From-To: freebsd-bugs->freebsd-jail Responsible-Changed-By: linimon Responsible-Changed-When: Sat May 4 21:25:51 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=178302