From owner-freebsd-jail@FreeBSD.ORG Thu May 2 13:49:59 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 158DDEBA for ; Thu, 2 May 2013 13:49:59 +0000 (UTC) (envelope-from fbsd8@a1poweruser.com) Received: from mail-03.name-services.com (mail-03.name-services.com [69.64.155.195]) by mx1.freebsd.org (Postfix) with ESMTP id E4A941DCD for ; Thu, 2 May 2013 13:49:58 +0000 (UTC) Received: from [10.0.10.1] ([173.88.202.176]) by mail-03.name-services.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 2 May 2013 06:49:51 -0700 Message-ID: <51826EF7.30302@a1poweruser.com> Date: Thu, 02 May 2013 09:49:43 -0400 From: Joe User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Anders Hagman Subject: Re: vnet jail with ipfw having logging problem References: <44AC45947DA14449AEDFB13B9F6C5F7DAF3E1FA5@ltcfiswmsgmb25> <517A7BCB.8060604@a1poweruser.com> <13CA24D6AB415D428143D44749F57D7201F22068@ltcfiswmsgmb21> <517D3426.1090703@a1poweruser.com> <51805EFB.6050806@a1poweruser.com> <20130502021830.O30818@sola.nimnet.asn.au> <51818C67.7070708@a1poweruser.com> <20130502142443.V30818@sola.nimnet.asn.au> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 May 2013 13:49:51.0366 (UTC) FILETIME=[EB1F4260:01CE473B] X-Sender: fbsd8@a1poweruser.com X-Authenticated-Sender: fbsd8@a1poweruser.com X-EchoSenderHash: [fbsd8]-[a1poweruser*com] Cc: freebsd-jail X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 13:49:59 -0000 Anders Hagman wrote: > Hi > > 2 maj 2013 kl. 07:42 skrev Ian Smith : > >> On Wed, 1 May 2013 17:43:03 -0400, Joe wrote: >>>>> I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using >>>> the >>>>> jail(8) definition statements for starting and stopping the vnet jail. >>>> As a >>>>> side note non-vnet jails are working as expected. >>>>>> The host is running a custom kernel with modules and with >>>>> options VIMAGE >>>>> nooptions SCTP >>>>> options IPFIREWALL >>>>> options IPFIREWALL_VERBOSE >>>>> options IPFIREWALL_VERBOSE_LIMIT=10 >> Please maintain attributions for the archives. I wrote: >> >>>> What steps have you taken during testing to override this ridiculously low >>>> limit on logging? Otherwise, after e.g. just 5 pings and 5 ping responses >>>> are logged, all logging ceases until issuing 'ipfw resetlog'. >>> /usr/src/sys/conf/NOTES says IPFIREWALL_VERBOSE_LIMIT; limits the number of >>> times a matching entry can be logged. Says nothing about this limit being the >>> maximum number of log records allowed after which the log file is closed for >>> business. Are you saying the /usr/src/sys/conf/NOTES info is no longer true? >> You showed one (1) 'log' rule for each of the host's and jail's ruleset. >> Once that one rule has been logged 'logamount' times (default as per >> NOTES is 100, but in your case is 10) then logging for THAT rule stops, >> therefore with only one 'log' rule, ALL logging stops. Understand? >> >> If you take the time to properly study the correct reference, ipfw(8), >> all of this will become clear. See especially section SYSCTL VARIABLES, >> and read thoroughly 'log [logamount number]', at the very least. Ignore >> the Handbook section on ipfw, it's full of errors and misunderstandings. >> >>> Without IPFIREWALL_VERBOSE and IPFIREWALL_VERBOSE_LIMIT where does the logged >>> packets get written to? /var/log/security >> See above. Both of these options merely set defaults for the sysctls. >> >>> I have not used ipfw since it's ipfw2 rewrite so my knowledge is dated. >> Indeed it is; that's a very long time ago. >> >>>>> options IPFIREWALL_DEFAULT_TO_ACCEPT >>>>> options IPFIREWALL_IPDIVERT >>>> You'd likely do better using in-kernel NAT; natd doesn't get much love. >>>> >>> I kept getting kernel compile errors using "options IPFIREWALL_NAT". I >>> thought the error was caused by vimage. Now I know "options LIBALIAS" is >>> required. Could not find info on internet search for IPFIREWALL_NAT with >>> vimage kernel. >> Apart from FIREWALL_FORWARD (not even that in 10.x), none of that needs >> to be in the kernel, it's all loadable as modules; see /etc/rc.d/ipfw. >> >> If you're doing NAT in the vimage jail, you must have at least two >> interfaces assigned to the jail. Care to show your config for that? >> >>> Do you have first hand experience getting "ipfw kernel nat" to work in a >>> vimage jail or having logging work on the host and within the vnet jail? >> No, but I have just on 15 years experience managing ipfw firewalls :) > > When you are new at things you do mistakes, remember. > > To try to answer Joes question: > > You don't need to compile anything into the kernel regarding ipfw. > > Just load the ipfw module in the host system with: > > kldload ipfw > > By default a deny all rule is added, so add a allow rule to the host system. > > ipfw add 10 allow ip from any to any > > To log things you change the sysctl value net.inet.ip.fw.verbose to 1 > > sysctl net.inet.ip.fw.verbose=1 > > If you keep net.inet.ip.fw.verbose_limit=0 you don't have a log limit, and for tests thats fine. > > log in to the jail system. Change the sysctl value net.inet.ip.fw.verbose to 1 > > sysctl net.inet.ip.fw.verbose=1 > > Add a logging firewall rule > > ipfw add 10 allow log ip from any to any > > Do a ping to an external system. > Look inside /var/log/security in the jail system and its empty. > Go to the main host and look at the /var/log/security file and you will find log entries. > > I can confirm Joes bug. I don't have a log rule in the main host but still get log messages. > All log messages are from the log rule in the jail system. > > System used: 9.1-RELEASE-p2 > > BR > /Anders Thank you Anders, your reply was direct and to the point. Lets talk about this bug. The console.log parameter creates a log file in the hosts /var/log directory for each jail. I would think the ipfw log file should behave the same way. IE: the host ipfw log should be going to the hosts /var/log/security file with each ipfw jail creating it's own /var/log/jailname.security file on the host and not create a /var/log/security file in the jails filesystem. I searched the PR database for any PR's with vnet or vimage and ipfw logging and came up with no hits. Should I submit a PR about this problem? I tested doing a kldload ipfw and fall into the default deny problem. Is there a sysctl to flip the default deny to default accept? Thanks Joe