From owner-freebsd-jail@FreeBSD.ORG Fri Jul 11 07:11:45 2014 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B825C08 for ; Fri, 11 Jul 2014 07:11:45 +0000 (UTC) Received: from mail1.chu658.uecomm.net.au (mail1.chu658.uecomm.net.au [218.185.10.246]) by mx1.freebsd.org (Postfix) with ESMTP id 036E7276C for ; Fri, 11 Jul 2014 07:11:44 +0000 (UTC) Received: from mail.fdrive.com.au (unknown [115.186.196.106]) by mail1.chu658.uecomm.net.au (Postfix) with ESMTP id 988571E92; Fri, 11 Jul 2014 17:11:33 +1000 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.fdrive.com.au (Postfix) with ESMTP id 2F4BAE5E38; Fri, 11 Jul 2014 17:11:24 +1000 (EST) X-Virus-Scanned: amavisd-new at fdrive.com.au Received: from mail.fdrive.com.au ([127.0.0.1]) by localhost (mail.fdrive.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JlkpcZVfzYqI; Fri, 11 Jul 2014 17:11:19 +1000 (EST) Received: from PetersBigBox (ws-pross.vv.fda [192.168.50.199]) by mail.fdrive.com.au (Postfix) with ESMTPS id 36BBDE5D82; Fri, 11 Jul 2014 17:11:19 +1000 (EST) Received: from localhost (localhost [127.0.0.1]) by PetersBigBox (Postfix) with ESMTP id DA1B819201EE; Fri, 11 Jul 2014 17:11:27 +1000 (EST) Date: Fri, 11 Jul 2014 17:11:27 +1000 (EST) From: Peter Ross X-X-Sender: petros@PetersBigBox To: Peter Toth Subject: RE: vnet jail and ipfw/nat on host - keep-state problem? In-Reply-To: Message-ID: References: User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-ID: Content-Type: TEXT/PLAIN; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-jail@freebsd.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 07:11:45 -0000 On Thu, 10 Jul 2014, Peter Toth wrote: > Hi Peter, > Try to make these changes: > > net.inet.ip.forwarding=1       # Enable IP forwarding between interfaces > net.link.bridge.pfil_onlyip=0  # Only pass IP packets when pfil is enabled > net.link.bridge.pfil_bridge=0  # Packet filter on the bridge interface > net.link.bridge.pfil_member=0  # Packet filter on the member interface > > You can find some info > here http://iocage.readthedocs.org/en/latest/help-no-internet.html > > I've had these issues before with PF and IPFW, by default these will be > filtering on your bridge and member interfaces. Thanks. It did not change anything. Now, inside_ the jail I run "ipfw allow ip from any to any". This on the host system: 01000 check-state 01100 allow tcp from any to any established 01200 allow ip from any to any frag 00100 divert 8668 ip4 from any to any via age0 03100 allow udp from any to 10.0.10.1 dst-port 53 keep-state 03200 allow udp from any to me dst-port 53 keep-state (with natd redirecting "redirect_port udp 10.0.10.1:53 external.ip:53") If I add 03300 allow udp from me 53 to any it works.. So it makes me think check-state isn't usable - because 03200 allow udp from any to me dst-port 53 keep-state should cover the returning packets. I played with your parameters but it did not help. But thanks for the idea. Here again the setup: Internet->age0(host interface with natd and external IP) ->bridge10(10.0.10.254)->epair1a ->epair1b(10.0.10.1 in bind vnet jail) I wonder what kind of restrictions exist with vnet.. it does not seem to work _exactly_ as a "real" network stack (the issues with pf inside the jail let me think of it too) Did I find a restriction, a bug - or just that I've got it wrong? Regards Peter From owner-freebsd-jail@FreeBSD.ORG Fri Jul 11 08:50:51 2014 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 587CD42A for ; Fri, 11 Jul 2014 08:50:51 +0000 (UTC) Received: from mail-ig0-x232.google.com (mail-ig0-x232.google.com [IPv6:2607:f8b0:4001:c05::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 267492078 for ; Fri, 11 Jul 2014 08:50:51 +0000 (UTC) Received: by mail-ig0-f178.google.com with SMTP id hn18so747305igb.17 for ; Fri, 11 Jul 2014 01:50:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=w2ble7LUrJWVxK+7CzL1ZllkeFKQBFGrucVSW7Nn4Jc=; b=NJPs4fcA5XcDTZnSsoWMyVHFw1BQN//nYQz5bMsBfgsaa3LTB2T9W6KNSUZgColvp4 KjmWaUmraGwwo3Ip512VGCMqIG/dEtjO4N9k+ggVfZjIjebbcovAvt5Zk8tIgPpMw1Tc i9ztfbhnGsEOjWdZpTjDh8hVFR0xe1EBbc1Ojr5va0jtOlGaW5SVuvvkutQj0INe7O12 12bOaxknfj5nYvBS39j+R89zDRy4MAeurEAzMZegNf8C9zqAi6T1PA0VvnY+9bOehMQX 43uhvRaIWu/RZUOeq0NujSGjrmpFgsganS521bA3IyAnRiOFBbh3hxDzP68iXF08cD5N 5oeA== MIME-Version: 1.0 X-Received: by 10.43.13.132 with SMTP id pm4mr3125916icb.6.1405068650537; Fri, 11 Jul 2014 01:50:50 -0700 (PDT) Received: by 10.42.168.194 with HTTP; Fri, 11 Jul 2014 01:50:50 -0700 (PDT) In-Reply-To: References: Date: Fri, 11 Jul 2014 20:50:50 +1200 Message-ID: Subject: Re: vnet jail and ipfw/nat on host - keep-state problem? From: Peter Toth To: Peter Ross Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-jail@freebsd.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 08:50:51 -0000 Have not used natd with IPFW much as always preferred PF to do everything on the host. I have only a wild guess - the "me" keyword in IPFW is substituted only to the host's IPs known to itself. The host's IPFW firewall most likely doesn't know anything about IPs assigned to vnet interfaces inside the jail. Vnet jails behave more like separate physical hosts. Internet ---> [host] ------- (10.0.10.0 LAN) ------> [vnet jail] The PF issue inside a jail is a separate problem, PF is not fully VIMAGE/VNET aware as far as I know. Can someone comment on these or correct me? P On Fri, Jul 11, 2014 at 7:11 PM, Peter Ross wrote: > On Thu, 10 Jul 2014, Peter Toth wrote: > > Hi Peter, >> Try to make these changes: >> >> net.inet.ip.forwarding=1 # Enable IP forwarding between interfaces >> net.link.bridge.pfil_onlyip=0 # Only pass IP packets when pfil is enabled >> net.link.bridge.pfil_bridge=0 # Packet filter on the bridge interface >> net.link.bridge.pfil_member=0 # Packet filter on the member interface >> >> You can find some info >> here http://iocage.readthedocs.org/en/latest/help-no-internet.html >> >> I've had these issues before with PF and IPFW, by default these will be >> filtering on your bridge and member interfaces. >> > > Thanks. It did not change anything. > > Now, inside_ the jail I run "ipfw allow ip from any to any". > > This on the host system: > > 01000 check-state > 01100 allow tcp from any to any established > 01200 allow ip from any to any frag > 00100 divert 8668 ip4 from any to any via age0 > 03100 allow udp from any to 10.0.10.1 dst-port 53 keep-state > 03200 allow udp from any to me dst-port 53 keep-state > > (with natd redirecting "redirect_port udp 10.0.10.1:53 external.ip:53") > > If I add > > 03300 allow udp from me 53 to any > > it works.. > > So it makes me think check-state isn't usable - because > > 03200 allow udp from any to me dst-port 53 keep-state > > should cover the returning packets. > > I played with your parameters but it did not help. But thanks for the idea. > > Here again the setup: > > Internet->age0(host interface with natd and external IP) > ->bridge10(10.0.10.254)->epair1a > ->epair1b(10.0.10.1 in bind vnet jail) > > I wonder what kind of restrictions exist with vnet.. it does not seem to > work _exactly_ as a "real" network stack (the issues with pf inside the > jail let me think of it too) > > Did I find a restriction, a bug - or just that I've got it wrong? > > Regards > Peter