Date: Sun, 11 Nov 2018 11:33:45 +0100 From: "Kristof Provost" <kristof@sigsegv.be> To: "Ernie Luzar" <luzar722@gmail.com> Cc: freebsd-questions@freebsd.org, freebsd-jail@freebsd.org Subject: Re: 12.0-beta3 pf firewall NAT rule syntax for vnet jail using pf Message-ID: <CE5DE9B5-C24A-435A-83FE-080F9418EFFD@sigsegv.be> In-Reply-To: <5BE5CE9D.9030503@gmail.com> References: <5BE5CE9D.9030503@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 9 Nov 2018, at 19:14, Ernie Luzar wrote: > Hello lists; > > testing 12.0-beta3 vnet jail that is using pf firewall. > net.inet.ip.forwarding =1 for the vnet jail. > Host is running ipfilter firewall. > The kldload pf.ko pflog.ko command has been issued. > 10.0.10.30 is the ip address assigned to the vnet jail in the > jail.conf. > Using this nat rule > > nat on epair2b from 10.0.0.30/24 to any -> (vge0) > Is this rule set on the pf inside the jail? > vge0 is the hosts interface facing the public internet and a member of > bridge2 along with member epair2a. > Is this bridge on the host, so outside the jail? If so, how can the jail see the vge0 interface? Best regards, Kristof From owner-freebsd-jail@freebsd.org Sun Nov 11 17:00:51 2018 Return-Path: <owner-freebsd-jail@freebsd.org> Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E2F05110AE60; Sun, 11 Nov 2018 17:00:50 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mail-it1-x133.google.com (mail-it1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3FDD888B36; Sun, 11 Nov 2018 17:00:50 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mail-it1-x133.google.com with SMTP id v11so10093014itj.0; Sun, 11 Nov 2018 09:00:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-transfer-encoding; bh=gp1BV0/iJqTtEgTLOlg5jd6IxxB63WqTANeALc95YlI=; b=IBk1SC823nsN9SBxdnWIe6GCopuAjd3Kw2Y9Ot/R4Ww1Mo6V+GfFjyoxbyN5HUQNK2 ER79yFuvctIHHMrB5vP7jHgoguqJOV3dRjOcTkhAWZyoxlLQ0zEU0V7/1VAesMSlGqqk 3K0HerPHjQ5wwVp88EUwLqBMTcniaM2VylYpqWCOlCR5wvS9rx3YMgxD4q+eS3VKJePi XitwpkDA6+1QZU4HbMciYIMp0gx6CMcQqulyHF5rPN9x18t4/m8iCQ3Sso3zZWwiyO6o ZeTAYMeVAuN8iJljjrKX5A/E39JZ8RVv0fOFL1tvIv7VNEM+DEL3G5vZlXaaMGYKS0Sc 0CQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-transfer-encoding; bh=gp1BV0/iJqTtEgTLOlg5jd6IxxB63WqTANeALc95YlI=; b=ScWvEsRpCYQMDW01KbWS9YRcL+pJwaddDdIK31m60qLNe0k4wKmVNRMsGF5CaZtA4L YAIezdW6ALeIkH3EKFOaFnYBjBjc7so5rSqrF0YHgdGVcb+rhwtQHvoAFfElY4AXVQRO u65gIUk0sDvMDN8H/MZmcrY9bl5skMlg8yyM1ix8KIqjnoaugDJCPiLQFeI8CA4gci85 qy7NS6XinJEjgWJV2du6CSfj/ErsTk3VVBbAGt4IJo47zCFNBdGB7Ut7ClYElMuFslya UIdQa5pG9G2JHK4FKDkvFtAPPKVw8AMDEnD3Vwc8X25KEov8uho98agRUHnqapIOzGDJ ut+A== X-Gm-Message-State: AGRZ1gLCo8AjHF1BynVaNEyOUGDsauCmbvpjEavY21W14PH0ius3ULpj opSk79yX6EpZ4K7J+GrQK+twSpH9 X-Google-Smtp-Source: AJdET5e4lliMJuO+hiDmShACnbt+KaucQLmO8+R2S5bYn5SnPHSxGHYTj+Cr9X7JdYyd6KEv/9pKIA== X-Received: by 2002:a02:4ac1:: with SMTP id s62-v6mr14936595jad.44.1541955649665; Sun, 11 Nov 2018 09:00:49 -0800 (PST) Received: from [10.0.10.7] (cpe-65-25-48-31.neo.res.rr.com. [65.25.48.31]) by smtp.googlemail.com with ESMTPSA id y190-v6sm2504782itg.3.2018.11.11.09.00.48 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 11 Nov 2018 09:00:49 -0800 (PST) Message-ID: <5BE86041.9070900@gmail.com> Date: Sun, 11 Nov 2018 12:00:49 -0500 From: Ernie Luzar <luzar722@gmail.com> User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: Kristof Provost <kristof@sigsegv.be> CC: freebsd-questions@freebsd.org, freebsd-jail@freebsd.org Subject: Re: 12.0-beta3 pf firewall NAT rule syntax for vnet jail using pf References: <5BE5CE9D.9030503@gmail.com> <CE5DE9B5-C24A-435A-83FE-080F9418EFFD@sigsegv.be> In-Reply-To: <CE5DE9B5-C24A-435A-83FE-080F9418EFFD@sigsegv.be> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 3FDD888B36 X-Spamd-Result: default: False [-6.73 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-2.73)[ip: (-8.79), ipnet: 2607:f8b0::/32(-2.84), asn: 15169(-1.94), country: US(-0.09)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" <freebsd-jail.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-jail>, <mailto:freebsd-jail-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-jail/>; List-Post: <mailto:freebsd-jail@freebsd.org> List-Help: <mailto:freebsd-jail-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-jail>, <mailto:freebsd-jail-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sun, 11 Nov 2018 17:00:51 -0000 Kristof Provost wrote: > On 9 Nov 2018, at 19:14, Ernie Luzar wrote: > > Hello lists; > > testing 12.0-beta3 vnet jail that is using pf firewall. > net.inet.ip.forwarding =1 for the vnet jail. > Host is running ipfilter firewall. > The kldload pf.ko pflog.ko command has been issued. > 10.0.10.30 is the ip address assigned to the vnet jail in the jail.conf. > Using this nat rule > > nat on epair2b from 10.0.0.30/24 to any -> (vge0) > > Is this rule set on the pf inside the jail? YES > > vge0 is the hosts interface facing the public internet and a member > of bridge2 along with member epair2a. > > Is this bridge on the host, so outside the jail? YES > > If so, how can the jail see the vge0 interface? Through the bridge? I don't really know. Just guessing. > > Best regards, > Kristof > I added pass to the pf nat rule so inbound packets that match entry in state table get passed automatically. Now using this pf nat rule nat pass on epair2b from 10.0.0.30/24 to any -> (epair2b) This is the ifconfig -a on the host after the vnet jail is started. em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING, VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER> ether d0:50:99:93:75:98 inet 10.0.10.2 netmask 0xff000000 broadcast 10.255.255.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> vge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=3899<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM, WOL_UCAST,WOL_MCAST,WOL_MAGIC> ether 00:16:36:4e:35:86 hwaddr 10:00:60:21:00:93 inet xx.xx.xx.xx netmask 0xfffff000 broadcast 255.255.255.255 media: Ethernet autoselect (1000baseT <full-duplex,master>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> pflog0: flags=0<> metric 0 mtu 33160 groups: pflog bridge2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 02:5c:98:6f:9d:0a id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 6 priority 128 path cost 2000 member: vge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 2 priority 128 path cost 20000 groups: bridge nd6 options=1<PERFORMNUD> epair2a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 02:d9:a3:a8:e7:0a inet6 fe80::d9:a3ff:fea8:e70a%epair2a prefixlen 64 scopeid 0x6 groups: epair media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> Here are the pf rules in the vnet jail oif=epair2b set block-policy drop set fail-policy drop set state-policy if-bound scrub in on $oif all set skip on lo0 nat pass on $oif inet from 10.0.0.30/24 to any -> ($oif) block out log quick on $oif inet proto tcp from any to any port 43 pass log (all) on $oif pass out quick on $oif all I test vnet jail by issuing ping 8.8.8.8 and get "time to live exceeded" message. ping 10.0.10.2 get all lost packets normal message. Is there some other way to test vnet jails from the host to verify they are working? There will come a time when I will need to test vnet jails from the public internet. Its easy to enable ssh on the vnet jail and then use some other isp to ssh into the vnet jail. What would be the syntax of the remote ssh command to do this? It's my understanding that vnet jails have their own network stack which means there is no interaction with the hosts network stack. Which also means there is no vnet firewall interaction with the hosts firewall. Is this correct? Since I want all my vnet jails to access the public internet, can their epair just be added to a single bridge as another member or does each one need it's own bridge? How is public internet traffic targeted to an individual vnet jail running on the host? Thanks for your help on this. Ernie Luzar
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CE5DE9B5-C24A-435A-83FE-080F9418EFFD>