From owner-freebsd-net@freebsd.org Sun Jun 26 02:32:14 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65152B7321D for ; Sun, 26 Jun 2016 02:32:14 +0000 (UTC) (envelope-from James@Lodge.me.uk) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1on0137.outbound.protection.outlook.com [157.56.112.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BABD71BA2 for ; Sun, 26 Jun 2016 02:32:12 +0000 (UTC) (envelope-from James@Lodge.me.uk) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gavinlodge.onmicrosoft.com; s=selector1-Lodge-me-uk; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=S9eVB76aGmFyajcFYXv2e7k6SkIkRtFbohGrvZV29D4=; b=WAYSM3ZdldMyBUGZZ7q4E+g7Abprk9xavHmuvXFSzwxYNidVixZZE0kxt7vkt2fnphMPwujYwTXug3fGII/ACpGJSTDVSPyyz395F7wXujoMU58mcnZ5iQZdS6YOAF/PsqIFrsFzS8urbOo4msSiLiMFNA6Rm1Po/W2TMM/okN0= Received: from DB5PR06MB1718.eurprd06.prod.outlook.com (10.165.213.16) by DB5PR06MB1718.eurprd06.prod.outlook.com (10.165.213.16) with Microsoft SMTP Server (TLS) id 15.1.528.8; Sun, 26 Jun 2016 02:32:04 +0000 Received: from DB5PR06MB1718.eurprd06.prod.outlook.com ([10.165.213.16]) by DB5PR06MB1718.eurprd06.prod.outlook.com ([10.165.213.16]) with mapi id 15.01.0528.014; Sun, 26 Jun 2016 02:32:04 +0000 From: James Lodge To: "org.freebsd.security@io7m.com" CC: "freebsd-net@freebsd.org" Subject: Re: Filtering outbound traffic for private address jails? Thread-Topic: Filtering outbound traffic for private address jails? Thread-Index: AQHRzy5kidDlXJVON0eMaGq8Ct2cf5/7B32N Date: Sun, 26 Jun 2016 02:32:04 +0000 Message-ID: References: <20160625220137.1ed8de16@copperhead.int.arc7.info> In-Reply-To: <20160625220137.1ed8de16@copperhead.int.arc7.info> Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=James@Lodge.me.uk; x-originating-ip: [213.205.198.133] x-ms-office365-filtering-correlation-id: d89105a3-39e2-4170-e63e-08d39d6a0fc3 x-microsoft-exchange-diagnostics: 1; DB5PR06MB1718; 6:cBiE9xTBiYhW70BDU5pACFpkwzd/15XiC86EEX5LoVw0zYAkNQqfB1VmDqUd5+QacLGnj7iSrhI47bgabt4BUH6H6ykT2/+dzHFHPLwAvHHjCVojRIPW2y6i4aS7uJmAUZnhUSmion0h7LOhLfsNWxSxNjPd8eLHZcimBctC77in0yF/3sQEsWGFwx1ymBdobdCHUy0MXtSk/TvKKxteO+sdgN+6D2qInKCddQWo+wdEMfwaYe0/RSditX9opOFduy181rmaPrF6z9wt7ruCLsi9ihcu4+23Dk6h/YK4JhO5hegG/llupF94Q05IMTob; 5:yK60g0R3SWbxNhujilAHpyUnVswQWft0nnSTFOF+zRcwcQrgWA4x1qelGF4aQLTmDpPNJLblyOJ8KD2BdqmEqrV1MrmQcAjTnmtLl/0EEIufH2drN6VOzRw/38lciX1uvLR1gi5hs+Er5H/24N9N2A==; 24:xPc35VIQ4d6QE577E0zY2XgRytgT8WmHRQ8G+rStWSjZ0xk6RSE3NO0aAiOKGfMGDhc1pGOXyF0mlQbh21L8WormWb1R7/jBwkPZUY9UT8A=; 7:KTWj1bQRO+UZN0s9bgTUoCDSwb5Mspb/34G9uiVMIJ28XV+CcEOtw8Ihqp3JYbx81QkxMyDmAYC17e6eu8nt9H7HXOrMq1PwUmtp5EGWwmhFkl0bX/lsmSu64SjgyfvnxBPn6ti1roBTwUmIa7PQGWZ5U+357AVV7FerN++SbOq9IvRmWyhDxfv/SUUJK5bVfESDcvYylNZ9H5Ayw9e3cpm2g9Ny4yjfsuOLNzRX5jxpo26U/FkpOl7jNkBzzvN9 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB5PR06MB1718; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(209352067349851)(75325880899374); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040130)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6041072)(6043046); SRVR:DB5PR06MB1718; BCL:0; PCL:0; RULEID:; SRVR:DB5PR06MB1718; x-forefront-prvs: 0985DA2459 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(7916002)(24454002)(189002)(199003)(105586002)(82746002)(2501003)(77096005)(19580405001)(50986999)(19580395003)(54356999)(76176999)(15975445007)(106356001)(33656002)(3660700001)(83716003)(16236675004)(5003630100001)(3280700002)(2906002)(2900100001)(2950100001)(189998001)(5002640100001)(19617315012)(66066001)(92566002)(68736007)(5640700001)(81156014)(8676002)(86362001)(4326007)(586003)(3846002)(11100500001)(102836003)(6116002)(81166006)(8936002)(80792005)(2351001)(106116001)(36756003)(10400500002)(7736002)(122556002)(74482002)(110136002)(7906003)(87936001)(101416001)(97736004)(7846002)(104396002); DIR:OUT; SFP:1102; SCL:1; SRVR:DB5PR06MB1718; H:DB5PR06MB1718.eurprd06.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: Lodge.me.uk does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: Lodge.me.uk X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Jun 2016 02:32:04.3480 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: ded56ae9-7c77-4cf6-bbfd-39e6a505742d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR06MB1718 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Jun 2016 02:32:14 -0000 Sent from my iPhone On 25 Jun 2016, at 23:10, "org.freebsd.security@io7m.com" > wrote: Hello. I have been searching for the best part of a day for a solution to this problem and quite frankly cannot believe that I've spent this long on something that appears to be so simple and that used to be fairly easy to achieve. Many years ago, I solved this problem on FreeBSD 6, but the way I did it there seems to no longer work on modern releases. The problem is this: I have a single public IP address. I want to run multiple jails. Back in the days of FreeBSD 6.*, the accepted way to do this seemed to be to create a new loopback device: # ifconfig lo1 create ... and then add a lot of private 127.0.0.* addresses, one per jail. Then, the real network adapter and the new loopback device were both added to a bridge (if_bridge). Unfortunately, I can't remember the exact details, but I believe that NAT was then enabled on the real interface. In order to filter traffic to, from, and between jails, pf rules were written that filtered the bridge device. This meant that jails could correctly send outbound traffic and receive responses (via pf states), could correctly receive specific inbound traffic (via rdr rules), and traffic in both directions could be filtered based on packets entering and leaving the bridge. However (see my other mailing list post), it seems that now with FreeBSD 10, you just can't add loopback devices to bridges. I can find no evidence of anyone online doing this, or even using the old bridge method that I just described! I can find one post in russian that seems to have the same error that I encounter, but nobody has any idea why it's happening. I can find dozens of blog posts describing how to set up jails on private IP addresses. They all follow the same pattern: 1. Create a loopback device. 2. Create a 127.0.0.* address on the loopback device. 3. Create a jail using the address you just added. 4. Set up pf and enable NAT between the real network adapter and the new loopback device. Unfortunately, at this point, you completely lose the ability to filter outbound jail traffic; All packets sent from a jail will obviously have their source address changed to that of the host and therefore it's not possible to distinguish between outbound host traffic and outbound jail traffic in filter rules. As far as I can tell, people are just not filtering outbound traffic, which seems insane! Is it really impossible to do this with FreeBSD 10? M _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" If you clone lo1, give it a 192.168.x.x/32 IP and then use the following pf= .conf Do you need to bridge the interfaces? You may need to add gateway_enable=3D= "YES" to rc.conf Not sure if that's what you're trying to do? James IP_PUB=3D"Your Public IP Address Here" IP_JAIL=3D"192.168.0.2" NET_JAIL=3D"192.168.0.0/24" PORT_JAIL=3D"{80,443,2020}" scrub in all nat pass on em0 from $NET_JAIL to any -> $IP_PUB rdr pass on em0 proto tcp from any to $IP_PUB port $PORT_WWW -> $IP_JAIL