From owner-freebsd-current@FreeBSD.ORG Thu Nov 25 20:08:48 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 847D21065781 for ; Thu, 25 Nov 2010 20:08:48 +0000 (UTC) (envelope-from kma@mrecic.gov.ar) Received: from mx1.mrecic.gov.ar (mx1.mrecic.gov.ar [200.16.99.221]) by mx1.freebsd.org (Postfix) with ESMTP id A78D18FC1A for ; Thu, 25 Nov 2010 20:08:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArYDAJtK7kyMvzAmgWdsb2JhbACCAoFNnmdYFQEBFiIir0qQRoRUcwSKZYNB X-IronPort-AV: E=Sophos;i="4.59,256,1288580400"; d="scan'208,217";a="21720925" Received: from mrelmx07.mrec.ar ([140.191.48.38]) by mx1.mrecic.gov.ar with ESMTP; 25 Nov 2010 16:38:58 -0300 Received: from localhost (localhost.localdomain [127.0.0.1]) by mrelmx07.mrec.ar (Postfix) with ESMTP id 0AB8671DF9 for ; Thu, 25 Nov 2010 16:38:59 -0300 (ART) X-Virus-Scanned: amavisd-new at mrelmx07.mrec.ar Received: from mrelmx07.mrec.ar ([127.0.0.1]) by localhost (mrelmx07.mrec.ar [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Wqu+g7sXz1PT for ; Thu, 25 Nov 2010 16:38:58 -0300 (ART) Received: from mrelmx06.mrec.ar (mrelmx10.mrec.ar [140.191.48.45]) by mrelmx07.mrec.ar (Postfix) with ESMTP id 824F971DF8 for ; Thu, 25 Nov 2010 16:38:58 -0300 (ART) Date: Thu, 25 Nov 2010 17:38:58 -0200 (ARST) From: Kevin Mai To: freebsd-current@freebsd.org Message-ID: <1832177924.3260.1290713938422.JavaMail.root@mrelmx10.mrec.ar> In-Reply-To: <682762629.3247.1290713696638.JavaMail.root@mrelmx10.mrec.ar> MIME-Version: 1.0 X-Originating-IP: [140.191.48.40] X-Mailer: Zimbra 6.0.6_GA_2330.DEBIAN5_64 (ZimbraWebClient - FF3.0 (Linux)/6.0.6_GA_2330.DEBIAN5_64) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Issues with Jails/Routes/FIBs X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Nov 2010 20:08:48 -0000 Hi folks! I'm facing an issue here while trying to define separate routing tables for each jail and host. Let me show you briefly how it's done: The server has 3 physical NICs, each one connected to a different network (say, public network A, public network B, and LAN). Currently, the default gateway is set to be the LAN gateway, even though the two jails can see their own public network subnet. Internet: Destination Gateway Flags Refs Use Netif Expire default 172.16.2.1 UGS 1 3935 bce2 127.0.0.1 link#5 UH 0 0 lo0 172.16.2.0/24 link#3 U 0 0 bce2 172.16.2.127 link#3 UHS 0 0 lo0 100.16.97.0/24 link#1 U 0 0 bce0 100.16.97.5 link#1 UHS 0 0 lo0 100.16.98.0/24 link#2 U 0 0 bce1 100.16.98.5 link#2 UHS 0 0 lo0 100.16.97.0/24 and 100.16.98.0/24 are the two public networks and 172.16.2.0/24 is the LAN. I have already tried removing devfs rules from the jails, setting securelevel to -1 but I'm still out of luck.. I know setfib can define alternate routing tables, and I even created a default gateway for two fibs, 1 & 2: [root@mrefns09 ~]# setfib 2 netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 100.16.98.100 UGS 14 906 bce1 127.0.0.1 link#5 UH 0 0 lo0 172.16.2.0/24 link#3 U 0 0 bce2 100.16.97.0/24 link#1 U 0 39 bce0 100.16.98.0/24 link#2 U 0 0 bce1 [root@mrefns09 ~]# setfib 1 netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 100.16.97.100 UGS 0 1758 bce0 127.0.0.1 link#5 UH 0 0 lo0 172.16.2.0/24 link#3 U 0 0 bce2 100.16.97.0/24 link#1 U 0 44 bce0 100.16.98.0/24 link#2 U 0 4 bce1 And i've added the proper settings in rc.conf.. jail_athea97_ip="100.16.97.5 netmask 255.255.255.0" jail_athea97_fib=1 jail_athea98_ip="100.16.98.5 netmask 255.255.255.0" jail_athea98_fib=2 Am I missing something? because once I get into the jail the routing table is the same: [root@athea97 /]# netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 172.16.2.1 UGS 13 6175 bce2 127.0.0.1 link#5 UH 0 0 lo0 172.16.2.0/24 link#3 U 0 0 bce2 172.16.2.127 link#3 UHS 0 0 lo0 100.16.97.0/24 link#1 U 0 0 bce0 100.16.97.5 link#1 UHS 0 0 lo0 100.16.98.0/24 link#2 U 0 0 bce1 100.16.98.5 link#2 UHS 0 0 lo0 [root@athea97 /]# setfib 1 netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 100.16.97.100 UGS 15 1814 bce0 127.0.0.1 link#5 UH 0 0 lo0 172.16.2.0/24 link#3 U 0 0 bce2 100.16.97.0/24 link#1 U 0 44 bce0 100.16.98.0/24 link#2 U 0 4 bce1 The other jail is acting the same way. I know that since I'm doing a jexec, the shell will have the host's route because, but, how can I know if it's getting the alternate routing table? Thanks, Kevin