From owner-freebsd-virtualization@FreeBSD.ORG Wed Oct 7 04:50:08 2009 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AB5C106568B for ; Wed, 7 Oct 2009 04:50:08 +0000 (UTC) (envelope-from remodeler@alentogroup.org) Received: from courriel.marmotmail.com (courriel.marmotmail.com [85.17.36.172]) by mx1.freebsd.org (Postfix) with ESMTP id 314A68FC13 for ; Wed, 7 Oct 2009 04:50:07 +0000 (UTC) Received: from bruce.epifora.com (localhost.local [127.0.0.1]) by courriel.marmotmail.com (Postfix) with ESMTP id 629202396EE for ; Wed, 7 Oct 2009 08:05:32 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by bruce.epifora.com (Postfix) with ESMTP id 6B6924761F9 for ; Wed, 7 Oct 2009 01:03:24 -0400 (EDT) Received: from bruce.epifora.com ([127.0.0.1]) by localhost (bruce.epifora.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 23380-10 for ; Wed, 7 Oct 2009 01:03:22 -0400 (EDT) Received: from alentogroup.org (localhost [127.0.0.1]) by bruce.epifora.com (Postfix) with ESMTP id 673B24761F8 for ; Wed, 7 Oct 2009 01:03:22 -0400 (EDT) From: "remodeler" To: freebsd-virtualization@freebsd.org Date: Wed, 7 Oct 2009 01:03:22 -0400 Message-Id: <20091007043721.M28730@alentogroup.org> In-Reply-To: <4ACBF0ED.2070905@elischer.org> References: <20091007002615.M76095@alentogroup.org> <4ACBF0ED.2070905@elischer.org> X-OriginatingIP: 127.0.0.1 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: Re: can't find routing entry for network routes X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Oct 2009 04:50:08 -0000 On Tue, 06 Oct 2009 18:37:49 -0700, Julian Elischer wrote > please recap with a script that fails Thank you for your response Julian. I very much respect the work everyone has done on netgraph / vimage / jails, and also the help extended to me. Kernel options in addition to amd64 GENERIC are geom_journal, ufs_gjournal, geom_mirror, geom_eli, vimage, netgraph, netgraph_bridge, netgraph_ether, and netgraph_eiface. Additional devices are crypto. World and kernel are in sync. I have been testing by csup'ing from head, but I have the same errors in 8.0 beta1, beta3, and rc1. # make a bridge and connect the physical ethernet interface to it ngctl mkpeer msk0: bridge lower link0 ngctl name msk0:lower bridge0 ngctl connect msk0: bridge0: upper link1 # Start Name Server Jail jail -c -l -U root -n ns host.hostname=ns.my.org path=\ /jail/j/ns vnet persist mount -t procfs proc /jail/j/ns/proc mount -t devfs dev /jail/j/ns/dev devfs -m /jail/j/ns/dev rule -s 4 applyset mount -t fdescfs null /jail/j/ns/dev/fd ngctl mkpeer eiface ether ether ngctl connect ngeth0: bridge0: ether link2 ifconfig ngeth0 vnet ns vimage ns ifconfig lo0 localhost vimage ns ifconfig ngeth0 link 02:0a:0b:0c:01:01 vimage ns ifconfig ngeth0 172.26.64.10 vimage ns route add default -link 00:23:54:08:2b:f7 This results in an error on applying the devfs ruleset, so I see all of root's /dev in the jail. I receive an error on the route command. I get identical errors when I specify / as the path and omit the mounts/devfs command. My expectation is that this would leave me with two network stacks, msk0 and ngeth0, with msk0 connected to the ng_bridge by its upper and lower hooks and ngeth0 by its ether hook. I would expect network connectivity over the bridge, and to be able to manipulate the vnet jail's FIB from the host to add a default route. Most of what I expect happens: # ngctl list There are 5 total nodes: Name: bridge0 Type: bridge ID: 00000004 Num hooks: 3 Name: ipfw Type: ipfw ID: 00000001 Num hooks: 0 Name: ngeth0 Type: eiface ID: 00000008 Num hooks: 1 Name: ngctl1633 Type: socket ID: 0000000a Num hooks: 0 Name: msk0 Type: ether ID: 00000002 Num hooks: 2 # vimage -l ns # jls JID IP Address Hostname Path 1 - ns.my.org /jail/j/ns but the devfs ruleset command fails (executed on the host): # devfs -m /jail/j/ns/dev rule -s 4 applyset devfs rule: ioctl DEVFSIO_SAPPLY: No such process and in the jail: ns# # devfs -m /jail/j/ns/dev rule -s 4 applyset #: Command not found. ns# fs rule: ioctl DEVFSIO_SAPPLY: No such processdevfs rule: ioctl DEVFSIO_SAPPLY: No such process adding the route from the host: # vimage ns route add default -link 00:23:54:08:2b:f7 route: writing to routing socket: Network is unreachable add net default: gateway 00:23:54:08:2b:f7: Network is unreachable and from the jail: ns# route add default -link 00:23:54:08:2b:f7 route: writing to routing socket: Network is unreachable add net default: gateway 00:23:54:08:2b:f7: Network is unreachable I get the same error for netstat -r from the host and the jail: # netstat -r netstat: kvm not available: /dev/mem: Permission denied Routing tables rt_tables: symbol not in namelist Before I compiled in Bjorn's patches, netstat -r worked properly on the host. The host has network connectivity. If I boot without starting the jail, everything works as I expect on the host (haven't tested that very far since the patches). Thank you.