From owner-freebsd-stable@FreeBSD.ORG Tue Jan 15 14:55:52 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7E16A914 for ; Tue, 15 Jan 2013 14:55:52 +0000 (UTC) (envelope-from lattera@gmail.com) Received: from mail-qc0-f178.google.com (mail-qc0-f178.google.com [209.85.216.178]) by mx1.freebsd.org (Postfix) with ESMTP id 44413DFA for ; Tue, 15 Jan 2013 14:55:52 +0000 (UTC) Received: by mail-qc0-f178.google.com with SMTP id j34so114345qco.23 for ; Tue, 15 Jan 2013 06:55:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sWevvWWjZm0YXhYj06JjvuvmmQkGoMoSHQ5X21ObgTw=; b=ySJxcsp0Tln3sEcZ3dciG6W3W3dvqcY1BSJoG4Aa+Af/AtRbbWPR8dwu5rF4MtXmcq Yr54wm+em+r+uI5kNW33cOiEupAXKYwCJtyML9DLMjOGrpRrIY9OADa76PFiiIl1M+1c T03U8XOcf4RTOc6KFvQ7ujcFI9uaeTYiAUk43RF+osbx0YUCpQozPZKc3sRGbvSQFE4p ypJhuU+hXz6H9P5YD7ZSEJXOh5ScLla05/r/0V9dIzYUumOJv3D9Jn2bTplM3aDIVgn+ ErYVkyJmFiaTYe/NfYMO8w2uUxKRDQ5viKxrYqRN+4GM8zLzmUkzOqgCR5MKwRjCj+jM auYQ== MIME-Version: 1.0 Received: by 10.224.181.135 with SMTP id by7mr72787664qab.51.1358261751621; Tue, 15 Jan 2013 06:55:51 -0800 (PST) Received: by 10.49.25.234 with HTTP; Tue, 15 Jan 2013 06:55:51 -0800 (PST) In-Reply-To: <20130115052937.GA44328@anubis.morrow.me.uk> References: <20130115052937.GA44328@anubis.morrow.me.uk> Date: Tue, 15 Jan 2013 09:55:51 -0500 Message-ID: Subject: Re: IPv6 Tunnel Shared With Jails via epair Devices From: Shawn Webb To: Ben Morrow Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-stable@freebsd.org" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jan 2013 14:55:52 -0000 On Tue, Jan 15, 2013 at 12:29 AM, Ben Morrow wrote: > Quoth Shawn Webb : > > > > I've been working on sharing a 6in4 IPv6 tunnel (via a gif device) I have > > with Hurricane Electric (tunnelbroker.net) to my jails via epair > devices. > > My setup is a bit unique in that the IPv6 tunnel is behind an OpenVPN > > connection. I've had varying degrees of success. I might have a bug to > > report, but I thought I'd post here to get input from people who know > > better than I do about these kinds of things. > > > > I have a bridge device (we'll call it bridge0) with a /64 IPv6 address > > (2001:470:8142:1::1). Each jail's epair[n]b device will get an IPv6 > address > > in that same prefix. For example, one of my jails is 2001:470:8142:1::3. > > The default IPv6 gateway is the IPv6 address of bridge0. > > > > Giving one jail an IP address works fine. For each jail after that, the > > IPv6 address stays in tentative mode. FreeBSD gets stuck trying to use > DAD > > to figure out if there's an address conflict. It never leaves tentative > > mode. This is the bug I'm working out. > > > > Here's bridge0's config: > > > > # ifconfig bridge0 > > bridge0: flags=8843 metric 0 mtu > > 1500 > > ether 02:fe:21:34:d3:00 > > inet6 2001:470:8142:1::1 prefixlen 64 > > nd6 options=21 > > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > > member: epair0a flags=143 > > ifmaxaddr 0 port 19 priority 128 path cost 2000 > > member: epair1a flags=143 > > ifmaxaddr 0 port 21 priority 128 path cost 2000 > > member: bge0 flags=143 > > ifmaxaddr 0 port 5 priority 128 path cost 200000 > > Why have you added the physical interface to the bridge? AFAICT you > don't need to: a bridge will bridge epairs just fine, and as you > explained in that blog post you have to route rather than bridge into > the tunnel, since the tunnel isn't an Ethernet device. > I did it so that I have an IPv4 address directly on the LAN for each of my jails. > > > Here's the relevant epair device for the jail whose IPv6 stack is > working: > > > > # jexec "ClamAV_Dev" ifconfig epair1b > > epair1b: flags=8843 metric 0 mtu > > 1500 > > options=8 > > ether 02:fb:c0:00:16:0b > > inet6 2001:470:8142:1::3 prefixlen 64 > > inet6 fe80::fb:c0ff:fe00:160b%epair1b prefixlen 64 scopeid 0x2 > > inet 10.7.1.172 netmask 0xfffffe00 broadcast 10.7.1.255 > > nd6 options=21 > > media: Ethernet 10Gbase-T (10Gbase-T ) > > status: active > > > > Here's the relevant epair device for the jail whose IPv6 stack isn't > > working: > > > > # jexec "Dev Template" ifconfig epair0b > > epair0b: flags=8843 metric 0 mtu > > 1500 > > options=8 > > ether 02:80:03:00:14:0b > > inet6 2001:470:8142:1::5 prefixlen 64 tentative > > inet6 fe80::80:3ff:fe00:140b%epair0b prefixlen 64 tentative scopeid 0x2 > > inet 10.7.1.92 netmask 0xfffffe00 broadcast 10.7.1.255 > > nd6 options=29 > > I suspect the addresses are only marked tentative because the interface > has been marked IFDISABLED. This causes all current addresses to be > marked tentative, because the kernel isn't allowed to send or receive > IPv6 packets and so can't defend the addresses any more. > > Is it possible something in the jail's startup scripts is causing the > interface to be marked IFDISABLED after the inet6 address has been > assigned? Some of the functions in network.subr mark interfaces > IFDISABLED automatically if they don't think they have IPv6 addresses. > I was thinking the same thing. One problem is that I can't remove the IFDISABLED flag. This is what happens when I try: # jexec "Dev Template" ifconfig epair0b -ifdisabled ifconfig: ioctl(SIOCGIFINFO_IN6): Invalid argument > > > media: Ethernet 10Gbase-T (10Gbase-T ) > > status: active > > > > I brought up the "Dev Template" jail after bringing up the ClamAV_Dev > jail. > > If there's any other output you'd like to see, let me know. If you're > > confused about my setup, visit my blog post about the subject here: > > > http://0xfeedface.org/blog/lattera/2013-01-12/tunneled-ipv6-freebsd-jails > > > > I'm curious to know if I've got a legit bug or if it's something I'm > doing > > wrong. The one thing I haven't tried is setting up rtadvd on the bridge. > > That'd be kindof interesting, since my physical NIC is a member on the > > bridge. I'd rather not dish out IPv6 addresses for all devices on the > > network (a network with lots of devices I don't own or control). > > As I said, I don't believe you need the physical interface on the > bridge, unless you have to for IPv4 (and you can't route or proxyarp > instead). However, before you can run rtadvd you will need to give the > bridge its proper link-local address, which probably also means locking > down its hardware address in rc.conf. Bridges don't get auto link-local > addresses, for reasons I've never entirely understood, and RAs have to > use ll addresses. > > You will need to set up routing so that packets coming in through the > tunnel destined for the jails get routed out of the bridge, and packets > coming in on the bridge destined for the IPv6 Internet get routed out of > the tunnel. Probably that will have happened already, just by assigning > an inet6 address and prefixlen to the bridge and the default inet6 route > to the tunnel. > Routing is already set up properly. The first jail that boots up has a working IPv6 stack. The problem is with jails booted up after the first one has been booted up. > > Ben > > Thanks for the help, Ben.