From owner-freebsd-jail@freebsd.org Sat Oct 24 07:47:44 2015 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9ACBA1D49B for ; Sat, 24 Oct 2015 07:47:44 +0000 (UTC) (envelope-from James@Lodge.me.uk) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1on0108.outbound.protection.outlook.com [157.56.112.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F022DFB7 for ; Sat, 24 Oct 2015 07:47:43 +0000 (UTC) (envelope-from James@Lodge.me.uk) Received: from VI1PR06MB1037.eurprd06.prod.outlook.com (10.162.123.156) by VI1PR06MB1038.eurprd06.prod.outlook.com (10.162.123.157) with Microsoft SMTP Server (TLS) id 15.1.306.13; Sat, 24 Oct 2015 07:47:33 +0000 Received: from VI1PR06MB1037.eurprd06.prod.outlook.com ([10.162.123.156]) by VI1PR06MB1037.eurprd06.prod.outlook.com ([10.162.123.156]) with mapi id 15.01.0306.003; Sat, 24 Oct 2015 07:47:33 +0000 From: James Lodge To: "freebsd-jail@freebsd.org" Subject: Re: Freebsd 10.1 - Ezjail - OpenVPN - Tun Interface Thread-Topic: Freebsd 10.1 - Ezjail - OpenVPN - Tun Interface Thread-Index: AQHRDaWCwzvCKcmAUUSD4bPPASu+QJ55WMqAgAABxDKAAA86AIAACF/vgAAUIwCAAAK2PYAABGCAgAAEqWiAACtZgIAAhw68 Date: Sat, 24 Oct 2015 07:47:32 +0000 Message-ID: References: <562A7147.5080002@freebsd.org> <562A7F88.4070106@freebsd.org> <562A9772.5050408@freebsd.org> <562A9D63.809@freebsd.org> , <562AC5A9.1090106@freebsd.org> In-Reply-To: <562AC5A9.1090106@freebsd.org> Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=James@Lodge.me.uk; x-originating-ip: [46.101.56.132] x-microsoft-exchange-diagnostics: 1; VI1PR06MB1038; 5:xJu1UFtFAaM9K+hHCJ2AEk1AlxO1rBO7+P2cj1041n7hsawqj9GJpnXEQZZUms2Yv6Ftyj9BEKKnQtWagbHNVjnB6qaoQTlF66lx7X37uDom+R72L2E9MoNgLaye+sLQdtr4JGIe2o5tpOA5lEuQpg==; 24:D8TcO5gwkqsVjNp1zVXGRGvi4L9WqdwUbx78nUCsUvy5Z0NGh7rxMbpIqrpWdJDjEsfT+ovU1bv2OUA95r9/zQVoe+YQ2e1z/odFR/hfQqQ=; 20:FGQBf3Y5cGX9YqFcjM0FIANxvD/biil/Ml4kHT1nfZlozg9MxmCRbhR8G7QPT1Q/bN2UuIgixeW3FNa2Z4nVRw== x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(42134001)(42139001); SRVR:VI1PR06MB1038; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(265634631926514); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(520078)(8121501046)(5005006)(3002001)(102215026); SRVR:VI1PR06MB1038; BCL:0; PCL:0; RULEID:; SRVR:VI1PR06MB1038; x-forefront-prvs: 073966E86B x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(53754006)(24454002)(377424004)(45984002)(199003)(40764003)(189002)(86362001)(87936001)(101416001)(54356999)(76176999)(50986999)(11100500001)(5004730100002)(74482002)(122556002)(5008740100001)(10400500002)(107886002)(40100003)(5007970100001)(92566002)(2501003)(5001960100002)(110136002)(450100001)(2900100001)(2950100001)(81156007)(97736004)(80792005)(4001150100001)(5002640100001)(77096005)(15975445007)(189998001)(102836002)(33656002)(5003600100002)(66066001)(2351001)(105586002)(93886004)(106116001)(106356001)(76576001)(19580395003)(19580405001)(74316001); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR06MB1038; H:VI1PR06MB1037.eurprd06.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: Lodge.me.uk does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Lodge.me.uk X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Oct 2015 07:47:32.6104 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: ded56ae9-7c77-4cf6-bbfd-39e6a505742d X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR06MB1038 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2015 07:47:44 -0000 >On 2015-10-23 17:25, James Lodge wrote: > >> On 2015-10-23 16:45, James Lodge wrote: >> >>> On 2015-10-23 15:15, James Lodge wrote: >>> On 2015-10-23 14:13, James Lodge wrote: >>>>> On 2015-10-23 11:37, James Lodge wrote: >>>>> Hello all, >>>>> >>>>> >>>>> I'm trying to build a jail on FreeBSD 10.1 using ezjail in order to r= un OpenVPN. I'm not using vimage and don't particularly want to but I'm hav= ing an issue with networking. >>>>> >>>>> >>>>> OpenVPN daemon is up and running and I can connect successfully as a = client. I receive an IP address as expected, but I cannot route traffic to/= from client/server. The routing table on the client (which is a Windows mac= hine) looks fine so I assume the issue is on the server side. I have a tun = interface created on the host and exposed to the jail via devfs rules. The = IP address on the tun interface is configure on the host and not from the j= ail. I can ping the tun interface IP from the host and the jail, but not fr= om the client when connected. >>>>> >>>>> >>>>> Client---------public IP --------- lo1 (Jail alias Interface)------tu= n0 (OpenVPN Interface) >>>>> >>>>> 10.8.06 x.x.x.x 172.16.1.8 = 10.8.0.1 >>>>> >>>>> >>>>> >>>>> OpenVPN Jail Routing Table: >>>>> >>>>> Internet: >>>>> Destination Gateway Flags Netif Expire >>>>> 172.16.1.8 link#4 UH lo1 >>>>> >>>>> Jail Host Routing Table: >>>>> Internet: >>>>> Destination Gateway Flags Netif Expire >>>>> default x.x.0.1 UGS vtnet0 >>>>> 10.8.0.0 10.8.0.2 UGS tun0 >>>>> 10.8.0.1 link#5 UHS lo0 >>>>> 10.8.0.2 link#5 UH tun0 >>>>> x.x.0.0/18 link#1 U vtnet0 >>>>> x.x.x.x link#1 UHS lo0 >>>>> localhost link#3 UH lo0 >>>>> 172.16.1.1 link#4 UH lo1 >>>>> 172.16.1.2 link#4 UH lo1 >>>>> 172.16.1.3 link#4 UH lo1 >>>>> 172.16.1.4 link#4 UH lo1 >>>>> 172.16.1.5 link#4 UH lo1 >>>>> 172.16.1.6 link#4 UH lo1 >>>>> 172.16.1.7 link#4 UH lo1 >>>>> 172.16.1.8 link#4 UH lo1 >>>>> >>>>> Client Routing Table: >>>>> >>>>> IPv4 Route Table >>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D >>>>> Active Routes: >>>>> Network Destination Netmask Gateway Interface = Metric >>>>> 0.0.0.0 0.0.0.0 10.8.0.5 10.8.0.6 = 20 >>>>> 10.8.0.1 255.255.255.255 10.8.0.5 10.8.0.6 = 20 >>>>> 10.8.0.4 255.255.255.252 On-link 10.8.0.6 = 276 >>>>> 10.8.0.6 255.255.255.255 On-link 10.8.0.6 = 276 >>>>> 10.8.0.7 255.255.255.255 On-link 10.8.0.6 = 276 >>>>> >>>>> >>>>> >>>>> I'm a little stumped as to how to trouble shoot the issue so any help= much appreciated. >>>>> >>>>> >>>>> James >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> freebsd-jail@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-jail >>>>> To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.or= g" >>>>> >>>> >>>>> Try running 'tcpdump -i tun0 -n' on the host, while pining from the >>>>> windows machine, and see if the packets are arriving. >>>>> >>>>> -- >>>>> Allan Jude >>>> >>>> >>>> Thank you Allan, >>>> >>>> I should have thought of tcpdump. So traffic is being received at the = host from the windows client. >>>> >>>> Results from Host tcpdump -i tun0 -n >>>> >>>> 18:44:02.464291 IP 10.8.0.6 > 10.8.0.1: ICMP echo request, id 1, seq 1= 0577, length 40 >>>> 18:44:02.605212 IP 10.8.0.6.56054 > 192.168.0.112.80: Flags [S], seq 5= 12633761, win 8192, options [mss 1368,nop,nop,sackOK], length 0 >>>> 18:44:02.872693 IP 10.8.0.6.57441 > 8.8.8.8.53: 44379+ A? dns.msftncsi= .com. (34) >>>> 18:44:03.864800 IP 10.8.0.6.57441 > 8.8.8.8.53: 44379+ A? dns.msftncsi= .com. (34) >>>> >>>> After that I thought I'd see if the traffic is reaching the jail. Afte= r allow the jail access to /dev/bpf I get the same results as the host, tra= ffic is received. >>>> >>>> Results from Jail tcpdump -i tun0 -n >>>> >>>> 19:09:11.899714 IP 10.8.0.6.58706 > 8.8.8.8.53: 33345+ A? dns.msftncsi= .com. (34) >>>> 19:09:12.728708 IP 10.8.0.6.62332 > 8.8.8.8.53: 22238+ A? dns.msftncsi= .com. (34) >>>> 19:09:12.802903 IP 10.8.0.6.58706 > 8.8.8.8.53: 33345+ A? dns.msftncsi= .com. (34) >>>> 19:09:13.825053 IP 10.8.0.6.57107 > 212.56.71.30.443: Flags [S], seq 3= 139281876, win 8192, options [mss 1368,nop,wscale 8,nop,nop,sackOK], length= 0 >>>> 19:09:13.981307 IP 10.8.0.6.57108 > 212.56.71.30.443: Flags [S], seq 4= 152048904, win 8192, options [mss 1368,nop,wscale 8,nop,nop,sackOK], length= 0 >>>> 19:09:14.628697 IP 10.8.0.6.57100 > 192.168.0.112.80: Flags [S], seq 3= 107463099, win 65535, options [mss 1368,nop,nop,sackOK], length 0 >>>> 19:09:14.814392 IP 10.8.0.6.58706 > 8.8.8.8.53: 33345+ A? dns.msftncsi= .com. (34) >>>> >>>> >>>> Regards >>>> James >>>> _______________________________________________ >>>> freebsd-jail@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-jail >>>> To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org= " >>>> >>>> >>>> Can you include the output of 'ifconfig' from inside the jail?, and >>>> 'netstat -rn' >>>> >>>> It looks like the packets are reaching you on tun0 >>>> >>>> -- >>>> Allan Jude >>> >>> ifconfig from Jail >>> ---------------------- >>> >>> vtnet0: flags=3D8843 metric 0 m= tu 1500 >>> options=3D6c03bb >>> ether 04:01:5d:21:c3:01 >>> media: Ethernet 10Gbase-T >>> status: active >>> >>> vtnet1: flags=3D8802 metric 0 mtu 1500 >>> options=3D6c03bb >>> ether 04:01:5d:21:c3:02 >>> media: Ethernet 10Gbase-T >>> status: active >>> >>> lo0: flags=3D8049 metric 0 mtu 16384 >>> options=3D600003 >>> >>> lo1: flags=3D8049 metric 0 mtu 16384 >>> options=3D600003 >>> inet 172.16.1.8 netmask 0xffffffff >>> >>> tun0: flags=3D8051 metric 0 mtu 1500 >>> options=3D80000 >>> Opened by PID 9024 >>> >>> pflog0: flags=3D141 metric 0 mtu 33160 >>> >>> >>> netstat -rn from Jail >>> --------------------------- >>> >>> Routing tables >>> >>> Internet: >>> Destination Gateway Flags Netif Expire >>> 172.16.1.8 link#4 UH lo1 >>> >>> >>> Regards >>> James >>> >>> >>> >>> >>> _______________________________________________ >>> freebsd-jail@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-jail >>> To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" >>> >>> >>> Look at 'jls' on the host, as your jail doesn't seem to have any IP >>> addresses on tun0. >>> >>> Or, where are you expecting to receive the traffic? >>> >>> -- >>> Allan Jude >> >> >> I expect the traffic to be received within the jail. I find it strange t= hat I don't see the same IP address as what I see on the host. Could this b= e a devfs rule issue? what should I be looking for with jls? >> >> ifconfig from host >> _______________ >> >> >> tun0: flags=3D8051 metric 0 mtu 1500 >> options=3D80000 >> inet 10.8.0.1 --> 10.8.0.2 netmask 0xffffffff >> nd6 options=3D29 >> Opened by PID 9024 >> >> Regards >> James >> >> _______________________________________________ >> freebsd-jail@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-jail >> To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" >> >> >> Jails are only allowed to see the IP addresses that are defined for that >> jail, so you need to add 10.8.0.1 to the list of IP addresses for that >> jail. In ezjail, edit /usr/local/etc/ezjail/jail_name and add the 2nd ip >> after the first, separated with a comma. >> >> -- >> Allan Jude > > Thanks Allan, > > You learn something new everyday! > > So now ifconfig from jail > > tun0: flags=3D8051 metric 0 mtu 1500 > options=3D80000 > inet 10.8.0.1 --> 10.8.0.2 netmask 0xffffffff > Opened by PID 11132 > > > and after allow ICMP through PF on the host I can now ping the tun0 from = the client, so thank you very much for your help. One last thing you might = be able to point me in the right direction of. I need to route client traff= ic on to the Internet. My understanding is IP forwarding can't be enabled w= ithin the jail and adding routes to the jails routing table isn't possible = either. I'm doing NAT at the host, but how do I get the traffic from inside= the jail there. > > Regards > James > > > > > _______________________________________________ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" > > > You should be able to do: > sysrc gateway_enable=3D"YES" > >(temporarily: sysctl net.inet.ip.forwarding=3D1) > >and that should allow packets to move between interfaces. > >-- >Allan Jude Thanks Allan, As always you're a fountain of knowledge.=20 After enabling IP forwarding and amending my PF NAT rule everything is now = working. Once again thank you for your help.=20 Regards James