From owner-freebsd-net@FreeBSD.ORG Mon Jun 9 10:08:55 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FC3E570 for ; Mon, 9 Jun 2014 10:08:55 +0000 (UTC) Received: from 0.mx.codelabs.ru (0.mx.codelabs.ru [144.206.233.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C36E323E4 for ; Mon, 9 Jun 2014 10:08:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codelabs.ru; s=three; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=hXDxkRQrHZj/enlHHqQfandVqnhcmAFZwJu8cDBCVcU=; b=iHOS9ZOMenwgfDCKYFp4gSVNHtGG3xu+pVEvy6RKCIK4GPRTsVYApbXFsr9jZrif0surecWJlL8Jf1TLdH6NYiXeTDVhCbAzv3FCy8lYndlO+JGIdVbTwkvLoRjwwqBuFerCx4QTwearaD9BQOSxNOvco4UhU0hlysqOU7ZTJ9bx7v9C54NXbJ3Z+zNGnpeNMNP2RvBnIW/jCkyXHd/UkqwZHs07Bp3m12pXk88QRjcsB4zg59DNIcBkkw36/IGHKCbfcV5+lcdtar/Rmva3nPsEFrIkNrROCVNZq3OSD5BxYJCks7yV0xgBvSEbyl7RuB+UHT+jbIShKbDeUMLcKzrH+a3GaNkWjYmcTBZmq0cvvwgnJcAQ+vZS+0zmsClyvaLvkcsXdFLH3o7gJP5OKp4hH0Xa50mJyzn1hd9HIMEeND6m1aCal62oVWe8vKN6tP9tvsNu/ZW7YgA1gc7Y8tTCuUo3G/Q27NunC6k1+6drbL5fD9R43GnH7T4Ok1vWZwuRtjA5tBomt0/N9M0oQxRH14TArQhxeckHmwT53HkmzsJP+pBkEOd3IufCDYF5KtdVNBnuo4Nzdj55aHqxoAUsRn6BFp7mL0Zk9/65WcWBv9z6KafCOnADSv19dAbuxDiEyAxlZ8HXXxV4TyIYA0pD8ZYVPOHq8j3hL9xhO2c=; Received: from void.codelabs.ru (void.codelabs.ru [144.206.233.66]) by 0.mx.codelabs.ru with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) id 1WtwVP-000A7I-8A; Mon, 09 Jun 2014 14:08:43 +0400 Date: Mon, 9 Jun 2014 14:08:41 +0400 From: Eygene Ryabinkin To: None Secure Subject: Re: Does FreeBSD have the ability to properly forward UDP traffic ? Message-ID: References: <1402122333.57974.YahooMailNeo@web162101.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="X0vpKvTpCy87tk9a" Content-Disposition: inline In-Reply-To: <1402122333.57974.YahooMailNeo@web162101.mail.bf1.yahoo.com> Sender: rea@codelabs.ru Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2014 10:08:55 -0000 --X0vpKvTpCy87tk9a Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Good day. Fri, Jun 06, 2014 at 11:25:33PM -0700, None Secure via freebsd-net wrote: > I would like very much to use sshuttle for an informal VPN. >=20 > However, sshuttle sets up a lot of complexity in order to route DNS > requests over the ssh tunnel ... it uses divert rules for dns > traffic, and I don't think they even tested it because it fails to > start or utilize natd. sshuttle doesn't need natd, divert(4) sockets are in use. It is the mechanism that is used to forward packets from, say, ipfw to the userland and back to ipfw. You have divert rule in ipfw that puts a packet to the divert(4) socket that is identified by it's numeric port (that has nothing to do with the TCP/UDP ports, it is just an identified for different divert(4) listeners). User-space application listent to this socket and can read and write packets, thus consuming input ones and generating output ones. Sshuttle listens to the divert socket: {{{ if dnsport: divertsock =3D socket.socket(socket.AF_INET, socket.SOCK_RAW, IPPROTO_DIVERT) divertsock.bind(('0.0.0.0', port)) # IP field is ignored nslist =3D resolvconf_nameservers() for ip in nslist: # relabel and then catch outgoing DNS requests ipfw('add', sport, 'divert', sport, 'udp', 'from', 'any', 'to', '%s/32' % ip, '53', 'not', 'ipttl', '42') # relabel DNS responses ipfw('add', sport, 'divert', sport, 'udp', 'from', 'any', str(dnsport), 'to', 'any', 'not', 'ipttl', '42') def do_wait(): while 1: r,w,x =3D select.select([sys.stdin, divertsock], [], []) if divertsock in r: _handle_diversion(divertsock, dnsport) if sys.stdin in r: return else: do_wait =3D None return do_wait def _handle_diversion(divertsock, dnsport): p,tag =3D divertsock.recvfrom(4096) src,dst =3D _udp_unpack(p) debug3('got diverted packet from %r to %r\n' % (src, dst)) if dst[1] =3D=3D 53: # outgoing DNS debug3('...packet is a DNS request.\n') _real_dns_server[0] =3D dst dst =3D ('127.0.0.1', dnsport) elif src[1] =3D=3D dnsport: if islocal(src[0]): debug3('...packet is a DNS response.\n') src =3D _real_dns_server[0] else: log('weird?! unexpected divert from %r to %r\n' % (src, dst)) assert(0) newp =3D _udp_repack(p, src, dst) divertsock.sendto(newp, tag) }}} so it doesn't need natd at all. > The stated reason by sshuttle project is that you can't just forward > UDP traffic properly with BSD, like you can with linux - they say it > doesn't keep track of port numbers or connections properly. Are you referring to the following comment from https://github.com/apenwarr/sshuttle/blob/master/firewall.py {{{ # This part is much crazier than it is on Linux, because MacOS (at least # 10.6, and probably other versions, and maybe FreeBSD too) doesn't # correctly fixup the dstip/dstport for UDP packets when it puts them # through a 'fwd' rule. It also doesn't fixup the srcip/srcport in the # response packet. In Linux iptables, all that happens magically for us, # so we just redirect the packets and relax. # # On MacOS, we have to fix the ports ourselves. For that, we use a # 'divert' socket, which receives raw packets and lets us mangle them. # # Here's how it works. Let's say the local DNS server is 1.1.1.1:53, # and the remote DNS server is 2.2.2.2:53, and the local transproxy port # is 10.0.0.1:12300, and a client machine is making a request from # 10.0.0.5:9999. We see a packet like this: # 10.0.0.5:9999 -> 1.1.1.1:53 # Since the destip:port matches one of our local nameservers, it will # match a 'fwd' rule, thus grabbing it on the local machine. However, # the local kernel will then see a packet addressed to *:53 and # not know what to do with it; there's nobody listening on port 53. Thu= s, # we divert it, rewriting it into this: # 10.0.0.5:9999 -> 10.0.0.1:12300 # This gets proxied out to the server, which sends it to 2.2.2.2:53, # and the answer comes back, and the proxy sends it back out like this: # 10.0.0.1:12300 -> 10.0.0.5:9999 # But that's wrong! The original machine expected an answer from # 1.1.1.1:53, so we have to divert the *answer* and rewrite it: # 1.1.1.1:53 -> 10.0.0.5:9999 # # See? Easy stuff. }}} ipfw(8) 'fwd' rule isn't touching the internals of the packet, it really just forwards it "as is". This has nothing to do with UDP, 'fwd' is just intended to be used for transparent proxying, not for "real" forwarding that implies rewriting packet contents. sshuttle folks say about UDP just because of the following property of SSH tunnelling: when you're doing it, you have something that listens to localhost:N on TCP. So you can forward packets to localhost:N without any particular problems, they will be consumed by the proper application (SSH) and be forwarded to the remote end. But for DNS packets sshuttle does the following: it just forwards them to the local host and some local port where sshuttle's own DNS proxy lives. So, in reality, there is no forwarding or tunnelling of DNS/UDP requests in the normal sense: sshuttle wants to proxy them via itself. And for this it has to mangle the packets via divert sockets on FreeBSD. What I don't quite understand is that why sshuttle needs this dance of diverting and can't just consume incoming DNS packet like SSH's tunneling port will, reinterepret it and put the answer back with the proper src/dst fields inside the UDP packet. But probably there's some explanation: I'll try to think of it later. > Or is it possible to properly forward UDP traffic with ipfw rules, > and not use natd/divert ? What is your real problem with sshuttle? If you concerned that no natd(8) is spawned, then you shouldn't worry about that. If something doesn't work for you, please, be specific, what doesn't work, which FreeBSD and sshuttle versions you're using and what is your configuration. Hope that helps. --=20 Eygene Ryabinkin ,,,^..^,,, [ Life's unfair - but root password helps! | codelabs.ru ] [ 82FE 06BC D497 C0DE 49EC 4FF0 16AF 9EAE 8152 ECFB | freebsd.org ] --X0vpKvTpCy87tk9a Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iL4EABEKAGYFAlOVh6lfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDgyRkUwNkJDRDQ5N0MwREU0OUVDNEZGMDE2 QUY5RUFFODE1MkVDRkIACgkQFq+eroFS7Pu5WAD/RzMECcPfreYGH2fzllTJanwU kWX8BqBW3G03ScSIx/gA/3eSXHbjDCMzpfTVcAugSPtYpJLrhdrlyWE1TMsDYH0I =JlhY -----END PGP SIGNATURE----- --X0vpKvTpCy87tk9a--