From owner-freebsd-net@FreeBSD.ORG Mon Apr 6 18:26:54 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F955295 for ; Mon, 6 Apr 2015 18:26:54 +0000 (UTC) Received: from mail-ob0-x22f.google.com (mail-ob0-x22f.google.com [IPv6:2607:f8b0:4003:c01::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3ECA7F0F for ; Mon, 6 Apr 2015 18:26:54 +0000 (UTC) Received: by obbfy7 with SMTP id fy7so53425244obb.2 for ; Mon, 06 Apr 2015 11:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Nve5fEhzz+eENzyoH1ewYNIa4bBKuzhCBhjRxTALCCw=; b=YArzdpxjHzgI3WyASVFsi9uwtWuOrOmnZ1+KGUN1tRm/XrQBEt+cC2jX0+s+DX7BWc xvGMdi9+YlzLTiOCAXoKtHiW+gnsoXqR56wVJ4S5rkBueaFrWY3kbjqrRHwEmVE/9WyX U45SvDeydZzBUtFiGF1qFOoQtrr1D8OiZL01SHM30gSyywlz0DnaFJvl+QC44AYSNL7t N5rBZIlVM3L8vVjcFioYlRJy1FAoZP/gsHiVIQpdkzgIh0Ck7FnnRTN4VjBZsZfPTJK+ ESEXK6QnH7Ms0pMaG3DLBnCp5xuM5HPhxHy+DN/1aOydaj1oK8Q/wB2vWbB92LPdh7se qtlw== MIME-Version: 1.0 X-Received: by 10.182.241.99 with SMTP id wh3mr20352307obc.81.1428344813417; Mon, 06 Apr 2015 11:26:53 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.202.215.7 with HTTP; Mon, 6 Apr 2015 11:26:53 -0700 (PDT) In-Reply-To: References: Date: Mon, 6 Apr 2015 12:26:53 -0600 X-Google-Sender-Auth: i4dnUevXStnj3BcbMpcUPxztXGc Message-ID: Subject: Re: FreeBSD sometimes uses the router for packets on the local network From: Alan Somers To: Anton Farber Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 18:26:54 -0000 On Mon, Apr 6, 2015 at 12:15 PM, Anton Farber wrote: > Good evening, > > I've opened a thread on the FreeBSD networking forum (https://forums.free= bsd.org/threads/jail-fails-to-connect-to-main-host.50833/) as sometime ago = my FreeBSD server (initially running 10.1, now CURRENT) started to behave s= trangely after an upgrade from 10.0 to 10.1. I first noticed that a jail (1= 92.168.1.5) wasn't able to contact the base system (192.168.1.1). Running a= tcpdump revealed the following: the jail is using em0 instead of lo0 for c= ommunicating with the base system: You need to look at your routing tables. From inside the jail, run "netstat -rn -f inet". You probably won't see any entry for 127.0.0.1 or 127.0.0.0/8. Those are the entries that your jail needs in order to talk to the base system. You can add them, but think carefully. Many server processes, such as ntpd, have reduced security for connections coming over 127.0.0.1. Whether or not it is appropriate to add those routes depends on why you are using a jail. -Alan > > ------------------------ > * tcpdump on em0 > Source Destination Protocol Length Info > 192.168.1.5 192.168.1.1 TCP 74 28848=E2=86=9222 [SYN] > * tcpdump on lo0 > Source Destination Protocol Length Info > 192.168.1.1 192.168.1.5 TCP 64 22=E2=86=9228848 [SYN, ACK] > ------------------------ > > I don't think that this is the way it's supposed to work. Next thing was,= that random hosts on the network were unable to contact the server. The se= rver on the other hand thought it needed to reach those hosts (in this exam= ple 192.168.1.61) via the router: > > ------------------------ > # ping -c 1 192.168.1.61 > PING 192.168.1.61 (192.168.1.61): 56 data bytes > 36 bytes from router.local.lan (192.168.1.254): Redirect Host(New addr: 1= 92.168.1.61) > Vr HL TOS Len ID Flg off TTL Pro cks Src Dst > 4 5 00 0054 a734 0 0000 40 01 4fe6 192.168.1.1 192.168.1.61 > > 64 bytes from 192.168.1.61: icmp_seq=3D0 ttl=3D64 time=3D1.009 ms > > --- 192.168.1.61 ping statistics --- > 1 packets transmitted, 1 packets received, 0.0% packet loss > round-trip min/avg/max/stddev =3D 1.009/1.009/1.009/0.000 ms# > ------------------------ > > Other hosts on the same network were able to reach 192.168.1.61 directly,= so it can't be neither 192.168.1.61 nor the router (I do not have any stat= ic routes for 192.168.1.61/32 on my router) that are causing the problems. = Status on the server: > > ------------------------ > # arp -a > server.local.Lan (192.168.1.1) at 00:1c:c0:6f:c2:60 on em0 permanent [eth= ernet] > laptop.local.Lan (192.168.1.32) at 5c:26:0a:2a:37:10 on em0 expires in 11= 97 seconds [ethernet] > jail.local.Lan (192.168.1.5) at 00:1c:c0:6f:c2:60 on em0 permanent [ether= net] > WinXP.local.Lan (192.168.1.4) at 08:00:27:80:b8:10 on em0 expires in 1200= seconds [ethernet] > ? (192.168.1.61) at 00:04:20:05:31:38 on em0 expires in 1150 seconds [eth= ernet] > ? (192.168.1.255) at (incomplete) on em0 expired [ethernet] > Router.local.Lan (192.168.1.254) at 00:0d:b9:00:11:68 on em0 expires in 1= 150 seconds [ethernet] > Phone.local.Lan (192.168.1.21) at 00:0e:08:bc:ed:94 on em0 expires in 799= seconds [ethernet] > # netstat -rn > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > default 192.168.1.254 UGS em0 > 127.0.0.1 link#2 UH lo0 > 192.168.1.0/24 link#1 U em0 > 192.168.1.1 link#1 UHS lo0 > 192.168.1.5 link#1 UHS lo0 > 192.168.1.5/32 link#1 U em0 > > Internet6: > Destination Gateway Flags = Netif Expire > ::/96 ::1 UGRS = lo0 > ::1 link#2 UH = lo0 > ::ffff:0.0.0.0/96 ::1 UGRS = lo0 > fe80::/10 ::1 UGRS = lo0 > fe80::%lo0/64 link#2 U = lo0 > fe80::1%lo0 link#2 UHS = lo0 > fe80::%tun0/64 link#4 U = tun0 > fe80::21c:c0ff:fe6f:c260%tun0 link#4 UHS = lo0 > ff02::/16 ::1 UGRS = lo0 > ------------------------ > > So the server knew the (correct) MAC address of .61 but still tried to re= ach it via the router. I've run a tcpdump on em0, this is where things get = interesting: > > ------------------------ > No. Time Source Destination Protocol Length Info > 186 1.028186 192.168.1.1 192.168.1.61 ICMP 98 Echo (ping) requ= est id=3D0x2220, seq=3D0/0, ttl=3D64 (reply in 190) > > Ethernet II, Src: IntelCor_6f:c2:60 (00:1c:c0:6f:c2:60), Dst: PcEngine_00= :11:68 (00:0d:b9:00:11:68) > > No. Time Source Destination Protocol Length Info > 189 1.029008 192.168.1.254 192.168.1.1 ICMP 70 Redirect = (Redirect for host) > > Ethernet II, Src: PcEngine_00:11:68 (00:0d:b9:00:11:68), Dst: IntelCor_6f= :c2:60 (00:1c:c0:6f:c2:60) > > No. Time Source Destination Protocol Length Info > 190 1.029392 192.168.1.61 192.168.1.1 ICMP 98 Echo (ping) reply= id=3D0x2220, seq=3D0/0, ttl=3D64 (request in 186) > > Ethernet II, Src: SlimDevi_05:31:38 (00:04:20:05:31:38), Dst: IntelCor_6f= :c2:60 (00:1c:c0:6f:c2:60) > ------------------------ > > The server issues an echo request for .61 but is using the router's MAC a= ddress (:68)! Obviously the router answers with a redirect.. > > I've shut down all daemons and kernel modules if possible until just the = absolute bare minimum (sshd, init and some other base services) was running= . I've upgraded to CURRENT. This didn't change anything. What helped in the= end was issuing a "route flush ; route add default 192.168.1.254": the ser= ver was immediately able to contact .61 directly. > > The problem seemingly appears randomly, sometimes it takes weeks for it t= o reappear. > > Sorry for the long post, I've tried to keep it as compact as possible. Do= es anyone have any idea what might be causing those problems? > > Regards, Anton > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"