From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 16:42:20 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 693CCE46 for ; Wed, 26 Mar 2014 16:42:20 +0000 (UTC) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D8AFB1A for ; Wed, 26 Mar 2014 16:42:20 +0000 (UTC) Received: by mail-ob0-f170.google.com with SMTP id uz6so2821238obc.15 for ; Wed, 26 Mar 2014 09:42:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=/8YG+rkmeuUd2RV9dT8F039RXfptjIAp11YNZYCk91I=; b=ijw7SKXGnWxldqdeHtOvDwHgkJ/17fjzrsWZA/OK+fbZBflyA5KsDHBq2cTP4guvel hj/u5qZ/PmdGqtan+D+EvV074UrwRhfg4cX2MitYJgvIywWZSRMH3SidZLFUY8ZCzWpn QOkO5qI7yDWRkqM5kWpr1+yde/m/strHSDP0+HXMOYIrjj5NL1r2OpwCMO8/5LDASIVe 4sEebR7j9S/MUnzJ/vMF+0C7hfpSRtJT9YHUA12tDeMip/HFhWIbPTABMKGhHXDS8KKJ MY4dl39YwFkW9RPteU4UKRgAgSVJ9GO5LraBK+8r3rNV/sljKX6I/Ahte0tPJ3xTWxEd iCpw== MIME-Version: 1.0 X-Received: by 10.182.22.227 with SMTP id h3mr27388189obf.36.1395852139444; Wed, 26 Mar 2014 09:42:19 -0700 (PDT) Received: by 10.76.12.34 with HTTP; Wed, 26 Mar 2014 09:42:19 -0700 (PDT) In-Reply-To: References: <531771C8.1040207@yandex.ru> <20140306145231.Q75313@sola.nimnet.asn.au> <20140307024724.T75313@sola.nimnet.asn.au> Date: Wed, 26 Mar 2014 17:42:19 +0100 Message-ID: Subject: Re: ipfw / routing issue on 9.2-RELEASE From: Andreas Nilsson To: Ian Smith , FreeBSD Net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 16:42:20 -0000 ... snip ... >> I'm wondering what's happening on the outbound path, most of your rules >> handle inbound (to kernel) and it seems that rule 65535 deals with most >> outbound, except those specifically acting on both paths. >> > So do I :) > >> >> Maybe try adding to the above: >> ipfw add 63510 count log ip from table(1) to any out recv table(8) >> and >> ipfw add 64510 count log ip from table(1) to any out recv table(8) >> >> which should catch them on the outbound path before the default rule; >> outbound packets have both receive and transmit interfaces available. >> >> They never get that far :( Those rules log nothing. So I see the packet > coming in, triggering the skipto, triggering the count log ... in recv but > not the count log ... out. > > >> I guess you're sure that these packets are actually going out to some >> external address, not a localhost or alias address (which of course are >> received and not sent out anywhere)? >> > Yes, when the go through they go to external address, which is in the > routing table. > >> >> I guess the above new log lines should show the interface (if any) >> these packets are leaving on, after routing (maybe a routing issue?) >> >> Good luck hunting. If in doubt, throw ever more logging at it! :) And >> please let the list know if you solve it - or not! >> >> cheers, Ian >> > > To make it even more interesting, it tested this on another machine, where > I'm unable to reproduce this "problem". I'm thinking that a reboot might > help, but this is kind of fascinating so I would like to actually find the > cause. I will investigate further. > > Best regards > Andreas > And now it happened on another machine, but different type of traffic. Traffic triggering the divert rule work fine, some traffic not triggering the divert rule does not. After everything works as expected. Before reboot I flushed the firewall so that only default rule was in play ( allow all from any to any ), and then *no* traffic for that destination went out of the box. There are something like ~1000 routes in the routing table. Routes are added and removed according to some triggers, so during uptime of the machine there is a bit of messing with the routing table. At this stage I'm at a loss on how to continue investigating this, so I'll just implement the forwarding via fwd rules in ipfw and altogether ignore the routing table. Best regards Andreas