From owner-freebsd-net@FreeBSD.ORG Tue Mar 5 17:39:40 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E953118D for ; Tue, 5 Mar 2013 17:39:40 +0000 (UTC) (envelope-from ncrogers@gmail.com) Received: from mail-vb0-x231.google.com (mail-vb0-x231.google.com [IPv6:2607:f8b0:400c:c02::231]) by mx1.freebsd.org (Postfix) with ESMTP id A22A81D3 for ; Tue, 5 Mar 2013 17:39:40 +0000 (UTC) Received: by mail-vb0-f49.google.com with SMTP id s24so1348745vbi.22 for ; Tue, 05 Mar 2013 09:39:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=7F9ZNFFu2MhjeuJJobpDMVVbM3mN86LBeufB7a0f1Jo=; b=0U0dbviakVyoj1lH4prDPDIkTry/sBF8Isqij9cQqC12SKIu5eAtP9NaYb0KbwiFb0 4Ram6q07P082eBYcKF6TRAkIU89J0iWnVcQ0A3/zhWTtLpaImErjDO2W9TFOhwE6SnwL GhadL0nD/PcbSytEsUo6lUAGS1OtOJKv8jNw4ITfx69MAuroMq1BStSYZzx5OfBfIpID m8uzKxYbX3zl/c5LqLVKP1raqCzb2Aus/icS8u1CwC4AFFyRzPxgi2yYcd18HzO6CpJj 5/4/oOMn+3HPmTbWxhs/T80nvuviS2Ja4//MsCTT1fvsxGzs8nFxMvrLkqeaCXdqPEDI lxGQ== MIME-Version: 1.0 X-Received: by 10.220.227.131 with SMTP id ja3mr8434935vcb.54.1362505180089; Tue, 05 Mar 2013 09:39:40 -0800 (PST) Received: by 10.52.176.131 with HTTP; Tue, 5 Mar 2013 09:39:39 -0800 (PST) Date: Tue, 5 Mar 2013 09:39:39 -0800 Message-ID: Subject: Default route changes unexpectedly From: Nick Rogers To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 17:39:41 -0000 Hello, I am attempting to create awareness of a serious issue affecting users of FreeBSD 9.x and PF. There appears to be a bug that allows the kernel's routing table to be corrupted by traffic routing through the system. Under heavy traffic load, the default route can seemingly randomly change to an IP address that is not directly connected to the network (i.e., is not configured anywhere). Dhclient is not in the mix, nor is routed, bgpd, etc. Running `route monitor` shows no evidence of the change in the default route. The one commonality between all the systems experiencing this problem seems to be the use of PF. Obviously this is a serious problem as it causes all Internet-bound traffic to stop routing until the default route is corrected. Some users, including myself, are working around this problem by installing a script that runs multiple times a second to check if the default route is incorrect and fixing it if necessary, which mitigates the amount of downtime caused by the bug. Please refer to these past posts for more examples and evidence of other users experiencing this problem: http://forums.freebsd.org/showthread.php?p=211610#post211610 http://freebsd.1045724.n5.nabble.com/Default-route-quot-random-quot-gateway-modification-bug-td5750820.html http://lists.freebsd.org/pipermail/freebsd-net/2012-March/031879.html http://lists.freebsd.org/pipermail/freebsd-ipfw/2010-September/004361.html There is also a PR that was incorrectly labeled as an IPFW issue. Myself and others believe this issue is not restricted to the use of IPFW and that the PR should be relabeled. I am inclined to think it is strictly a PF issue since I am not using IPFW, however there is evidence of the default route changing on people using IPFW for past versions of FreeBSD (7.x/8.x), so perhaps this is related. http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/174749 Another PR for the same problem but specific to IPFW and 8.2-RELEASE http://www.freebsd.org/cgi/query-pr.cgi?pr=157796 I am hoping someone reading this can give the problem the attention it deserves. Thank you. -Nick