From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 6 06:00:30 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1C489106568C for ; Tue, 6 Mar 2012 06:00:30 +0000 (UTC) (envelope-from fmdlc.unix@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id C0E0C8FC14 for ; Tue, 6 Mar 2012 06:00:29 +0000 (UTC) Received: by ghrr20 with SMTP id r20so2335397ghr.13 for ; Mon, 05 Mar 2012 22:00:25 -0800 (PST) Received-SPF: pass (google.com: domain of fmdlc.unix@gmail.com designates 10.236.76.133 as permitted sender) client-ip=10.236.76.133; Authentication-Results: mr.google.com; spf=pass (google.com: domain of fmdlc.unix@gmail.com designates 10.236.76.133 as permitted sender) smtp.mail=fmdlc.unix@gmail.com; dkim=pass header.i=fmdlc.unix@gmail.com Received: from mr.google.com ([10.236.76.133]) by 10.236.76.133 with SMTP id b5mr31461081yhe.0.1331013625487 (num_hops = 1); Mon, 05 Mar 2012 22:00:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:reply-to:organization:user-agent:mime-version :to:subject:x-enigmail-version:content-type :content-transfer-encoding; bh=1mi7MkSqw6ArDc+ZI3zjBPJgdQ9Bz0wl3eQeqzI++d0=; b=JlEt4Ph7BIAYjk7hlXQWWFTGW1b7KZr677psyWQTWviZkd/z8AtPbGDVm62cNGzryI lfFda1m3yfNpwimlROmRjjwMpK5gHQGkrJHysSoJwhJiXw3Gxy0o9mdp4G3W6oNLvKtt h+t61k6O+lvV8CPMG40zlB8QzDicm/eG4AZoY56zr75RNyfJunD1a3NFASlgWvVy9hnh uXnTBDHopvY5u0k86KfGZ4zGVp9yEZr43nY53Lo83ggPo67JDreJwk2O7Uz0a9GcQphW sRqdF02ZzQpmRFvXuYmx1eh93g0cUBVPfM5jszTtIxsI7iy0Ym0NsP3+SQ5SbFW16y0c htLA== Received: by 10.236.76.133 with SMTP id b5mr24734036yhe.0.1331012212608; Mon, 05 Mar 2012 21:36:52 -0800 (PST) Received: from [192.168.0.123] ([190.55.104.8]) by mx.google.com with ESMTPS id l9sm28615810anm.15.2012.03.05.21.36.51 (version=SSLv3 cipher=OTHER); Mon, 05 Mar 2012 21:36:52 -0800 (PST) Message-ID: <4F55A271.7030902@gmail.com> Date: Tue, 06 Mar 2012 02:36:49 -0300 From: "Facundo M. de la Cruz" Organization: Code4Life User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.1) Gecko/20120216 Thunderbird/10.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org, Fer Gleiser X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Kernel Panic: Page fault in nat_new function (ipnat) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fmdlc@code4life.com.ar List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Mar 2012 06:00:30 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi guys, At my work we has a network firewall with FreeBSD 8.2-RELEASE 64 bits on a generic box running like a network gateway (making NAT between inbound and outbound connections). This firewall has four PCI ethernet network cards: em0, rl0, rl1, and rl2. The interface *em0* is configured for the WAN connection, and here is where we're using *ipnat*. We has some map and bimap rules at /etc/ipnat.rules file. Repeatedly, the box crashs and the OS is reboteed writting a coredump in /var/crash directory. Checking the coredump file, I can see that the crash is produced by the *fr_checknatin()* function (at /usr/src/sys/modules/ipfilter/../../contrib/ipfilter/netinet/ip_nat.c), in the em0 interface, that then call to a trap (number 12) which becomes fatal and then that produces a kernel panic. Looking in the source code of ip_nat.c i can see the following (I was researching about that :-P) The fr_checknatin() function has two arguments: * fin(I) -> Pointer to the packet information. * passp(I) -> Pointer to the filtering result flag. And the nat_new() function has five arguments: * fin(I) -> Pointer to the packet information. * np(I) -> Pointer to the map rule. * natsave(I) -> Pointer to where to store the NAT struct pointer- * flags(I) -> Flags describing the current packet. * direction(I) -> Direction of the packet (IN/OUT). I understand that this issue is generated by the nat_new function, that create a new data structure for the NAT mappings without checks, and the error is produced when the entry is duplicated. The backtrace generated by the kernel crash is the following: Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 04 fault virtual address = 0x4 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80e25e5c stack pointer = 0x28:0xffffff800010d630 frame pointer = 0x28:0xffffff800010d730 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (em0 taskq) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: #0 0xffffffff805f4e0e at kdb_backtrace+0x5e #1 0xffffffff805c2d07 at panic+0x187 #2 0xffffffff808ac600 at trap_fatal+0x290 #3 0xffffffff808ac9df at trap_pfault+0x28f #4 0xffffffff808acebf at trap+0x3df #5 0xffffffff80894fb4 at calltrap+0x8 #6 0xffffffff80e295d2 at fr_checknatin+0x382 #7 0xffffffff80e432b3 at fr_check+0x793 #8 0xffffffff8067e35c at pfil_run_hooks+0x9c #9 0xffffffff806d2159 at ip_input+0x279 #10 0xffffffff8067d76e at netisr_dispatch_src+0x7e #11 0xffffffff80673b3d at ether_demux+0x14d #12 0xffffffff80673f2d at ether_input+0x17d #13 0xffffffff8032e50a at em_rxeof+0x1ca #14 0xffffffff8032e928 at em_handle_que+0x48 #15 0xffffffff805ffeb5 at taskqueue_run_locked+0x85 #16 0xffffffff8060004e at taskqueue_thread_loop+0x4e #17 0xffffffff805994f8 at fork_exit+0x118 My idea is make the minimal changes in the ipnat syntax and not to migrate to PF, i only need to stop theses reboots, do you know any way to fix it?. Thanks! :-) Regards. - -- Facundo M. de la Cruz (tty0) IT Consultant & Security Researcher RHCSA/RHCE (Red Hat Certified Engineer) GPG fingerprint: DF2F 514A 5167 00F5 C753 BF3B D797 C8E1 5726 0789 "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning." - Rich Cook -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBAgAGBQJPVaJxAAoJENeXyOFXJgeJeIwQAJseoYt2p651LQUyLcoVccQP U3WsNcXb+KAxuP72p1G5t48LaYHJxHoTXjn9o8we2fl9ID0tyJOwZBiFqycoqL3k tw98wLU2czQTxf9zsp9VgAzAdKmpVAgfVu1SHdhC7e0/QT/LI7NJuI0XXwJIJm2L ZVVwGd52+H9wh7t7uzP7s3Qj6uoFRrIZVxqGsl0vq5HwKAf7BtOO1QytYofbjrUc DMDVof78iiQFJYllC3kiQMcalYMUuYY1aBqi3sVU+obq1voYgDAHgpJsSlKnfwhl 2NlXttQ0Qyy7PBxU3RqDSuz1S5ASV2ZV5i4wmtSRTkNWvx2mIVhNgRkSYUx9JgNt TbdNthVtSt+BAFm5Urd2zq3QUGeDH8t/xZT422i0AmOk/2ylTr/AVZQRy2rLlcbt CrHsN8FMjT5l6bouasShh30LosRqp3ruVnAvFGQ1hdSqob0s5I+cgm5wKPl2Qz2c G/MFPdcoad8hlzKBOpl3mRXquTOamMmAuTFEZtjMXdcltr1X85+QpaEBYtVUh2Lt wr3MqkBkorfrMpS9cco+PltD1FFh9+/Nf+LWqOL+n2nwo3JscCZxEAyPzD0Auu1k K5UNR1vgTZL+YvellsOeiyFgOrOHx0UwHksjg8fkq/b5HvgtnME6eu0XQ56K9l2D L8lhY/1TdjVlfCYGUqEr =dvYH -----END PGP SIGNATURE-----