From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 6 05:47:42 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0880F106566B for ; Tue, 6 Mar 2012 05:47:42 +0000 (UTC) (envelope-from fmdlc.unix@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id AC93C8FC23 for ; Tue, 6 Mar 2012 05:47:41 +0000 (UTC) Received: by ghrr20 with SMTP id r20so2333359ghr.13 for ; Mon, 05 Mar 2012 21:47:41 -0800 (PST) Received-SPF: pass (google.com: domain of fmdlc.unix@gmail.com designates 10.236.116.73 as permitted sender) client-ip=10.236.116.73; Authentication-Results: mr.google.com; spf=pass (google.com: domain of fmdlc.unix@gmail.com designates 10.236.116.73 as permitted sender) smtp.mail=fmdlc.unix@gmail.com; dkim=pass header.i=fmdlc.unix@gmail.com Received: from mr.google.com ([10.236.116.73]) by 10.236.116.73 with SMTP id f49mr30320834yhh.97.1331012861132 (num_hops = 1); Mon, 05 Mar 2012 21:47:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:reply-to:organization:user-agent:mime-version :to:subject:x-enigmail-version:content-type :content-transfer-encoding; bh=3ZHrMimTKaF9gNwLCfh/YgN75FpOxUGULrcwX+NSblo=; b=W+QhVvdraWnw/Vn0lqHHFP/5/JxTOF62YloFOS5W+D0Piep6Ww3DVAOUR4dPDqWxoL X3bT7wIFb8iMGDjN7fz3vDbvSvKAdhFFc+agRB+44gBbCkAeg+9mmwdRm8czDM4eaRcA HH24+UYA3z0VNkR2LgoS/Seq7NgRFtfJNQzjvuVAJI8fHLYYFur8epjyP4PIBHFyUDTz NVdXnSgGRp2OU3mFXbBHVIQZM+8RhuzQY4ZEMpxXtdSjomgUynCPBkjLibNoXhfzh4ap a63ilk9hJyqQ6otBP5ao+a8JEyRbZ57HWqAFbYfhQ2gfvsKNkML7OtrWOuU6cgJVEHmQ uFnw== Received: by 10.236.116.73 with SMTP id f49mr23971917yhh.97.1331012861079; Mon, 05 Mar 2012 21:47:41 -0800 (PST) Received: from [192.168.0.123] ([190.55.104.8]) by mx.google.com with ESMTPS id w44sm46304131yhk.17.2012.03.05.21.47.40 (version=SSLv3 cipher=OTHER); Mon, 05 Mar 2012 21:47:40 -0800 (PST) Message-ID: <4F55A4FA.10404@gmail.com> Date: Tue, 06 Mar 2012 02:47:38 -0300 From: "Facundo M. de la Cruz" Organization: Code4Life User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.1) Gecko/20120216 Thunderbird/10.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Kernel Panic: Page fault in nat_new function (ipnat). X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: fmdlc@code4life.com.ar List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Mar 2012 05:47:42 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi guys, At my work we has a network firewall with FreeBSD 8.2-RELEASE 64 bits on a generic box running like a network gateway (making NAT between inbound and outbound connections). This firewall has four PCI ethernet network cards: em0, rl0, rl1, and rl2. The interface *em0* is configured for the WAN connection, and here is where we're using *ipnat*. We has some map and bimap rules at /etc/ipnat.rules file. Repeatedly, the box crashs and the OS is reboteed writting a coredump in /var/crash directory. Checking the coredump file, I can see that the crash is produced by the *fr_checknatin()* function (at /usr/src/sys/modules/ipfilter/../../contrib/ipfilter/netinet/ip_nat.c), in the em0 interface, that then call to a trap (number 12) which becomes fatal and then that produces a kernel panic. Looking in the source code of ip_nat.c i can see the following (I was researching about that :-P) The fr_checknatin() function has two arguments: * fin(I) -> Pointer to the packet information. * passp(I) -> Pointer to the filtering result flag. And the nat_new() function has five arguments: * fin(I) -> Pointer to the packet information. * np(I) -> Pointer to the map rule. * natsave(I) -> Pointer to where to store the NAT struct pointer- * flags(I) -> Flags describing the current packet. * direction(I) -> Direction of the packet (IN/OUT). I understand that this issue is generated by the nat_new function, that create a new data structure for the NAT mappings without checks, and the error is produced when the entry is duplicated. The backtrace generated by the kernel crash is the following: Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 04 fault virtual address = 0x4 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80e25e5c stack pointer = 0x28:0xffffff800010d630 frame pointer = 0x28:0xffffff800010d730 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (em0 taskq) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: #0 0xffffffff805f4e0e at kdb_backtrace+0x5e #1 0xffffffff805c2d07 at panic+0x187 #2 0xffffffff808ac600 at trap_fatal+0x290 #3 0xffffffff808ac9df at trap_pfault+0x28f #4 0xffffffff808acebf at trap+0x3df #5 0xffffffff80894fb4 at calltrap+0x8 #6 0xffffffff80e295d2 at fr_checknatin+0x382 #7 0xffffffff80e432b3 at fr_check+0x793 #8 0xffffffff8067e35c at pfil_run_hooks+0x9c #9 0xffffffff806d2159 at ip_input+0x279 #10 0xffffffff8067d76e at netisr_dispatch_src+0x7e #11 0xffffffff80673b3d at ether_demux+0x14d #12 0xffffffff80673f2d at ether_input+0x17d #13 0xffffffff8032e50a at em_rxeof+0x1ca #14 0xffffffff8032e928 at em_handle_que+0x48 #15 0xffffffff805ffeb5 at taskqueue_run_locked+0x85 #16 0xffffffff8060004e at taskqueue_thread_loop+0x4e #17 0xffffffff805994f8 at fork_exit+0x118 My idea is make the minimal changes in the ipnat syntax and not to migrate to PF, i only need to stop theses reboots, do you know any way to fix it?. Thanks! Regards. - -- Facundo M. de la Cruz (tty0) IT Consultant & Security Researcher RHCSA/RHCE (Red Hat Certified Engineer) GPG fingerprint: DF2F 514A 5167 00F5 C753 BF3B D797 C8E1 5726 0789 "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning." - Rich Cook -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBAgAGBQJPVaT6AAoJENeXyOFXJgeJjDMP/3dRke43zDT5PFEodLtLIwjN yvhVLUs3tg0FAIo9sxWH+7k/z3CxaW84jTWJ7dcqYowoS34iD58+T8qVXBl0NJ/y FcGiXDYFJDxm6F3qSkC9VLa9bM+SWI1u7Fhp4Oj39tdC30W7WpDwoctwR/Q/ojJm vxq7GP8CRVqIqgbHnWOgS+0QDQAyQcb6mm8GDa1Nxs49owH+EAYelhLn9S5dbBJT 1pOoftxs2Q6LixNHHZkBDRQJxoGHtPQI+lkZtFm90ODgyiGY0jmV5NmZHaXfoMUj 5s/lg+ePKemnEPL0rhxyquyqG3BBrgsCmeAGBhP1C6Z2nK4W8N+ZAVD4pX54ePbN dEZJiLL4EvMfY06YdXepVHIPEzmbeyxo6P8KFYFD/KBwIcxUoCxTHuBzcVZ4OIZH +Jk2HhL8rNn9LN1QeQiuw13/N6YUAP8sRfs7p/EZPRH4nX8Ng+F5CbLRUFkcOjwP nJY7na8tZ8U2HkAKohs0gmryOHaVS6pNxtmSKRWNC0wZtBG3pWH6ANUy+FG7c2FO E+s9C2vKT7FGVO1JE1aCuVJMs0CMXhktcxL8cPVvh6xWAdauoSa3+YO3YqevOi6T aDpDByvfur8uVVok7jS7Vh9VpFG12ZOitv439X4tShDxxNsQLQz6jvM40MJz995U 9+eaBCK4iUuRsMfJbNOl =BBkH -----END PGP SIGNATURE-----