From owner-freebsd-pf@FreeBSD.ORG Mon May 21 07:29:47 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8BA0A1065673; Mon, 21 May 2012 07:29:47 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id BF6078FC14; Mon, 21 May 2012 07:29:46 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4L7Oa6q084013; Mon, 21 May 2012 09:26:08 +0200 (CEST) (envelope-from jpulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4L7Q6D9084047 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 21 May 2012 09:26:06 +0200 (CEST) (envelope-from jpulz@frm2.tum.de) Received: from hades.admin.frm2 (localhost [127.0.0.1]) by hades.admin.frm2 (8.14.4/8.14.3) with ESMTP id q4L7Q6ku064259; Mon, 21 May 2012 09:26:06 +0200 (CEST) (envelope-from jpulz@frm2.tum.de) Received: (from jpulz@localhost) by hades.admin.frm2 (8.14.4/8.14.3/Submit) id q4L7Q6m9064258; Mon, 21 May 2012 09:26:06 +0200 (CEST) (envelope-from jpulz) Date: Mon, 21 May 2012 09:26:06 +0200 (CEST) Message-Id: <201205210726.q4L7Q6m9064258@hades.admin.frm2> To: FreeBSD-gnats-submit@freebsd.org From: Joerg Pulz X-send-pr-version: 3.113 X-GNATS-Notify: X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Mon, 21 May 2012 09:26:06 +0200 (CEST) Cc: freebsd-pf@freebsd.org Subject: panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 07:29:47 -0000 >Submitter-Id: current-users >Originator: Joerg Pulz >Organization: TU Muenchen / FRM II >Confidential: no >Synopsis: panic when using pf and route-to (maybe: bad fragment handling?) >Severity: critical >Priority: high >Category: kern >Class: sw-bug >Release: FreeBSD 9.0-RELEASE-p1 >Environment: System: FreeBSD charon 9.0-RELEASE-p1 FreeBSD 9.0-RELEASE-p1 #3: Sun May 20 08:42:19 CEST 2012 root@charon:/usr/obj/usr/src/sys/IPSEC amd64 >Description: We have a dual home VPN (IPSec) gateway running ipsec-tools. All packets coming from VPN clients have to hit the main router to generate and evaluate states. Therefor we use pf(4) and the route-to feature. Unfortunately the system panics (and it paniced with FreeBSD-8.x too) at irregular intervals with: "panic: m_copym, offset > size of mbuf chain" I'm unable to reproduce the network traffic to force the problem to appear. I recompiled the Kernel and configured the system to keep dumps to find the find the relevant place in the code for this issue. As i'm not that deep in network processing and pf code i can only guess that handling fragmented packets with pf(4) and route-to is buggy somewhere. Below is the kgdb(1) output for the last dump with some more information. If someone can give me some advise i can produce more and detailed output. If necessary i can upload all requiered stuff somewhere. >How-To-Repeat: >Fix: --- kgdb.out begins here --- GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: m_copym, offset > size of mbuf chain cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 m_copym() at m_copym+0x280 ip_fragment() at ip_fragment+0x1e5 pf_route() at pf_route+0x75c pf_test() at pf_test+0xc29 pf_route() at pf_route+0x30a pf_test() at pf_test+0xc29 pf_check_out() at pf_check_out+0x3a pfil_run_hooks() at pfil_run_hooks+0xd2 ip_output() at ip_output+0x655 ip_forward() at ip_forward+0x175 ip_input() at ip_input+0x5fd swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 649 out of 4077 MB:..3%..13%..23%..33%..42%..52%..62%..72%..82%..92% Reading symbols from /boot/kernel.old/geom_mirror.ko...Reading symbols from /boot/kernel.old/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel.old/geom_mirror.ko Reading symbols from /boot/kernel.old/ipmi.ko...Reading symbols from /boot/kernel.old/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel.old/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff806e9079 in m_copym (m=0x0, off0=1500, len=1480, wait=1) at /usr/src/sys/kern/uipc_mbuf.c:541 541 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); (kgdb) list 536 KASSERT(len >= 0, ("m_copym, negative len %d", len)); 537 MBUF_CHECKSLEEP(wait); 538 if (off == 0 && m->m_flags & M_PKTHDR) 539 copyhdr = 1; 540 while (off > 0) { 541 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); 542 if (off < m->m_len) 543 break; 544 off -= m->m_len; 545 m = m->m_next; (kgdb) p off $1 = 1166 (kgdb) up #11 0xffffffff8077fe1f in ip_fragment (ip=0xfffffe0005b7a974, m_frag=0xffffff8000241428, mtu=) at /usr/src/sys/netinet/ip_output.c:816 816 m->m_next = m_copym(m0, off, len, M_DONTWAIT); (kgdb) list 811 len = ip->ip_len - off; 812 m->m_flags |= M_LASTFRAG; 813 } else 814 mhip->ip_off |= IP_MF; 815 mhip->ip_len = htons((u_short)(len + mhlen)); 816 m->m_next = m_copym(m0, off, len, M_DONTWAIT); 817 if (m->m_next == NULL) { /* copy failed */ 818 m_free(m); 819 error = ENOBUFS; /* ??? */ 820 IPSTAT_INC(ips_odropped); (kgdb) p *m0 $2 = {m_hdr = {mh_next = 0xfffffe0052a2e200, mh_nextpkt = 0x0, mh_data = 0xfffffe0005b7a974 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 334, flowid = 0, csum_flags = 1, csum_data = 27136, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00050a90c0}}, MH_dat = {MH_ext = { ext_buf = 0x493a017c0045
, ext_free = 0x493a014e0045, ext_arg1 = 0x4947cb4f287c0437, ext_arg2 = 0x4e01004557b3bb81, ext_size = 3586, ref_cnt = 0xc6be758202079b0a, ext_type = 89195267}, MH_databuf = "E\000|\001:I\000\000E\000N\001:I\000\0007\004|(OËGI\201»³WE\000\001N\002\016\000\000?\001$É\n\233\a\002\202u¾Æ\003\003Q\005\000\000\000\000E\000\0012ûS\000\0000\021;\217\202u¾Æ\n\233\a\002\aÑùÃ\001\036éQKS\000\000\001B\000\000\000\001\v\001ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000N\001\000\000\000\000\000\000\001\000\000\000\000j\000\000\000\000\000\000ÞÀ­ÞÀ\220\n\005\000þÿÿE\000|\001:I\000\000E\000N\001:I\000\0007\004|(OËGI\201»³WE\000\001N\002\016\000\000?\001$É\n\233\a\002\202u¾Æ\003\003Q\005\000\000\000\000E\000\0012ûS\000\0000\021;\217\202u¾Æ\n\233\a\002\aÑùÃ\001\036éQKS\000\000\001B\000\000\000\001\v\001ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p off $3 = 1500 (kgdb) p len $4 = 1480 (kgdb) p hlen $5 = 20 (kgdb) up #12 0xffffffff8032842a in pf_route (m=0xffffff8000241658, r=0xfffffe0005dc8af8, dir=) at /usr/src/sys/contrib/pf/net/pf.c:6138 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); (kgdb) list 6133 /* 6134 * XXX: is cheaper + less error prone than own function 6135 */ 6136 NTOHS(ip->ip_len); 6137 NTOHS(ip->ip_off); 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); 6139 #else 6140 error = ip_fragment(m0, ifp, ifp->if_mtu); 6141 #endif 6142 if (error) { (kgdb) p *ip $6 = {ip_hl = 5 '\005', ip_v = 4 '\004', ip_tos = 0 '\0', ip_len = 19969, ip_id = 3586, ip_off = 0, ip_ttl = 63 '?', ip_p = 1 '\001', ip_sum = 51492, ip_src = {s_addr = 34052874}, ip_dst = {s_addr = 3334370690}} (kgdb) p *m0 $7 = {m_hdr = {mh_next = 0xfffffe0052a2e200, mh_nextpkt = 0x0, mh_data = 0xfffffe0005b7a974 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 334, flowid = 0, csum_flags = 1, csum_data = 27136, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00050a90c0}}, MH_dat = {MH_ext = { ext_buf = 0x493a017c0045
, ext_free = 0x493a014e0045, ext_arg1 = 0x4947cb4f287c0437, ext_arg2 = 0x4e01004557b3bb81, ext_size = 3586, ref_cnt = 0xc6be758202079b0a, ext_type = 89195267}, MH_databuf = "E\000|\001:I\000\000E\000N\001:I\000\0007\004|(OËGI\201»³WE\000\001N\002\016\000\000?\001$É\n\233\a\002\202u¾Æ\003\003Q\005\000\000\000\000E\000\0012ûS\000\0000\021;\217\202u¾Æ\n\233\a\002\aÑùÃ\001\036éQKS\000\000\001B\000\000\000\001\v\001ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000N\001\000\000\000\000\000\000\001\000\000\000\000j\000\000\000\000\000\000ÞÀ­ÞÀ\220\n\005\000þÿÿE\000|\001:I\000\000E\000N\001:I\000\0007\004|(OËGI\201»³WE\000\001N\002\016\000\000?\001$É\n\233\a\002\202u¾Æ\003\003Q\005\000\000\000\000E\000\0012ûS\000\0000\021;\217\202u¾Æ\n\233\a\002\aÑùÃ\001\036éQKS\000\000\001B\000\000\000\001\v\001ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p *m1 $8 = {m_hdr = {mh_next = 0xfffffe0052a2e200, mh_nextpkt = 0x0, mh_data = 0xfffffe0005b7a974 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 334, flowid = 0, csum_flags = 1, csum_data = 27136, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00050a90c0}}, MH_dat = {MH_ext = { ext_buf = 0x493a017c0045
, ext_free = 0x493a014e0045, ext_arg1 = 0x4947cb4f287c0437, ext_arg2 = 0x4e01004557b3bb81, ext_size = 3586, ref_cnt = 0xc6be758202079b0a, ext_type = 89195267}, MH_databuf = "E\000|\001:I\000\000E\000N\001:I\000\0007\004|(OËGI\201»³WE\000\001N\002\016\000\000?\001$É\n\233\a\002\202u¾Æ\003\003Q\005\000\000\000\000E\000\0012ûS\000\0000\021;\217\202u¾Æ\n\233\a\002\aÑùÃ\001\036éQKS\000\000\001B\000\000\000\001\v\001ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000N\001\000\000\000\000\000\000\001\000\000\000\000j\000\000\000\000\000\000ÞÀ­ÞÀ\220\n\005\000þÿÿE\000|\001:I\000\000E\000N\001:I\000\0007\004|(OËGI\201»³WE\000\001N\002\016\000\000?\001$É\n\233\a\002\202u¾Æ\003\003Q\005\000\000\000\000E\000\0012ûS\000\0000\021;\217\202u¾Æ\n\233\a\002\aÑùÃ\001\036éQKS\000\000\001B\000\000\000\001\v\001ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) --- kgdb.out ends here --- From owner-freebsd-pf@FreeBSD.ORG Mon May 21 08:28:47 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 641F1106567D; Mon, 21 May 2012 08:28:47 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3AD528FC1F; Mon, 21 May 2012 08:28:46 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4L8SjCE089223; Mon, 21 May 2012 08:28:45 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4L8Sj9a089219; Mon, 21 May 2012 08:28:45 GMT (envelope-from linimon) Date: Mon, 21 May 2012 08:28:45 GMT Message-Id: <201205210828.q4L8Sj9a089219@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-pf@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 08:28:47 -0000 Old Synopsis: panic when using pf and route-to (maybe: bad fragment handling?) New Synopsis: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Responsible-Changed-From-To: freebsd-bugs->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Mon May 21 08:28:29 UTC 2012 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=168190 From owner-freebsd-pf@FreeBSD.ORG Mon May 21 09:37:30 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7623A106564A; Mon, 21 May 2012 09:37:30 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id D19948FC17; Mon, 21 May 2012 09:37:29 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4L9RoCY030555 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Mon, 21 May 2012 11:27:50 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4L9Roep024167; Mon, 21 May 2012 11:27:50 +0200 (MEST) Date: Mon, 21 May 2012 11:27:50 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120521092750.GA20007@insomnia.benzedrine.cx> References: <201205210726.q4L7Q6m9064258@hades.admin.frm2> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205210726.q4L7Q6m9064258@hades.admin.frm2> User-Agent: Mutt/1.5.12-2006-07-14 Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 09:37:30 -0000 It looks like the byte order of ip_len is wrong, htons(334) = 19969, triggering fragmentation (334 < if_mtu, but 19969 > if_mtu). The reason is most likely the recursive pf_route() call: > m_copym() at m_copym+0x280 > ip_fragment() at ip_fragment+0x1e5 > pf_route() at pf_route+0x75c > pf_test() at pf_test+0xc29 > pf_route() at pf_route+0x30a > pf_test() at pf_test+0xc29 > pf_check_out() at pf_check_out+0x3a > pfil_run_hooks() at pfil_run_hooks+0xd2 > ip_output() at ip_output+0x655 i.e. the packet is filtered when going out on some interface, matching a route-to rule. Now the packet is filtered again on the routed-to interface. So far ok, this is expected. But now it matches a route-to rule again, possibly the same one, because it doesn't restrict the interface. Usually, this is not intentional (double route-to), and can be fixed by checking the route-to rule(s) and making them more restrictive. Semantics of such route-to chains are not well defined. There's an mbuf tag to prevent endless loops, but obviously even short chains are not working properly. I'd try to avoid them. Daniel From owner-freebsd-pf@FreeBSD.ORG Mon May 21 11:07:20 2012 Return-Path: Delivered-To: freebsd-pf@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 417A7106566B for ; Mon, 21 May 2012 11:07:20 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 2B0118FC21 for ; Mon, 21 May 2012 11:07:20 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4LB7Kn9049190 for ; Mon, 21 May 2012 11:07:20 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4LB7JSY049188 for freebsd-pf@FreeBSD.org; Mon, 21 May 2012 11:07:19 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 21 May 2012 11:07:19 GMT Message-Id: <201205211107.q4LB7JSY049188@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-pf@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-pf@FreeBSD.org X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 11:07:20 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/168190 pf [pf] panic when using pf and route-to (maybe: bad frag s kern/167057 pf [pf] PF firewall version 4.5 in FreeBSD 9.0 & 8.2 nolo o kern/166336 pf [pf] kern.securelevel 3 +pf reload o kern/165315 pf [pf] States never cleared in PF with DEVICE_POLLING o kern/164402 pf [pf] pf crashes with a particular set of rules when fi o kern/164271 pf [pf] not working pf nat on FreeBSD 9.0 [regression] o kern/163208 pf [pf] PF state key linking mismatch o kern/160370 pf [pf] Incorrect pfctl check of pf.conf o kern/155736 pf [pf] [altq] borrow from parent queue does not work wit o kern/153307 pf [pf] Bug with PF firewall o kern/148290 pf [pf] "sticky-address" option of Packet Filter (PF) blo o kern/148260 pf [pf] [patch] pf rdr incompatible with dummynet o kern/147789 pf [pf] Firewall PF no longer drops connections by sendin o kern/143543 pf [pf] [panic] PF route-to causes kernel panic o bin/143504 pf [patch] outgoing states are not killed by authpf(8) o conf/142961 pf [pf] No way to adjust pidfile in pflogd o conf/142817 pf [patch] etc/rc.d/pf: silence pfctl o kern/141905 pf [pf] [panic] pf kernel panic on 7.2-RELEASE with empty o kern/140697 pf [pf] pf behaviour changes - must be documented o kern/137982 pf [pf] when pf can hit state limits, random IP failures o kern/136781 pf [pf] Packets appear to drop with pf scrub and if_bridg o kern/135948 pf [pf] [gre] pf not natting gre protocol o kern/135162 pf [pfsync] pfsync(4) not usable with GENERIC kernel o kern/134996 pf [pf] Anchor tables not included when pfctl(8) is run w o kern/133732 pf [pf] max-src-conn issue o kern/132769 pf [pf] [lor] 2 LOR's with pf task mtx / ifnet and rtent f kern/132176 pf [pf] pf stalls connection when using route-to [regress o conf/130381 pf [rc.d] [pf] [ip6] ipv6 not fully configured when pf st o kern/129861 pf [pf] [patch] Argument names reversed in pf_table.c:_co o kern/127920 pf [pf] ipv6 and synproxy don't play well together o conf/127814 pf [pf] The flush in pf_reload in /etc/rc.d/pf does not w o kern/127439 pf [pf] deadlock in pf o kern/127121 pf [pf] [patch] pf incorrect log priority o kern/127042 pf [pf] [patch] pf recursion panic if interface group is o kern/125467 pf [pf] pf keep state bug while handling sessions between s kern/124933 pf [pf] [ip6] pf does not support (drops) IPv6 fragmented o kern/124364 pf [pf] [panic] Kernel panic with pf + bridge o kern/122773 pf [pf] pf doesn't log uid or pid when configured to o kern/122014 pf [pf] [panic] FreeBSD 6.2 panic in pf o kern/120281 pf [pf] [request] lost returning packets to PF for a rdr o kern/120057 pf [pf] [patch] Allow proper settings of ALTQ_HFSC. The c o bin/118355 pf [pf] [patch] pfctl(8) help message options order false o kern/114567 pf [pf] [lor] pf_ioctl.c + if.c s conf/110838 pf [pf] tagged parameter on nat not working on FreeBSD 5. o kern/103283 pf pfsync fails to sucessfully transfer some sessions o kern/103281 pf pfsync reports bulk update failures o kern/93825 pf [pf] pf reply-to doesn't work o sparc/93530 pf [pf] Incorrect checksums when using pf's route-to on s o kern/92949 pf [pf] PF + ALTQ problems with latency o bin/86635 pf [patch] pfctl(8): allow new page character (^L) in pf. o kern/82271 pf [pf] cbq scheduler cause bad latency 51 problems total. From owner-freebsd-pf@FreeBSD.ORG Mon May 21 14:15:53 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F14E1106566C; Mon, 21 May 2012 14:15:53 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 61A288FC17; Mon, 21 May 2012 14:15:53 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4LEEbL9000355; Mon, 21 May 2012 16:14:37 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4LEEXPD000351 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 21 May 2012 16:14:33 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Mon, 21 May 2012 16:14:30 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120521092750.GA20007@insomnia.benzedrine.cx> Message-ID: References: <201205210726.q4L7Q6m9064258@hades.admin.frm2> <20120521092750.GA20007@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Mon, 21 May 2012 16:14:33 +0200 (CEST) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 14:15:54 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 21 May 2012, Daniel Hartmeier wrote: > It looks like the byte order of ip_len is wrong, htons(334) = 19969, > triggering fragmentation (334 < if_mtu, but 19969 > if_mtu). > > The reason is most likely the recursive pf_route() call: > >> m_copym() at m_copym+0x280 >> ip_fragment() at ip_fragment+0x1e5 >> pf_route() at pf_route+0x75c >> pf_test() at pf_test+0xc29 >> pf_route() at pf_route+0x30a >> pf_test() at pf_test+0xc29 >> pf_check_out() at pf_check_out+0x3a >> pfil_run_hooks() at pfil_run_hooks+0xd2 >> ip_output() at ip_output+0x655 > > i.e. the packet is filtered when going out on some interface, matching a > route-to rule. > > Now the packet is filtered again on the routed-to interface. So far ok, > this is expected. > > But now it matches a route-to rule again, possibly the same one, because > it doesn't restrict the interface. > > Usually, this is not intentional (double route-to), and can be fixed by > checking the route-to rule(s) and making them more restrictive. > > Semantics of such route-to chains are not well defined. There's an mbuf > tag to prevent endless loops, but obviously even short chains are not > working properly. I'd try to avoid them. Daniel, thanks for looking at this and your explanation. The route-to rules are pretty specific: #### pf.conf ext_if="bge0" int_if="bge1" vpn_net="10.1.1.0/24" srv_net="172.16.1.0/24" gw_addr="172.16.1.254" scrub in all pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state #### pf.conf The default gateway for the host itself is reachable on the external interface and there is a static route for our internal networks (172.16.0.0/16) configured for the system itself. All client traffic has to hit the $gw_addr first regardless in which direction it goes afterwards, that's where the route-to rules kick in. Do they have to be even more specific if possible at all? Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPuk3JSPOsGF+KA+MRAh9BAJ9TRkTeB12NtqYOOmdRcDaTpBjPOgCdE3S1 6rcDkcoro92HI/db4pMLDn4= =w64E -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Mon May 21 14:20:08 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 513901065672 for ; Mon, 21 May 2012 14:20:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DA7C98FC12 for ; Mon, 21 May 2012 14:20:07 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4LEK4UE039522 for ; Mon, 21 May 2012 14:20:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4LEK4ds039516; Mon, 21 May 2012 14:20:04 GMT (envelope-from gnats) Date: Mon, 21 May 2012 14:20:04 GMT Message-Id: <201205211420.q4LEK4ds039516@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 14:20:08 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Mon, 21 May 2012 16:14:30 +0200 (CEST) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 21 May 2012, Daniel Hartmeier wrote: > It looks like the byte order of ip_len is wrong, htons(334) = 19969, > triggering fragmentation (334 < if_mtu, but 19969 > if_mtu). > > The reason is most likely the recursive pf_route() call: > >> m_copym() at m_copym+0x280 >> ip_fragment() at ip_fragment+0x1e5 >> pf_route() at pf_route+0x75c >> pf_test() at pf_test+0xc29 >> pf_route() at pf_route+0x30a >> pf_test() at pf_test+0xc29 >> pf_check_out() at pf_check_out+0x3a >> pfil_run_hooks() at pfil_run_hooks+0xd2 >> ip_output() at ip_output+0x655 > > i.e. the packet is filtered when going out on some interface, matching a > route-to rule. > > Now the packet is filtered again on the routed-to interface. So far ok, > this is expected. > > But now it matches a route-to rule again, possibly the same one, because > it doesn't restrict the interface. > > Usually, this is not intentional (double route-to), and can be fixed by > checking the route-to rule(s) and making them more restrictive. > > Semantics of such route-to chains are not well defined. There's an mbuf > tag to prevent endless loops, but obviously even short chains are not > working properly. I'd try to avoid them. Daniel, thanks for looking at this and your explanation. The route-to rules are pretty specific: #### pf.conf ext_if="bge0" int_if="bge1" vpn_net="10.1.1.0/24" srv_net="172.16.1.0/24" gw_addr="172.16.1.254" scrub in all pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state #### pf.conf The default gateway for the host itself is reachable on the external interface and there is a static route for our internal networks (172.16.0.0/16) configured for the system itself. All client traffic has to hit the $gw_addr first regardless in which direction it goes afterwards, that's where the route-to rules kick in. Do they have to be even more specific if possible at all? Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPuk3JSPOsGF+KA+MRAh9BAJ9TRkTeB12NtqYOOmdRcDaTpBjPOgCdE3S1 6rcDkcoro92HI/db4pMLDn4= =w64E -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Mon May 21 16:50:40 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 12E4D1065673 for ; Mon, 21 May 2012 16:50:40 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id 325D08FC14 for ; Mon, 21 May 2012 16:50:38 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4LGob9Y023260 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Mon, 21 May 2012 18:50:37 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4LGobeJ025532; Mon, 21 May 2012 18:50:37 +0200 (MEST) Date: Mon, 21 May 2012 18:50:37 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120521165037.GA29536@insomnia.benzedrine.cx> References: <201205211420.q4LEK4ds039516@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205211420.q4LEK4ds039516@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 16:50:40 -0000 On Mon, May 21, 2012 at 02:20:04PM +0000, Joerg Pulz wrote: > ext_if="bge0" > int_if="bge1" > vpn_net="10.1.1.0/24" > srv_net="172.16.1.0/24" > gw_addr="172.16.1.254" > > scrub in all > > pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state > pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state So something from $vpn_net comes in, gets routed to the default gateway (on $ext_if side), attempts to pass out on $ext_if, matches the first rule, route-to applies, packet gets re-routed to $gw_addr, passes out on $int_if, matches the second rule, double route-to. All you need to do is prevent the second rule from applying for packets where the first rule matched, like with tags: pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state tag from_vpn pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state pass out on $int_if from $vpn_net to $srv_net keep state tagged from_vpn i.e. you add 'tag from_vpn' to the first rule, so packets matching it get tagged, then you add a third rule without route-to that applies to tagged packets, which wins last-match for such packets. Or, instead of adding a third rule, add '! tagged from_vpn' to the second rule, if tagged packets can still pass out on $int_if by another rule. Kind regards, Daniel From owner-freebsd-pf@FreeBSD.ORG Mon May 21 16:57:31 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 610C9106566B for ; Mon, 21 May 2012 16:57:31 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id B30138FC1C for ; Mon, 21 May 2012 16:57:29 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4LGvSZ5015228 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Mon, 21 May 2012 18:57:28 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4LGvSF4016289; Mon, 21 May 2012 18:57:28 +0200 (MEST) Date: Mon, 21 May 2012 18:57:28 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120521165728.GB29536@insomnia.benzedrine.cx> References: <201205211420.q4LEK4ds039516@freefall.freebsd.org> <20120521165037.GA29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120521165037.GA29536@insomnia.benzedrine.cx> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 16:57:31 -0000 BTW, if the theory is correct, you should be able to reproduce the problem by sending a packet in from $vpn_net to a host in $srv_net of size 334 (=306+28), e.g. with $ ping -s 306 172.16.1.1 Daniel From owner-freebsd-pf@FreeBSD.ORG Mon May 21 18:31:35 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C0BE1106564A; Mon, 21 May 2012 18:31:35 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9397F8FC0A; Mon, 21 May 2012 18:31:35 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4LIVZxN080483; Mon, 21 May 2012 18:31:35 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4LIVZpF080474; Mon, 21 May 2012 18:31:35 GMT (envelope-from linimon) Date: Mon, 21 May 2012 18:31:35 GMT Message-Id: <201205211831.q4LIVZpF080474@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-pf@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/168200: [pf] pf crashes when receiving packets from an address in a table X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 18:31:35 -0000 Old Synopsis: pf crashes when receiving packets from an address in a table New Synopsis: [pf] pf crashes when receiving packets from an address in a table Responsible-Changed-From-To: freebsd-bugs->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Mon May 21 18:31:25 UTC 2012 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=168200 From owner-freebsd-pf@FreeBSD.ORG Mon May 21 18:49:36 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E53F1065670; Mon, 21 May 2012 18:49:36 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 95EFC8FC14; Mon, 21 May 2012 18:49:35 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4LImJxP008552; Mon, 21 May 2012 20:48:19 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4LImF4a008547 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 21 May 2012 20:48:18 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Mon, 21 May 2012 20:48:12 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120521165037.GA29536@insomnia.benzedrine.cx> Message-ID: References: <201205211420.q4LEK4ds039516@freefall.freebsd.org> <20120521165037.GA29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3469798045-1197118405-1337625847=:74709" Content-ID: X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Mon, 21 May 2012 20:48:18 +0200 (CEST) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 18:49:36 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-1197118405-1337625847=:74709 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: On Mon, 21 May 2012, Daniel Hartmeier wrote: > On Mon, May 21, 2012 at 02:20:04PM +0000, Joerg Pulz wrote: > >> ext_if="bge0" >> int_if="bge1" >> vpn_net="10.1.1.0/24" >> srv_net="172.16.1.0/24" >> gw_addr="172.16.1.254" >> >> scrub in all >> >> pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state >> pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > > So something from $vpn_net comes in, gets routed to the default gateway > (on $ext_if side), attempts to pass out on $ext_if, matches the first > rule, route-to applies, packet gets re-routed to $gw_addr, passes out > on $int_if, matches the second rule, double route-to. > > All you need to do is prevent the second rule from applying for packets > where the first rule matched, like with tags: > > pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state tag from_vpn > pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > pass out on $int_if from $vpn_net to $srv_net keep state tagged from_vpn > > i.e. you add 'tag from_vpn' to the first rule, so packets matching it > get tagged, then you add a third rule without route-to that applies to > tagged packets, which wins last-match for such packets. > > Or, instead of adding a third rule, add '! tagged from_vpn' to the > second rule, if tagged packets can still pass out on $int_if by another > rule. Daniel, thanks again. There is one main question left for me. How could a packet which is directed to a host in $srv_net (and $srv_net is the same broadcast domain as $int_if) ever leave through $ext_if and make the first route-to match and therefor lead to a double route-to? Regarding your other mail with the reproduce instructions > BTW, if the theory is correct, you should be able to reproduce the > problem by sending a packet in from $vpn_net to a host in $srv_net > of size 334 (=306+28), e.g. with > > $ ping -s 306 172.16.1.1 I tried this, but it did no harm to the system at all. Packets where flowing as they should and answers came back. PF ruleset is still unchanged, no tagging. Meanwhile the system panic'ed again, and this time there is no double route-to involved at all. kgdb(1) output below. If your assumption in your first response is right: > It looks like the byte order of ip_len is wrong, htons(334) = 19969, > triggering fragmentation (334 < if_mtu, but 19969 > if_mtu). Then it should read htons(175) = 44800 (175 < if_mtu, but 44800 > if_mtu") But where is it coming from if double route-to can't be the culprit in this case? Any thoughts on this? Kind regards Joerg ### kgdb.out2 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: m_copym, offset > size of mbuf chain cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 m_copym() at m_copym+0x280 ip_fragment() at ip_fragment+0x1e5 pf_route() at pf_route+0x75c pf_test() at pf_test+0xc29 pf_check_out() at pf_check_out+0x3a pfil_run_hooks() at pfil_run_hooks+0xd2 ip_output() at ip_output+0x655 ip_forward() at ip_forward+0x175 ip_input() at ip_input+0x5fd swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 628 out of 4077 MB:..3%..11%..21%..31%..41%..51%..62%..72%..82%..92% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff806e9079 in m_copym (m=0x0, off0=1500, len=1480, wait=1) at /usr/src/sys/kern/uipc_mbuf.c:541 541 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); (kgdb) list 536 KASSERT(len >= 0, ("m_copym, negative len %d", len)); 537 MBUF_CHECKSLEEP(wait); 538 if (off == 0 && m->m_flags & M_PKTHDR) 539 copyhdr = 1; 540 while (off > 0) { 541 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); 542 if (off < m->m_len) 543 break; 544 off -= m->m_len; 545 m = m->m_next; (kgdb) p off $1 = 1325 (kgdb) up #11 0xffffffff8077fe1f in ip_fragment (ip=0xfffffe0173b1fc74, m_frag=0xffffff8000241658, mtu=) at /usr/src/sys/netinet/ip_output.c:816 816 m->m_next = m_copym(m0, off, len, M_DONTWAIT); (kgdb) list 811 len = ip->ip_len - off; 812 m->m_flags |= M_LASTFRAG; 813 } else 814 mhip->ip_off |= IP_MF; 815 mhip->ip_len = htons((u_short)(len + mhlen)); 816 m->m_next = m_copym(m0, off, len, M_DONTWAIT); 817 if (m->m_next == NULL) { /* copy failed */ 818 m_free(m); 819 error = ENOBUFS; /* ??? */ 820 IPSTAT_INC(ips_odropped); (kgdb) p *m0 $2 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p off $3 = 1500 (kgdb) p len $4 = 1480 (kgdb) p hlen $5 = 20 (kgdb) up #12 0xffffffff8032842a in pf_route (m=0xffffff80002418e8, r=0xfffffe0005d87750, dir=) at /usr/src/sys/contrib/pf/net/pf.c:6138 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); (kgdb) list 6133 /* 6134 * XXX: is cheaper + less error prone than own function 6135 */ 6136 NTOHS(ip->ip_len); 6137 NTOHS(ip->ip_off); 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); 6139 #else 6140 error = ip_fragment(m0, ifp, ifp->if_mtu); 6141 #endif 6142 if (error) { (kgdb) p *ip $6 = {ip_hl = 5 '\005', ip_v = 4 '\004', ip_tos = 0 '\0', ip_len = 44800, ip_id = 11615, ip_off = 0, ip_ttl = 127 '\177', ip_p = 1 '\001', ip_sum = 18204, ip_src = {s_addr = 34052874}, ip_dst = {s_addr = 604051884}} (kgdb) p *m0 $7 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p *m1 $8 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) ### kgdb.out2 - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPuo3vSPOsGF+KA+MRAu+AAKDGT0lDqaIcYO4Q6Lx37oUX64GeCgCffGhf AtfXrgD94GTXHsX7roaKfAI= =wEqQ -----END PGP SIGNATURE----- --3469798045-1197118405-1337625847=:74709-- From owner-freebsd-pf@FreeBSD.ORG Mon May 21 18:50:10 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C350B106568C for ; Mon, 21 May 2012 18:50:10 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9F7808FC15 for ; Mon, 21 May 2012 18:50:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4LIo7sp092337 for ; Mon, 21 May 2012 18:50:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4LIo7D8092336; Mon, 21 May 2012 18:50:07 GMT (envelope-from gnats) Date: Mon, 21 May 2012 18:50:07 GMT Message-Id: <201205211850.q4LIo7D8092336@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 18:50:10 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Mon, 21 May 2012 20:48:12 +0200 (CEST) This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-1197118405-1337625847=:74709 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: On Mon, 21 May 2012, Daniel Hartmeier wrote: > On Mon, May 21, 2012 at 02:20:04PM +0000, Joerg Pulz wrote: > >> ext_if="bge0" >> int_if="bge1" >> vpn_net="10.1.1.0/24" >> srv_net="172.16.1.0/24" >> gw_addr="172.16.1.254" >> >> scrub in all >> >> pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state >> pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > > So something from $vpn_net comes in, gets routed to the default gateway > (on $ext_if side), attempts to pass out on $ext_if, matches the first > rule, route-to applies, packet gets re-routed to $gw_addr, passes out > on $int_if, matches the second rule, double route-to. > > All you need to do is prevent the second rule from applying for packets > where the first rule matched, like with tags: > > pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state tag from_vpn > pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > pass out on $int_if from $vpn_net to $srv_net keep state tagged from_vpn > > i.e. you add 'tag from_vpn' to the first rule, so packets matching it > get tagged, then you add a third rule without route-to that applies to > tagged packets, which wins last-match for such packets. > > Or, instead of adding a third rule, add '! tagged from_vpn' to the > second rule, if tagged packets can still pass out on $int_if by another > rule. Daniel, thanks again. There is one main question left for me. How could a packet which is directed to a host in $srv_net (and $srv_net is the same broadcast domain as $int_if) ever leave through $ext_if and make the first route-to match and therefor lead to a double route-to? Regarding your other mail with the reproduce instructions > BTW, if the theory is correct, you should be able to reproduce the > problem by sending a packet in from $vpn_net to a host in $srv_net > of size 334 (=306+28), e.g. with > > $ ping -s 306 172.16.1.1 I tried this, but it did no harm to the system at all. Packets where flowing as they should and answers came back. PF ruleset is still unchanged, no tagging. Meanwhile the system panic'ed again, and this time there is no double route-to involved at all. kgdb(1) output below. If your assumption in your first response is right: > It looks like the byte order of ip_len is wrong, htons(334) = 19969, > triggering fragmentation (334 < if_mtu, but 19969 > if_mtu). Then it should read htons(175) = 44800 (175 < if_mtu, but 44800 > if_mtu") But where is it coming from if double route-to can't be the culprit in this case? Any thoughts on this? Kind regards Joerg ### kgdb.out2 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: m_copym, offset > size of mbuf chain cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 m_copym() at m_copym+0x280 ip_fragment() at ip_fragment+0x1e5 pf_route() at pf_route+0x75c pf_test() at pf_test+0xc29 pf_check_out() at pf_check_out+0x3a pfil_run_hooks() at pfil_run_hooks+0xd2 ip_output() at ip_output+0x655 ip_forward() at ip_forward+0x175 ip_input() at ip_input+0x5fd swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 628 out of 4077 MB:..3%..11%..21%..31%..41%..51%..62%..72%..82%..92% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff806e9079 in m_copym (m=0x0, off0=1500, len=1480, wait=1) at /usr/src/sys/kern/uipc_mbuf.c:541 541 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); (kgdb) list 536 KASSERT(len >= 0, ("m_copym, negative len %d", len)); 537 MBUF_CHECKSLEEP(wait); 538 if (off == 0 && m->m_flags & M_PKTHDR) 539 copyhdr = 1; 540 while (off > 0) { 541 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); 542 if (off < m->m_len) 543 break; 544 off -= m->m_len; 545 m = m->m_next; (kgdb) p off $1 = 1325 (kgdb) up #11 0xffffffff8077fe1f in ip_fragment (ip=0xfffffe0173b1fc74, m_frag=0xffffff8000241658, mtu=) at /usr/src/sys/netinet/ip_output.c:816 816 m->m_next = m_copym(m0, off, len, M_DONTWAIT); (kgdb) list 811 len = ip->ip_len - off; 812 m->m_flags |= M_LASTFRAG; 813 } else 814 mhip->ip_off |= IP_MF; 815 mhip->ip_len = htons((u_short)(len + mhlen)); 816 m->m_next = m_copym(m0, off, len, M_DONTWAIT); 817 if (m->m_next == NULL) { /* copy failed */ 818 m_free(m); 819 error = ENOBUFS; /* ??? */ 820 IPSTAT_INC(ips_odropped); (kgdb) p *m0 $2 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p off $3 = 1500 (kgdb) p len $4 = 1480 (kgdb) p hlen $5 = 20 (kgdb) up #12 0xffffffff8032842a in pf_route (m=0xffffff80002418e8, r=0xfffffe0005d87750, dir=) at /usr/src/sys/contrib/pf/net/pf.c:6138 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); (kgdb) list 6133 /* 6134 * XXX: is cheaper + less error prone than own function 6135 */ 6136 NTOHS(ip->ip_len); 6137 NTOHS(ip->ip_off); 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); 6139 #else 6140 error = ip_fragment(m0, ifp, ifp->if_mtu); 6141 #endif 6142 if (error) { (kgdb) p *ip $6 = {ip_hl = 5 '\005', ip_v = 4 '\004', ip_tos = 0 '\0', ip_len = 44800, ip_id = 11615, ip_off = 0, ip_ttl = 127 '\177', ip_p = 1 '\001', ip_sum = 18204, ip_src = {s_addr = 34052874}, ip_dst = {s_addr = 604051884}} (kgdb) p *m0 $7 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p *m1 $8 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) ### kgdb.out2 - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPuo3vSPOsGF+KA+MRAu+AAKDGT0lDqaIcYO4Q6Lx37oUX64GeCgCffGhf AtfXrgD94GTXHsX7roaKfAI= =wEqQ -----END PGP SIGNATURE----- --3469798045-1197118405-1337625847=:74709-- From owner-freebsd-pf@FreeBSD.ORG Tue May 22 02:04:23 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 883E01065672; Tue, 22 May 2012 02:04:23 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 59CB48FC0A; Tue, 22 May 2012 02:04:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4M24NJk009958; Tue, 22 May 2012 02:04:23 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4M24MrC009953; Tue, 22 May 2012 02:04:22 GMT (envelope-from linimon) Date: Tue, 22 May 2012 02:04:22 GMT Message-Id: <201205220204.q4M24MrC009953@freefall.freebsd.org> To: daniel@benzedrine.cx, linimon@FreeBSD.org, gnats-admin@FreeBSD.org, freebsd-pf@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/168193: Re: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 02:04:23 -0000 Old Synopsis: Re: panic when using pf and route-to (maybe: bad fragment handling?) New Synopsis: Re: [pf] panic when using pf and route-to (maybe: bad fragment handling?) State-Changed-From-To: open->closed State-Changed-By: linimon State-Changed-When: Tue May 22 02:03:44 UTC 2012 State-Changed-Why: Misfiled followup to kern/168190; content migrated. Responsible-Changed-From-To: gnats-admin->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Tue May 22 02:03:44 UTC 2012 Responsible-Changed-Why: http://www.freebsd.org/cgi/query-pr.cgi?pr=168193 From owner-freebsd-pf@FreeBSD.ORG Tue May 22 06:07:00 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CEBE8106564A; Tue, 22 May 2012 06:07:00 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 75F728FC18; Tue, 22 May 2012 06:07:00 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4M65iG6021612; Tue, 22 May 2012 08:05:44 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4M65gJT021608 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 22 May 2012 08:05:42 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Tue, 22 May 2012 08:05:39 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120521165037.GA29536@insomnia.benzedrine.cx> Message-ID: References: <201205211420.q4LEK4ds039516@freefall.freebsd.org> <20120521165037.GA29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Tue, 22 May 2012 08:05:42 +0200 (CEST) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 06:07:01 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 21 May 2012, Daniel Hartmeier wrote: > On Mon, May 21, 2012 at 02:20:04PM +0000, Joerg Pulz wrote: > >> ext_if="bge0" >> int_if="bge1" >> vpn_net="10.1.1.0/24" >> srv_net="172.16.1.0/24" >> gw_addr="172.16.1.254" >> >> scrub in all >> >> pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state >> pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > > So something from $vpn_net comes in, gets routed to the default gateway > (on $ext_if side), attempts to pass out on $ext_if, matches the first > rule, route-to applies, packet gets re-routed to $gw_addr, passes out > on $int_if, matches the second rule, double route-to. > > All you need to do is prevent the second rule from applying for packets > where the first rule matched, like with tags: > > pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state tag from_vpn > pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > pass out on $int_if from $vpn_net to $srv_net keep state tagged from_vpn > > i.e. you add 'tag from_vpn' to the first rule, so packets matching it > get tagged, then you add a third rule without route-to that applies to > tagged packets, which wins last-match for such packets. > > Or, instead of adding a third rule, add '! tagged from_vpn' to the > second rule, if tagged packets can still pass out on $int_if by another > rule. And i got another panic, this time without pf(4) involved at all. Unfortunately "dump" in kdb is doing nothing but hang. :-( Here is what was displayed on the screen: panic: m_copym, offset > size of mbuf chain cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic_0x182 m_copym() at m_copym+0x280 ip_fragment() at ip_fragment+0x1e5 ip_output() at ip_output+0xeab ip_forward() at ip_forward+0x175 ip_input() at ip_input+0x5fd swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xfffff8000241d00, rbp = 0 --- KDB: enter: panic [ thread pid 12 tid 100008 ] Any thoughts about this one? Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPuyy2SPOsGF+KA+MRAtcgAJwO4zh4/AZN2tHhySI+24y+kozM3gCgn+HS /ZIUDnpvQCGdRWXYvvBauZw= =xctG -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Tue May 22 06:10:04 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E58ED106566B for ; Tue, 22 May 2012 06:10:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C77FD8FC16 for ; Tue, 22 May 2012 06:10:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4M6A39K036773 for ; Tue, 22 May 2012 06:10:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4M6A3oY036772; Tue, 22 May 2012 06:10:03 GMT (envelope-from gnats) Date: Tue, 22 May 2012 06:10:03 GMT Message-Id: <201205220610.q4M6A3oY036772@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 06:10:04 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Tue, 22 May 2012 08:05:39 +0200 (CEST) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 21 May 2012, Daniel Hartmeier wrote: > On Mon, May 21, 2012 at 02:20:04PM +0000, Joerg Pulz wrote: > >> ext_if="bge0" >> int_if="bge1" >> vpn_net="10.1.1.0/24" >> srv_net="172.16.1.0/24" >> gw_addr="172.16.1.254" >> >> scrub in all >> >> pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state >> pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > > So something from $vpn_net comes in, gets routed to the default gateway > (on $ext_if side), attempts to pass out on $ext_if, matches the first > rule, route-to applies, packet gets re-routed to $gw_addr, passes out > on $int_if, matches the second rule, double route-to. > > All you need to do is prevent the second rule from applying for packets > where the first rule matched, like with tags: > > pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state tag from_vpn > pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > pass out on $int_if from $vpn_net to $srv_net keep state tagged from_vpn > > i.e. you add 'tag from_vpn' to the first rule, so packets matching it > get tagged, then you add a third rule without route-to that applies to > tagged packets, which wins last-match for such packets. > > Or, instead of adding a third rule, add '! tagged from_vpn' to the > second rule, if tagged packets can still pass out on $int_if by another > rule. And i got another panic, this time without pf(4) involved at all. Unfortunately "dump" in kdb is doing nothing but hang. :-( Here is what was displayed on the screen: panic: m_copym, offset > size of mbuf chain cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic_0x182 m_copym() at m_copym+0x280 ip_fragment() at ip_fragment+0x1e5 ip_output() at ip_output+0xeab ip_forward() at ip_forward+0x175 ip_input() at ip_input+0x5fd swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xfffff8000241d00, rbp = 0 --- KDB: enter: panic [ thread pid 12 tid 100008 ] Any thoughts about this one? Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPuyy2SPOsGF+KA+MRAtcgAJwO4zh4/AZN2tHhySI+24y+kozM3gCgn+HS /ZIUDnpvQCGdRWXYvvBauZw= =xctG -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Tue May 22 08:06:24 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D0F5A1065672 for ; Tue, 22 May 2012 08:06:24 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id 3E7E18FC1C for ; Tue, 22 May 2012 08:06:23 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4M86MAM029649 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Tue, 22 May 2012 10:06:22 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4M86MvG026252; Tue, 22 May 2012 10:06:22 +0200 (MEST) Date: Tue, 22 May 2012 10:06:22 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120522080622.GC29536@insomnia.benzedrine.cx> References: <201205220610.q4M6A3oY036772@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205220610.q4M6A3oY036772@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 08:06:24 -0000 On Tue, May 22, 2012 at 06:10:03AM +0000, Joerg Pulz wrote: > And i got another panic, this time without pf(4) involved at all. > Unfortunately "dump" in kdb is doing nothing but hang. :-( > > Here is what was displayed on the screen: > > panic: m_copym, offset > size of mbuf chain > cpuid = 1 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > kdb_backtrace() at kdb_backtrace+0x37 > panic() at panic_0x182 > m_copym() at m_copym+0x280 > ip_fragment() at ip_fragment+0x1e5 > ip_output() at ip_output+0xeab > ip_forward() at ip_forward+0x175 > ip_input() at ip_input+0x5fd > swi_net() at swi_net+0x15a > intr_event_execute_handlers() at intr_event_execute_handlers+0x66 > ithread_loop() at ithread_loop+0xaf > fork_exit() at fork_exit+0x12a > fork_trampoline() at fork_trampoline+0xe > - --- trap 0, rip = 0, rsp = 0xfffff8000241d00, rbp = 0 --- > KDB: enter: panic > [ thread pid 12 tid 100008 ] > > Any thoughts about this one? It's pretty clear that ip_fragment() gets called because a packet has ip_len in the wrong byte order. The packet is smaller than mtu, it shouldn't get fragmented at all. The problem is not related to fragmentation itself. While the stack trace above doesn't show pf, it's still possible (and I'd say likely) that it's pf leaving the byte order wrong. There are several places in pf where the byte order of ip_len is swapped, some are local patches (#ifdef __FreeBSD__). I'd also guess that it's related to a more obscure functionality (like route-to, or double route-to), otherwise more people would see this. Another lead is interface checksum (or fragmentation) offloading, there are code paths that swap byte order depending on these features. You could try disabling them with ifconfig. If you look at egrep -i '[hn]to[nh]s\(.*ip_len\)' /usr/src/sys/netinet/* /usr/src/contrib/pf/net/* you'll see that it's quite a mess, because byte order is expected to be different depending on context. I'm still looking ;) Daniel From owner-freebsd-pf@FreeBSD.ORG Tue May 22 10:20:55 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CF7E106566B; Tue, 22 May 2012 10:20:55 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id B48DE8FC0A; Tue, 22 May 2012 10:20:54 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4MAJXmg032132; Tue, 22 May 2012 12:19:33 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4MAJXXE032128 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 22 May 2012 12:19:33 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Tue, 22 May 2012 12:19:30 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120522090227.GD29536@insomnia.benzedrine.cx> Message-ID: References: <201205210726.q4L7Q6m9064258@hades.admin.frm2> <20120522090227.GD29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3469798045-1358106948-1337681906=:89783" Content-ID: X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Tue, 22 May 2012 12:19:33 +0200 (CEST) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 10:20:55 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-1358106948-1337681906=:89783 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-15; FORMAT=flowed Content-Transfer-Encoding: 8BIT Content-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: On Tue, 22 May 2012, Daniel Hartmeier wrote: > On Mon, May 21, 2012 at 09:26:06AM +0200, Joerg Pulz wrote: > >> #12 0xffffffff8032842a in pf_route (m=0xffffff8000241658, >> r=0xfffffe0005dc8af8, dir=) at /usr/src/sys/contrib/pf/net/pf.c:6138 >> 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); >> (kgdb) list >> 6133 /* >> 6134 * XXX: is cheaper + less error prone than own function >> 6135 */ >> 6136 NTOHS(ip->ip_len); >> 6137 NTOHS(ip->ip_off); >> 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); >> 6139 #else >> 6140 error = ip_fragment(m0, ifp, ifp->if_mtu); >> 6141 #endif >> 6142 if (error) { > > Can you print *ifp in this context, please? > > Just to make sure if_mtu is sane. Daniel, thanks for all your effort. Here comes *ifp Joerg #### kgdb.out #12 0xffffffff8032842a in pf_route (m=0xffffff80002418e8, r=0xfffffe0005e05750, dir=Variable "dir" is not available. ) at /usr/src/sys/contrib/pf/net/pf.c:6138 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); (kgdb) p *ifp $1 = {if_softc = 0xffffff80007b1000, if_l2com = 0xfffffe000300ba40, if_vnet = 0x0, if_link = {tqe_next = 0xfffffe0003001000, tqe_prev = 0xfffffe0003001818}, if_xname = "bge1", '\0' , if_dname = 0xfffffe00028f07d8 "bge", if_dunit = 1, if_refcount = 1, if_addrhead = {tqh_first = 0xfffffe0003009800, tqh_last = 0xfffffe0005abdcb8}, if_pcount = 0, if_carp = 0x0, if_bpf = 0xfffffe00050e7900, if_index = 6, if_index_reserved = 0, if_vlantrunk = 0x0, if_flags = 34819, if_capabilities = 524443, if_capenable = 524443, if_linkmib = 0x0, if_linkmiblen = 0, if_data = { ifi_type = 6 '\006', ifi_physical = 0 '\0', ifi_addrlen = 6 '\006', ifi_hdrlen = 18 '\022', ifi_link_state = 2 '\002', ifi_spare_char1 = 0 '\0', ifi_spare_char2 = 0 '\0', ifi_datalen = 152 '\230', ifi_mtu = 1500, ifi_metric = 0, ifi_baudrate = 1000000000, ifi_ipackets = 54812, ifi_ierrors = 0, ifi_opackets = 34745, ifi_oerrors = 0, ifi_collisions = 0, ifi_ibytes = 41868704, ifi_obytes = 5296902, ifi_imcasts = 10095, ifi_omcasts = 0, ifi_iqdrops = 0, ifi_noproto = 0, ifi_hwassist = 3, ifi_epoch = 1, ifi_lastchange = {tv_sec = 1337441486, tv_usec = 788343}}, if_multiaddrs = {tqh_first = 0xfffffe00059137c0, tqh_last = 0xfffffe0005914300}, if_amcount = 0, if_output = 0xffffffff8073d4b5 , if_input = 0xffffffff8073ca8b , if_start = 0xffffffff803c2da7 , if_ioctl = 0xffffffff803c8fda , if_init = 0xffffffff803c8f94 , if_resolvemulti = 0xffffffff8073c44d , if_qflush = 0xffffffff807352f2 , if_transmit = 0xffffffff807351be , if_reassign = 0, if_home_vnet = 0x0, if_addr = 0xfffffe0003009800, if_llsoftc = 0x0, if_drv_flags = 64, if_snd = {ifq_head = 0x0, ifq_tail = 0x0, ifq_len = 0, ifq_maxlen = 511, ifq_drops = 0, ifq_mtx = {lock_object = { lo_name = 0xfffffe0003002028 "bge1", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006cf480}, mtx_lock = 4}, ifq_drv_head = 0x0, ifq_drv_tail = 0x0, ifq_drv_len = 0, ifq_drv_maxlen = 511, altq_type = 0, altq_flags = 1, altq_disc = 0x0, altq_ifp = 0xfffffe0003002000, altq_enqueue = 0, altq_dequeue = 0, altq_request = 0, altq_clfier = 0x0, altq_classify = 0, altq_tbr = 0x0, altq_cdnr = 0x0}, if_broadcastaddr = 0xffffffff80ada7c0 "ÿÿÿÿÿÿ", if_bridge = 0x0, if_label = 0x0, if_prefixhead = {tqh_first = 0x0, tqh_last = 0xfffffe0003002278}, if_afdata = {0x0, 0x0, 0xfffffe000581fa00, 0x0 , 0xfffffe0005814800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, if_afdata_initialized = 2, if_afdata_lock = { lock_object = {lo_name = 0xffffffff80ad9a5a "if_afdata", lo_flags = 69402624, lo_data = 0, lo_witness = 0xffffff80006cf400}, rw_lock = 1}, if_linktask = {ta_link = {stqe_next = 0x0}, ta_pending = 0, ta_priority = 0, ta_func = 0xffffffff80737799 , ta_context = 0xfffffe0003002000}, if_addr_mtx = {lock_object = { lo_name = 0xffffffff80acbb20 "if_addr_mtx", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006c8b80}, mtx_lock = 4}, if_clones = {le_next = 0x0, le_prev = 0x0}, if_groups = { tqh_first = 0xfffffe0005093ae0, tqh_last = 0xfffffe0005093ae8}, if_pf_kif = 0xfffffe0005889300, if_lagg = 0x0, if_description = 0x0, if_fib = 0, if_alloctype = 6 '\006', if_cspare = "\000\000", if_ispare = {0, 0, 0, 0}, if_pspare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}} #### kgdb.out - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPu2g1SPOsGF+KA+MRAoTIAJ9zBBTdm9naccUy+S2u89hqcXl2kACfRApP bJ+OVmJETP0NtLujBxb7Kqg= =MqcS -----END PGP SIGNATURE----- --3469798045-1358106948-1337681906=:89783-- From owner-freebsd-pf@FreeBSD.ORG Tue May 22 11:00:09 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4B206106564A for ; Tue, 22 May 2012 11:00:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 187958FC08 for ; Tue, 22 May 2012 11:00:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4MB08UA026031 for ; Tue, 22 May 2012 11:00:08 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4MB08Xl026030; Tue, 22 May 2012 11:00:08 GMT (envelope-from gnats) Date: Tue, 22 May 2012 11:00:08 GMT Message-Id: <201205221100.q4MB08Xl026030@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Theodor-Iulian Ciobanu Cc: Subject: Re: kern/168200: [pf] pf crashes when receiving packets from an address in a table X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Theodor-Iulian Ciobanu List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 11:00:09 -0000 The following reply was made to PR kern/168200; it has been noted by GNATS. From: Theodor-Iulian Ciobanu To: bug-followup@FreeBSD.org, hugo@barafranca.com Cc: Subject: Re: kern/168200: [pf] pf crashes when receiving packets from an address in a table Date: Tue, 22 May 2012 13:57:34 +0300 Hello, I've hit this same issue about a month ago. See the patch here (unfortunately, it doesn't seem to have been comitted yet): http://lists.freebsd.org/pipermail/freebsd-pf/2012-April/006534.html Since I applied it the system hasn't crashed once. -- Theo From owner-freebsd-pf@FreeBSD.ORG Tue May 22 11:26:08 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A51C0106564A for ; Tue, 22 May 2012 11:26:08 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id 2664D8FC08 for ; Tue, 22 May 2012 11:26:07 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4MBQ1a3009692 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Tue, 22 May 2012 13:26:01 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4MBQ1pX012555; Tue, 22 May 2012 13:26:01 +0200 (MEST) Date: Tue, 22 May 2012 13:26:01 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120522112601.GE29536@insomnia.benzedrine.cx> References: <201205220610.q4M6A3oY036772@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205220610.q4M6A3oY036772@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 11:26:08 -0000 This (or something similar) was reported before: help w/panic under heavy load - 5.4 http://www.mail-archive.com/freebsd-hackers@freebsd.org/msg52452.html panic on ip_input, ip_len byte ordering problem? http://lists.freebsd.org/pipermail/freebsd-net/2009-July/022473.html But no resolutions were posted. Maybe Max remembers? Are you using other pfil hooks (ipfw, ipfilter, etc.)? IP fast forwarding? divert? netgraph? dup-to? What network interfaces are used (enc, gre, gif, fxp0)? What checksumming support (ifconfig if)? Daniel From owner-freebsd-pf@FreeBSD.ORG Tue May 22 11:52:57 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AF5D9106564A; Tue, 22 May 2012 11:52:57 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 5007A8FC12; Tue, 22 May 2012 11:52:57 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4MBpwIY038517; Tue, 22 May 2012 13:51:58 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4MBpsux038513 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 22 May 2012 13:51:54 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Tue, 22 May 2012 13:51:51 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120522112601.GE29536@insomnia.benzedrine.cx> Message-ID: References: <201205220610.q4M6A3oY036772@freefall.freebsd.org> <20120522112601.GE29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Tue, 22 May 2012 13:51:54 +0200 (CEST) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 11:52:57 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 22 May 2012, Daniel Hartmeier wrote: > This (or something similar) was reported before: > > help w/panic under heavy load - 5.4 > http://www.mail-archive.com/freebsd-hackers@freebsd.org/msg52452.html > > panic on ip_input, ip_len byte ordering problem? > http://lists.freebsd.org/pipermail/freebsd-net/2009-July/022473.html > > But no resolutions were posted. Maybe Max remembers? > > Are you using other pfil hooks (ipfw, ipfilter, etc.)? > > IP fast forwarding? divert? netgraph? dup-to? > > What network interfaces are used (enc, gre, gif, fxp0)? > > What checksumming support (ifconfig if)? Daniel, mails to your personal eMail address are bouncing. relay=insomnia.benzedrine.cx. [62.65.145.30], dsn=4.0.0, stat=Deferred: insomnia.benzedrine.cx.: No route to host I've found another report and a patch which i already tried without success, so i reverted back to stock 9.0-p1. http://lists.freebsd.org/pipermail/freebsd-pf/2005-March/000922.html I've the following relevant options in the kernel configuration: options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=100 options IPFIREWALL_DEFAULT_TO_ACCEPT options IPDIVERT options IPFILTER options IPFILTER_LOG options IPSTEALTH options ALTQ options ALTQ_CBQ # Class Bases Queueing options ALTQ_RED # Random Early Drop options ALTQ_RIO # RED In/Out options ALTQ_HFSC # Hierarchical Packet Scheduler options ALTQ_CDNR # Traffic conditioner options ALTQ_PRIQ # Priority Queueing options ALTQ_NOPCC # Required for SMP build options IPSEC options IPSEC_NAT_T device crypto device cryptodev device hifn device enc device pf # PF OpenBSD packet-filter firewall device pflog # logging support interface for PF device pfsync # synchronization interface for PF device carp # common address redundancy protocol Only pf(4) is configured and used. net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 0 net.inet6.ip6.forwarding: 0 No netgraph, divert or dup-to. Interface list: bge0: flags=8843 metric 0 mtu 1500 options=8009b bge1: flags=8843 metric 0 mtu 1500 options=8009b pflog0: flags=0<> metric 0 mtu 33152 pfsync0: flags=0<> metric 0 mtu 1500 ipfw0: flags=8801 metric 0 mtu 65536 lo0: flags=8049 metric 0 mtu 16384 options=3 enc0: flags=0<> metric 0 mtu 1536 Only bge0 and bge1 are configured and used. bge0 ist $ext_if and bge1 is $int_if. Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPu33aSPOsGF+KA+MRAjkLAJ0Z6K0Smp5M2p9r/VcSAUy1nqnkAACgqMq7 oHMudSKOjU3nQIGaq3M0fAo= =SuIg -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Tue May 22 12:00:16 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BE4081065749 for ; Tue, 22 May 2012 12:00:16 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A78D48FC19 for ; Tue, 22 May 2012 12:00:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4MC0G1s085515 for ; Tue, 22 May 2012 12:00:16 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4MC0Gtg085514; Tue, 22 May 2012 12:00:16 GMT (envelope-from gnats) Date: Tue, 22 May 2012 12:00:16 GMT Message-Id: <201205221200.q4MC0Gtg085514@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 12:00:16 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Tue, 22 May 2012 13:51:51 +0200 (CEST) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 22 May 2012, Daniel Hartmeier wrote: > This (or something similar) was reported before: > > help w/panic under heavy load - 5.4 > http://www.mail-archive.com/freebsd-hackers@freebsd.org/msg52452.html > > panic on ip_input, ip_len byte ordering problem? > http://lists.freebsd.org/pipermail/freebsd-net/2009-July/022473.html > > But no resolutions were posted. Maybe Max remembers? > > Are you using other pfil hooks (ipfw, ipfilter, etc.)? > > IP fast forwarding? divert? netgraph? dup-to? > > What network interfaces are used (enc, gre, gif, fxp0)? > > What checksumming support (ifconfig if)? Daniel, mails to your personal eMail address are bouncing. relay=insomnia.benzedrine.cx. [62.65.145.30], dsn=4.0.0, stat=Deferred: insomnia.benzedrine.cx.: No route to host I've found another report and a patch which i already tried without success, so i reverted back to stock 9.0-p1. http://lists.freebsd.org/pipermail/freebsd-pf/2005-March/000922.html I've the following relevant options in the kernel configuration: options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=100 options IPFIREWALL_DEFAULT_TO_ACCEPT options IPDIVERT options IPFILTER options IPFILTER_LOG options IPSTEALTH options ALTQ options ALTQ_CBQ # Class Bases Queueing options ALTQ_RED # Random Early Drop options ALTQ_RIO # RED In/Out options ALTQ_HFSC # Hierarchical Packet Scheduler options ALTQ_CDNR # Traffic conditioner options ALTQ_PRIQ # Priority Queueing options ALTQ_NOPCC # Required for SMP build options IPSEC options IPSEC_NAT_T device crypto device cryptodev device hifn device enc device pf # PF OpenBSD packet-filter firewall device pflog # logging support interface for PF device pfsync # synchronization interface for PF device carp # common address redundancy protocol Only pf(4) is configured and used. net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 0 net.inet6.ip6.forwarding: 0 No netgraph, divert or dup-to. Interface list: bge0: flags=8843 metric 0 mtu 1500 options=8009b bge1: flags=8843 metric 0 mtu 1500 options=8009b pflog0: flags=0<> metric 0 mtu 33152 pfsync0: flags=0<> metric 0 mtu 1500 ipfw0: flags=8801 metric 0 mtu 65536 lo0: flags=8049 metric 0 mtu 16384 options=3 enc0: flags=0<> metric 0 mtu 1536 Only bge0 and bge1 are configured and used. bge0 ist $ext_if and bge1 is $int_if. Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPu33aSPOsGF+KA+MRAjkLAJ0Z6K0Smp5M2p9r/VcSAUy1nqnkAACgqMq7 oHMudSKOjU3nQIGaq3M0fAo= =SuIg -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Tue May 22 15:06:10 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F5FD1065672 for ; Tue, 22 May 2012 15:06:10 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id 567DF8FC0C for ; Tue, 22 May 2012 15:06:07 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4MF6412001862 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Tue, 22 May 2012 17:06:05 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4MF64Mw010988; Tue, 22 May 2012 17:06:04 +0200 (MEST) Date: Tue, 22 May 2012 17:06:04 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120522150603.GF29536@insomnia.benzedrine.cx> References: <201205221200.q4MC0Gtg085514@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205221200.q4MC0Gtg085514@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 15:06:10 -0000 If you have the chance, please try the patch below. It adds byte order checks all over the place, hoping for a panic closer to the source of the problem. Daniel Index: sys/sys/mbuf.h =================================================================== RCS file: /home/ncvs/src/sys/sys/mbuf.h,v retrieving revision 1.242.2.1 diff -u -r1.242.2.1 mbuf.h --- sys/sys/mbuf.h 23 Sep 2011 00:51:37 -0000 1.242.2.1 +++ sys/sys/mbuf.h 22 May 2012 14:15:00 -0000 @@ -824,6 +824,20 @@ /* Compatibility with 4.3. */ #define m_copy(m, o, l) m_copym((m), (o), (l), M_DONTWAIT) +#define ASSERT_NET_BYTE_ORDER(m) do { \ + struct ip *ip = mtod((m), struct ip *); \ + if (ip->ip_len != htons(ip->ip_len) && \ + ip->ip_len == (m)->m_pkthdr.len) \ + panic("ASSERT_NET_BYTE_ORDER"); \ +} while(0) + +#define ASSERT_HOST_BYTE_ORDER(m) do { \ + struct ip *ip = mtod((m), struct ip *); \ + if (ip->ip_len != htons(ip->ip_len) && \ + ntohs(ip->ip_len) == (m)->m_pkthdr.len) \ + panic("ASSERT_NET_BYTE_ORDER"); \ +} while(0) + extern int max_datalen; /* MHLEN - max_hdr */ extern int max_hdr; /* Largest link + protocol header */ extern int max_linkhdr; /* Largest link-level header */ Index: sys/contrib/pf/net/pf.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/pf/net/pf.c,v retrieving revision 1.78.2.6 diff -u -r1.78.2.6 pf.c --- sys/contrib/pf/net/pf.c 29 Feb 2012 09:47:26 -0000 1.78.2.6 +++ sys/contrib/pf/net/pf.c 22 May 2012 14:39:04 -0000 @@ -2560,6 +2560,7 @@ case AF_INET: #ifdef __FreeBSD__ /* icmp_error() expects host byte ordering */ + ASSERT_NET_BYTE_ORDER(m0); ip = mtod(m0, struct ip *); NTOHS(ip->ip_len); NTOHS(ip->ip_off); @@ -5894,6 +5895,8 @@ (dir != PF_IN && dir != PF_OUT) || oifp == NULL) panic("pf_route: invalid parameters"); + ASSERT_NET_BYTE_ORDER(*m); + #ifdef __FreeBSD__ if (pd->pf_mtag->routed++ > 3) { #else @@ -5977,6 +5980,7 @@ if (oifp != ifp) { #ifdef __FreeBSD__ + ASSERT_NET_BYTE_ORDER(m0); PF_UNLOCK(); if (pf_test(PF_OUT, ifp, &m0, NULL, NULL) != PF_PASS) { PF_LOCK(); @@ -5998,6 +6002,7 @@ goto bad; } ip = mtod(m0, struct ip *); + ASSERT_NET_BYTE_ORDER(m0); } #ifdef __FreeBSD__ @@ -6008,6 +6013,7 @@ /* * XXX: in_delayed_cksum assumes HBO for ip->ip_len (at least) */ + ASSERT_NET_BYTE_ORDER(m0); NTOHS(ip->ip_len); NTOHS(ip->ip_off); /* XXX: needed? */ in_delayed_cksum(m0); @@ -6017,6 +6023,8 @@ } m0->m_pkthdr.csum_flags &= ifp->if_hwassist; + ASSERT_NET_BYTE_ORDER(m0); + if (ntohs(ip->ip_len) <= ifp->if_mtu || (ifp->if_hwassist & CSUM_FRAGMENT && ((ip->ip_off & htons(IP_DF)) == 0))) { @@ -6104,6 +6112,7 @@ if (r->rt != PF_DUPTO) { #ifdef __FreeBSD__ /* icmp_error() expects host byte ordering */ + ASSERT_NET_BYTE_ORDER(m0); NTOHS(ip->ip_len); NTOHS(ip->ip_off); PF_UNLOCK(); @@ -6124,6 +6133,7 @@ /* * XXX: is cheaper + less error prone than own function */ + ASSERT_NET_BYTE_ORDER(m0); NTOHS(ip->ip_len); NTOHS(ip->ip_off); error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); @@ -6672,6 +6682,8 @@ #endif /* DIAGNOSTIC */ #endif + ASSERT_NET_BYTE_ORDER(m); + if (m->m_pkthdr.len < (int)sizeof(*h)) { action = PF_DROP; REASON_SET(&reason, PFRES_SHORT); Index: sys/contrib/pf/net/pf_ioctl.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/pf/net/pf_ioctl.c,v retrieving revision 1.50.2.4 diff -u -r1.50.2.4 pf_ioctl.c --- sys/contrib/pf/net/pf_ioctl.c 29 Feb 2012 09:47:26 -0000 1.50.2.4 +++ sys/contrib/pf/net/pf_ioctl.c 22 May 2012 14:37:44 -0000 @@ -4121,6 +4121,7 @@ if ((*m)->m_pkthdr.len >= (int)sizeof(struct ip)) { /* if m_pkthdr.len is less than ip header, pf will handle. */ + ASSERT_HOST_BYTE_ORDER(*m); h = mtod(*m, struct ip *); HTONS(h->ip_len); HTONS(h->ip_off); @@ -4134,6 +4135,7 @@ } if (*m != NULL) { /* pf_test can change ip header location */ + ASSERT_NET_BYTE_ORDER(*m); h = mtod(*m, struct ip *); NTOHS(h->ip_len); NTOHS(h->ip_off); @@ -4163,6 +4165,7 @@ } if ((*m)->m_pkthdr.len >= (int)sizeof(*h)) { /* if m_pkthdr.len is less than ip header, pf will handle. */ + ASSERT_HOST_BYTE_ORDER(*m); h = mtod(*m, struct ip *); HTONS(h->ip_len); HTONS(h->ip_off); @@ -4176,6 +4179,7 @@ } if (*m != NULL) { /* pf_test can change ip header location */ + ASSERT_NET_BYTE_ORDER(*m); h = mtod(*m, struct ip *); NTOHS(h->ip_len); NTOHS(h->ip_off); Index: sys/contrib/pf/net/pf_norm.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/pf/net/pf_norm.c,v retrieving revision 1.21.2.2 diff -u -r1.21.2.2 pf_norm.c --- sys/contrib/pf/net/pf_norm.c 29 Feb 2012 09:47:26 -0000 1.21.2.2 +++ sys/contrib/pf/net/pf_norm.c 22 May 2012 14:41:02 -0000 @@ -1190,6 +1190,8 @@ if (hlen < (int)sizeof(struct ip)) goto drop; + ASSERT_NET_BYTE_ORDER(m); + if (hlen > ntohs(h->ip_len)) goto drop; Index: sys/net/if_bridge.c =================================================================== RCS file: /home/ncvs/src/sys/net/if_bridge.c,v retrieving revision 1.144.2.2 diff -u -r1.144.2.2 if_bridge.c --- sys/net/if_bridge.c 17 Mar 2012 12:11:53 -0000 1.144.2.2 +++ sys/net/if_bridge.c 22 May 2012 14:44:14 -0000 @@ -3142,6 +3142,7 @@ */ ip = mtod(*mp, struct ip *); + ASSERT_NET_BYTE_ORDER(*mp); ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); @@ -3195,6 +3196,7 @@ if (ip == NULL) goto bad; } + ASSERT_HOST_BYTE_ORDER(*mp); ip->ip_len = htons(ip->ip_len); ip->ip_off = htons(ip->ip_off); ip->ip_sum = 0; @@ -3332,6 +3334,7 @@ } /* Retrieve the packet length. */ + ASSERT_NET_BYTE_ORDER(m); len = ntohs(ip->ip_len); /* Index: sys/net/if_enc.c =================================================================== RCS file: /home/ncvs/src/sys/net/if_enc.c,v retrieving revision 1.17.2.1 diff -u -r1.17.2.1 if_enc.c --- sys/net/if_enc.c 23 Sep 2011 00:51:37 -0000 1.17.2.1 +++ sys/net/if_enc.c 22 May 2012 14:43:27 -0000 @@ -274,6 +274,7 @@ * before calling the firewall, swap fields the same as * IP does. here we assume the header is contiguous */ + ASSERT_NET_BYTE_ORDER(*mp); ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); @@ -284,6 +285,7 @@ break; /* restore byte ordering */ + ASSERT_HOST_BYTE_ORDER(*mp); ip = mtod(*mp, struct ip *); ip->ip_len = htons(ip->ip_len); ip->ip_off = htons(ip->ip_off); Index: sys/net/pfil.c =================================================================== RCS file: /home/ncvs/src/sys/net/pfil.c,v retrieving revision 1.19.2.1 diff -u -r1.19.2.1 pfil.c --- sys/net/pfil.c 23 Sep 2011 00:51:37 -0000 1.19.2.1 +++ sys/net/pfil.c 22 May 2012 14:49:24 -0000 @@ -46,6 +46,8 @@ #include #include +#include +#include static struct mtx pfil_global_lock; @@ -79,10 +81,12 @@ for (pfh = pfil_hook_get(dir, ph); pfh != NULL; pfh = TAILQ_NEXT(pfh, pfil_link)) { if (pfh->pfil_func != NULL) { + ASSERT_HOST_BYTE_ORDER(m); rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, inp); if (rv != 0 || m == NULL) break; + ASSERT_HOST_BYTE_ORDER(m); } } PFIL_RUNLOCK(ph, &rmpt); Index: sys/netinet/ip_divert.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_divert.c,v retrieving revision 1.173.2.1 diff -u -r1.173.2.1 ip_divert.c --- sys/netinet/ip_divert.c 23 Sep 2011 00:51:37 -0000 1.173.2.1 +++ sys/netinet/ip_divert.c 22 May 2012 14:27:15 -0000 @@ -207,6 +207,7 @@ (m = m_pullup(m, sizeof(struct ip))) == 0) return; ip = mtod(m, struct ip *); + ASSERT_NET_BYTE_ORDER(m); /* Delayed checksums are currently not compatible with divert. */ if (m->m_pkthdr.csum_flags & CSUM_DELAY_DATA) { @@ -396,6 +397,7 @@ /* Convert fields to host order for ip_output() */ ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); + ASSERT_HOST_BYTE_ORDER(m); break; #ifdef INET6 case IPV6_VERSION >> 4: Index: sys/netinet/ip_fastfwd.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_fastfwd.c,v retrieving revision 1.57.2.1 diff -u -r1.57.2.1 ip_fastfwd.c --- sys/netinet/ip_fastfwd.c 23 Sep 2011 00:51:37 -0000 1.57.2.1 +++ sys/netinet/ip_fastfwd.c 22 May 2012 14:46:00 -0000 @@ -179,6 +179,7 @@ M_ASSERTVALID(m); M_ASSERTPKTHDR(m); + ASSERT_NET_BYTE_ORDER(m); bzero(&ro, sizeof(ro)); @@ -343,6 +344,7 @@ /* * Convert to host representation */ + ASSERT_NET_BYTE_ORDER(m); ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); @@ -361,6 +363,7 @@ M_ASSERTVALID(m); M_ASSERTPKTHDR(m); + ASSERT_HOST_BYTE_ORDER(m); ip = mtod(m, struct ip *); /* m may have changed by pfil hook */ dest.s_addr = ip->ip_dst.s_addr; @@ -442,12 +445,14 @@ if (!PFIL_HOOKED(&V_inet_pfil_hook)) goto passout; + ASSERT_HOST_BYTE_ORDER(m); if (pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_OUT, NULL) || m == NULL) { goto drop; } M_ASSERTVALID(m); M_ASSERTPKTHDR(m); + ASSERT_HOST_BYTE_ORDER(m); ip = mtod(m, struct ip *); dest.s_addr = ip->ip_dst.s_addr; @@ -511,6 +516,7 @@ goto consumed; } + ASSERT_HOST_BYTE_ORDER(m); #ifndef ALTQ /* * Check if there is enough space in the interface queue Index: sys/netinet/ip_icmp.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_icmp.c,v retrieving revision 1.145.2.2 diff -u -r1.145.2.2 ip_icmp.c --- sys/netinet/ip_icmp.c 19 Mar 2012 20:49:16 -0000 1.145.2.2 +++ sys/netinet/ip_icmp.c 22 May 2012 14:31:17 -0000 @@ -185,6 +185,7 @@ unsigned icmplen, icmpelen, nlen; KASSERT((u_int)type <= ICMP_MAXTYPE, ("%s: illegal ICMP type", __func__)); + ASSERT_HOST_BYTE_ORDER(n); #ifdef ICMPPRINTFS if (icmpprintfs) printf("icmp_error(%p, %x, %d)\n", oip, type, code); @@ -336,6 +337,7 @@ void (*ctlfunc)(int, struct sockaddr *, void *); int fibnum; + ASSERT_HOST_BYTE_ORDER(m); /* * Locate icmp structure in mbuf, and check * that not corrupted and of at least minimum length. @@ -866,6 +868,7 @@ register int hlen; register struct icmp *icp; + ASSERT_HOST_BYTE_ORDER(m); hlen = ip->ip_hl << 2; m->m_data += hlen; m->m_len -= hlen; Index: sys/netinet/ip_input.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_input.c,v retrieving revision 1.393.2.3 diff -u -r1.393.2.3 ip_input.c --- sys/netinet/ip_input.c 19 Mar 2012 20:49:16 -0000 1.393.2.3 +++ sys/netinet/ip_input.c 22 May 2012 14:23:45 -0000 @@ -385,6 +385,7 @@ struct in_addr odst; /* original dst address */ M_ASSERTPKTHDR(m); + ASSERT_NET_BYTE_ORDER(m); if (m->m_flags & M_FASTFWD_OURS) { /* @@ -467,6 +468,7 @@ goto bad; } ip->ip_off = ntohs(ip->ip_off); + ASSERT_HOST_BYTE_ORDER(m); /* * Check that the amount of data in the buffers @@ -1371,6 +1373,7 @@ struct route ro; int error, type = 0, code = 0, mtu = 0; + ASSERT_HOST_BYTE_ORDER(m); if (m->m_flags & (M_BCAST|M_MCAST) || in_canforward(ip->ip_dst) == 0) { IPSTAT_INC(ips_cantforward); m_freem(m); Index: sys/netinet/ip_ipsec.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_ipsec.c,v retrieving revision 1.28.2.1 diff -u -r1.28.2.1 ip_ipsec.c --- sys/netinet/ip_ipsec.c 23 Sep 2011 00:51:37 -0000 1.28.2.1 +++ sys/netinet/ip_ipsec.c 22 May 2012 14:25:41 -0000 @@ -346,6 +346,7 @@ (*m)->m_pkthdr.csum_flags &= ~CSUM_SCTP; } #endif + ASSERT_HOST_BYTE_ORDER(*m); ip->ip_len = htons(ip->ip_len); ip->ip_off = htons(ip->ip_off); Index: sys/netinet/ip_mroute.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_mroute.c,v retrieving revision 1.161.2.2 diff -u -r1.161.2.2 ip_mroute.c --- sys/netinet/ip_mroute.c 28 Mar 2012 12:45:35 -0000 1.161.2.2 +++ sys/netinet/ip_mroute.c 22 May 2012 14:32:54 -0000 @@ -1496,6 +1496,7 @@ vifi_t vifi; int plen = ip->ip_len; + ASSERT_HOST_BYTE_ORDER(m); VIF_LOCK_ASSERT(); /* @@ -2375,6 +2376,8 @@ struct mbuf *mb_copy = NULL; int mtu; + ASSERT_HOST_BYTE_ORDER(m); + /* Take care of delayed checksums */ if (m->m_pkthdr.csum_flags & CSUM_DELAY_DATA) { in_delayed_cksum(m); Index: sys/netinet/ip_output.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_output.c,v retrieving revision 1.329.2.2 diff -u -r1.329.2.2 ip_output.c --- sys/netinet/ip_output.c 10 Nov 2011 20:28:30 -0000 1.329.2.2 +++ sys/netinet/ip_output.c 22 May 2012 14:47:14 -0000 @@ -133,6 +133,7 @@ int no_route_but_check_spd = 0; #endif M_ASSERTPKTHDR(m); + ASSERT_HOST_BYTE_ORDER(m); if (inp != NULL) { INP_LOCK_ASSERT(inp); @@ -434,6 +435,8 @@ } } + ASSERT_HOST_BYTE_ORDER(m); + /* * Verify that we have any chance at all of being able to queue the * packet or packet fragments, unless ALTQ is enabled on the given @@ -505,6 +508,7 @@ /* Run through list of hooks for output packets. */ odst.s_addr = ip->ip_dst.s_addr; + ASSERT_HOST_BYTE_ORDER(m); error = pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_OUT, inp); if (error != 0 || m == NULL) goto done; @@ -596,6 +600,7 @@ * If small enough for interface, or the interface will take * care of the fragmentation for us, we can just send directly. */ + ASSERT_HOST_BYTE_ORDER(m); if (ip->ip_len <= mtu || (m->m_pkthdr.csum_flags & ifp->if_hwassist & CSUM_TSO) != 0 || ((ip->ip_off & IP_DF) == 0 && (ifp->if_hwassist & CSUM_FRAGMENT))) { @@ -628,6 +633,7 @@ * to avoid confusing lower layers. */ m->m_flags &= ~(M_PROTOFLAGS); + ASSERT_NET_BYTE_ORDER(m); error = (*ifp->if_output)(ifp, m, (struct sockaddr *)dst, ro); goto done; @@ -716,6 +722,8 @@ if (len < 8) return EMSGSIZE; + ASSERT_HOST_BYTE_ORDER(m0); + /* * If the interface will not calculate checksums on * fragmented packets, then do it here. From owner-freebsd-pf@FreeBSD.ORG Tue May 22 18:09:52 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B0F08106564A for ; Tue, 22 May 2012 18:09:52 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 61CC08FC15 for ; Tue, 22 May 2012 18:09:52 +0000 (UTC) Received: by qcsg15 with SMTP id g15so5308255qcs.13 for ; Tue, 22 May 2012 11:09:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=U5riKCD2POJWhMQKxBR/PfIHKTv03MaMOSXM5oMV/Ws=; b=GitJh7Iv7xV61Cpwc9fcj1smb/AMzVzNdxOaaeUN5sQuQBTOYY/Z6NeXF7KjFiXacJ 6iSgR1RR6MpTFRgfZjw24Gbes8wOhlEqbwPcPOUYOPKQ3iLHWaBxONeTLQl/VbwL7zw1 Un5zWD47m1Xtw8qzhwEJa7kiFpvORUAndt7WCYWO1nmVlLUjMd97ITYE94wNa9dchDvM qupN2r5FKCBjzprL5pWYsdEp1/ENkk+8ojP+5/bQcsk78lQFkdxHHkBWNTA8Sz0/qKs0 vcTScpFilVByVU7SSfvG4ZBWxD78IrtX9zJ6hdfWB1sDGd6HEqGMi+LJ/tB1+ZKCCixI aiYg== MIME-Version: 1.0 Received: by 10.229.135.130 with SMTP id n2mr12497069qct.35.1337710191591; Tue, 22 May 2012 11:09:51 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.229.89.143 with HTTP; Tue, 22 May 2012 11:09:51 -0700 (PDT) In-Reply-To: <20120522150603.GF29536@insomnia.benzedrine.cx> References: <201205221200.q4MC0Gtg085514@freefall.freebsd.org> <20120522150603.GF29536@insomnia.benzedrine.cx> Date: Tue, 22 May 2012 14:09:51 -0400 X-Google-Sender-Auth: iFaYcEhZS4eCB4lYtedGHrjKhIY Message-ID: From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Daniel Hartmeier Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 18:09:52 -0000 iirc this is from fastforwarding being enabled. Just from memory though, cause i remember seeing this panic as well. Again, from memory this is fastforwarding related, try disabling it. If it was pf(4) surely in pfSense would have been seen more frequently and in pfSense fastforwarding is not used but normal path.... On Tue, May 22, 2012 at 11:06 AM, Daniel Hartmeier w= rote: > If you have the chance, please try the patch below. > > It adds byte order checks all over the place, hoping for a panic closer > to the source of the problem. > > Daniel > > > Index: sys/sys/mbuf.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/sys/mbuf.h,v > retrieving revision 1.242.2.1 > diff -u -r1.242.2.1 mbuf.h > --- sys/sys/mbuf.h =A0 =A0 =A023 Sep 2011 00:51:37 -0000 =A0 =A0 =A01.242= .2.1 > +++ sys/sys/mbuf.h =A0 =A0 =A022 May 2012 14:15:00 -0000 > @@ -824,6 +824,20 @@ > =A0/* Compatibility with 4.3. */ > =A0#define =A0 =A0 =A0 =A0m_copy(m, o, l) m_copym((m), (o), (l), M_DONTWA= IT) > > +#define ASSERT_NET_BYTE_ORDER(m) do { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0\ > + =A0 =A0 =A0 struct ip *ip =3D mtod((m), struct ip *); =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 \ > + =A0 =A0 =A0 if (ip->ip_len !=3D htons(ip->ip_len) && =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0\ > + =A0 =A0 =A0 =A0 =A0 ip->ip_len =3D=3D (m)->m_pkthdr.len) =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0\ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 panic("ASSERT_NET_BYTE_ORDER"); =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 \ > +} while(0) > + > +#define ASSERT_HOST_BYTE_ORDER(m) do { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 \ > + =A0 =A0 =A0 struct ip *ip =3D mtod((m), struct ip *); =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 \ > + =A0 =A0 =A0 if (ip->ip_len !=3D htons(ip->ip_len) && =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0\ > + =A0 =A0 =A0 =A0 =A0 ntohs(ip->ip_len) =3D=3D (m)->m_pkthdr.len) =A0 =A0= =A0 =A0 =A0 =A0 \ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 panic("ASSERT_NET_BYTE_ORDER"); =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 \ > +} while(0) > + > =A0extern int =A0 =A0 =A0 =A0 =A0 =A0 max_datalen; =A0 =A0/* MHLEN - max_= hdr */ > =A0extern int =A0 =A0 =A0 =A0 =A0 =A0 max_hdr; =A0 =A0 =A0 =A0/* Largest = link + protocol header */ > =A0extern int =A0 =A0 =A0 =A0 =A0 =A0 max_linkhdr; =A0 =A0/* Largest link= -level header */ > Index: sys/contrib/pf/net/pf.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/contrib/pf/net/pf.c,v > retrieving revision 1.78.2.6 > diff -u -r1.78.2.6 pf.c > --- sys/contrib/pf/net/pf.c =A0 =A0 29 Feb 2012 09:47:26 -0000 =A0 =A0 = =A01.78.2.6 > +++ sys/contrib/pf/net/pf.c =A0 =A0 22 May 2012 14:39:04 -0000 > @@ -2560,6 +2560,7 @@ > =A0 =A0 =A0 =A0case AF_INET: > =A0#ifdef __FreeBSD__ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* icmp_error() expects host byte ordering= */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m0); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip =3D mtod(m0, struct ip *); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(ip->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(ip->ip_off); > @@ -5894,6 +5895,8 @@ > =A0 =A0 =A0 =A0 =A0 =A0(dir !=3D PF_IN && dir !=3D PF_OUT) || oifp =3D=3D= NULL) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0panic("pf_route: invalid parameters"); > > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(*m); > + > =A0#ifdef __FreeBSD__ > =A0 =A0 =A0 =A0if (pd->pf_mtag->routed++ > 3) { > =A0#else > @@ -5977,6 +5980,7 @@ > > =A0 =A0 =A0 =A0if (oifp !=3D ifp) { > =A0#ifdef __FreeBSD__ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m0); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0PF_UNLOCK(); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (pf_test(PF_OUT, ifp, &m0, NULL, NULL) = !=3D PF_PASS) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0PF_LOCK(); > @@ -5998,6 +6002,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto bad; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip =3D mtod(m0, struct ip *); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m0); > =A0 =A0 =A0 =A0} > > =A0#ifdef __FreeBSD__ > @@ -6008,6 +6013,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * XXX: in_delayed_cksum assumes HBO for i= p->ip_len (at least) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m0); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(ip->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(ip->ip_off); =A0 =A0 =A0/* XXX: need= ed? */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0in_delayed_cksum(m0); > @@ -6017,6 +6023,8 @@ > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0m0->m_pkthdr.csum_flags &=3D ifp->if_hwassist; > > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m0); > + > =A0 =A0 =A0 =A0if (ntohs(ip->ip_len) <=3D ifp->if_mtu || > =A0 =A0 =A0 =A0 =A0 =A0(ifp->if_hwassist & CSUM_FRAGMENT && > =A0 =A0 =A0 =A0 =A0 =A0((ip->ip_off & htons(IP_DF)) =3D=3D 0))) { > @@ -6104,6 +6112,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (r->rt !=3D PF_DUPTO) { > =A0#ifdef __FreeBSD__ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* icmp_error() expects ho= st byte ordering */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m0); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(ip->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(ip->ip_off); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0PF_UNLOCK(); > @@ -6124,6 +6133,7 @@ > =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 * XXX: is cheaper + less error prone than own function > =A0 =A0 =A0 =A0 */ > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m0); > =A0 =A0 =A0 =A0NTOHS(ip->ip_len); > =A0 =A0 =A0 =A0NTOHS(ip->ip_off); > =A0 =A0 =A0 =A0error =3D ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassi= st, sw_csum); > @@ -6672,6 +6682,8 @@ > =A0#endif /* DIAGNOSTIC */ > =A0#endif > > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m); > + > =A0 =A0 =A0 =A0if (m->m_pkthdr.len < (int)sizeof(*h)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0action =3D PF_DROP; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0REASON_SET(&reason, PFRES_SHORT); > Index: sys/contrib/pf/net/pf_ioctl.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/contrib/pf/net/pf_ioctl.c,v > retrieving revision 1.50.2.4 > diff -u -r1.50.2.4 pf_ioctl.c > --- sys/contrib/pf/net/pf_ioctl.c =A0 =A0 =A0 29 Feb 2012 09:47:26 -0000 = =A0 =A0 =A01.50.2.4 > +++ sys/contrib/pf/net/pf_ioctl.c =A0 =A0 =A0 22 May 2012 14:37:44 -0000 > @@ -4121,6 +4121,7 @@ > > =A0 =A0 =A0 =A0if ((*m)->m_pkthdr.len >=3D (int)sizeof(struct ip)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* if m_pkthdr.len is less than ip header,= pf will handle. */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(*m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0h =3D mtod(*m, struct ip *); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0HTONS(h->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0HTONS(h->ip_off); > @@ -4134,6 +4135,7 @@ > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0if (*m !=3D NULL) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* pf_test can change ip header location *= / > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(*m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0h =3D mtod(*m, struct ip *); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(h->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(h->ip_off); > @@ -4163,6 +4165,7 @@ > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0if ((*m)->m_pkthdr.len >=3D (int)sizeof(*h)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* if m_pkthdr.len is less than ip header,= pf will handle. */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(*m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0h =3D mtod(*m, struct ip *); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0HTONS(h->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0HTONS(h->ip_off); > @@ -4176,6 +4179,7 @@ > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0if (*m !=3D NULL) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* pf_test can change ip header location *= / > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(*m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0h =3D mtod(*m, struct ip *); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(h->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NTOHS(h->ip_off); > Index: sys/contrib/pf/net/pf_norm.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/contrib/pf/net/pf_norm.c,v > retrieving revision 1.21.2.2 > diff -u -r1.21.2.2 pf_norm.c > --- sys/contrib/pf/net/pf_norm.c =A0 =A0 =A0 =A029 Feb 2012 09:47:26 -000= 0 =A0 =A0 =A01.21.2.2 > +++ sys/contrib/pf/net/pf_norm.c =A0 =A0 =A0 =A022 May 2012 14:41:02 -000= 0 > @@ -1190,6 +1190,8 @@ > =A0 =A0 =A0 =A0if (hlen < (int)sizeof(struct ip)) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto drop; > > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m); > + > =A0 =A0 =A0 =A0if (hlen > ntohs(h->ip_len)) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto drop; > > Index: sys/net/if_bridge.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/net/if_bridge.c,v > retrieving revision 1.144.2.2 > diff -u -r1.144.2.2 if_bridge.c > --- sys/net/if_bridge.c 17 Mar 2012 12:11:53 -0000 =A0 =A0 =A01.144.2.2 > +++ sys/net/if_bridge.c 22 May 2012 14:44:14 -0000 > @@ -3142,6 +3142,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip =3D mtod(*mp, struct ip *); > > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(*mp); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_len =3D ntohs(ip->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_off =3D ntohs(ip->ip_off); > > @@ -3195,6 +3196,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (ip =3D=3D NULL) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto bad; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(*mp); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_len =3D htons(ip->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_off =3D htons(ip->ip_off); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_sum =3D 0; > @@ -3332,6 +3334,7 @@ > =A0 =A0 =A0 =A0} > > =A0 =A0 =A0 =A0/* Retrieve the packet length. */ > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m); > =A0 =A0 =A0 =A0len =3D ntohs(ip->ip_len); > > =A0 =A0 =A0 =A0/* > Index: sys/net/if_enc.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/net/if_enc.c,v > retrieving revision 1.17.2.1 > diff -u -r1.17.2.1 if_enc.c > --- sys/net/if_enc.c =A0 =A023 Sep 2011 00:51:37 -0000 =A0 =A0 =A01.17.2.= 1 > +++ sys/net/if_enc.c =A0 =A022 May 2012 14:43:27 -0000 > @@ -274,6 +274,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * before calling the fire= wall, swap fields the same as > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * IP does. here we assume= the header is contiguous > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(*mp); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_len =3D ntohs(ip->i= p_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_off =3D ntohs(ip->i= p_off); > > @@ -284,6 +285,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* restore byte ordering *= / > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(*mp)= ; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip =3D mtod(*mp, struct ip= *); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_len =3D htons(ip->i= p_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_off =3D htons(ip->i= p_off); > Index: sys/net/pfil.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/net/pfil.c,v > retrieving revision 1.19.2.1 > diff -u -r1.19.2.1 pfil.c > --- sys/net/pfil.c =A0 =A0 =A023 Sep 2011 00:51:37 -0000 =A0 =A0 =A01.19.= 2.1 > +++ sys/net/pfil.c =A0 =A0 =A022 May 2012 14:49:24 -0000 > @@ -46,6 +46,8 @@ > > =A0#include > =A0#include > +#include > +#include > > =A0static struct mtx pfil_global_lock; > > @@ -79,10 +81,12 @@ > =A0 =A0 =A0 =A0for (pfh =3D pfil_hook_get(dir, ph); pfh !=3D NULL; > =A0 =A0 =A0 =A0 =A0 =A0 pfh =3D TAILQ_NEXT(pfh, pfil_link)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (pfh->pfil_func !=3D NULL) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0rv =3D (*pfh->pfil_func)(p= fh->pfil_arg, &m, ifp, dir, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0inp); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (rv !=3D 0 || m =3D=3D = NULL) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0PFIL_RUNLOCK(ph, &rmpt); > Index: sys/netinet/ip_divert.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/netinet/ip_divert.c,v > retrieving revision 1.173.2.1 > diff -u -r1.173.2.1 ip_divert.c > --- sys/netinet/ip_divert.c =A0 =A0 23 Sep 2011 00:51:37 -0000 =A0 =A0 = =A01.173.2.1 > +++ sys/netinet/ip_divert.c =A0 =A0 22 May 2012 14:27:15 -0000 > @@ -207,6 +207,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0(m =3D m_pullup(m, sizeof(struct ip))) =3D=3D 0) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return; > =A0 =A0 =A0 =A0ip =3D mtod(m, struct ip *); > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m); > > =A0 =A0 =A0 =A0/* Delayed checksums are currently not compatible with div= ert. */ > =A0 =A0 =A0 =A0if (m->m_pkthdr.csum_flags & CSUM_DELAY_DATA) { > @@ -396,6 +397,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* Convert fields to host = order for ip_output() */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_len =3D ntohs(ip->i= p_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_off =3D ntohs(ip->i= p_off); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > =A0#ifdef INET6 > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0case IPV6_VERSION >> 4: > Index: sys/netinet/ip_fastfwd.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/netinet/ip_fastfwd.c,v > retrieving revision 1.57.2.1 > diff -u -r1.57.2.1 ip_fastfwd.c > --- sys/netinet/ip_fastfwd.c =A0 =A023 Sep 2011 00:51:37 -0000 =A0 =A0 = =A01.57.2.1 > +++ sys/netinet/ip_fastfwd.c =A0 =A022 May 2012 14:46:00 -0000 > @@ -179,6 +179,7 @@ > > =A0 =A0 =A0 =A0M_ASSERTVALID(m); > =A0 =A0 =A0 =A0M_ASSERTPKTHDR(m); > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m); > > =A0 =A0 =A0 =A0bzero(&ro, sizeof(ro)); > > @@ -343,6 +344,7 @@ > =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 * Convert to host representation > =A0 =A0 =A0 =A0 */ > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m); > =A0 =A0 =A0 =A0ip->ip_len =3D ntohs(ip->ip_len); > =A0 =A0 =A0 =A0ip->ip_off =3D ntohs(ip->ip_off); > > @@ -361,6 +363,7 @@ > > =A0 =A0 =A0 =A0M_ASSERTVALID(m); > =A0 =A0 =A0 =A0M_ASSERTPKTHDR(m); > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > > =A0 =A0 =A0 =A0ip =3D mtod(m, struct ip *); =A0 =A0 =A0/* m may have chan= ged by pfil hook */ > =A0 =A0 =A0 =A0dest.s_addr =3D ip->ip_dst.s_addr; > @@ -442,12 +445,14 @@ > =A0 =A0 =A0 =A0if (!PFIL_HOOKED(&V_inet_pfil_hook)) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto passout; > > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0if (pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_OUT, N= ULL) || m =3D=3D NULL) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto drop; > =A0 =A0 =A0 =A0} > > =A0 =A0 =A0 =A0M_ASSERTVALID(m); > =A0 =A0 =A0 =A0M_ASSERTPKTHDR(m); > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > > =A0 =A0 =A0 =A0ip =3D mtod(m, struct ip *); > =A0 =A0 =A0 =A0dest.s_addr =3D ip->ip_dst.s_addr; > @@ -511,6 +516,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto consumed; > =A0 =A0 =A0 =A0} > > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0#ifndef ALTQ > =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 * Check if there is enough space in the interface queue > Index: sys/netinet/ip_icmp.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/netinet/ip_icmp.c,v > retrieving revision 1.145.2.2 > diff -u -r1.145.2.2 ip_icmp.c > --- sys/netinet/ip_icmp.c =A0 =A0 =A0 19 Mar 2012 20:49:16 -0000 =A0 =A0 = =A01.145.2.2 > +++ sys/netinet/ip_icmp.c =A0 =A0 =A0 22 May 2012 14:31:17 -0000 > @@ -185,6 +185,7 @@ > =A0 =A0 =A0 =A0unsigned icmplen, icmpelen, nlen; > > =A0 =A0 =A0 =A0KASSERT((u_int)type <=3D ICMP_MAXTYPE, ("%s: illegal ICMP = type", __func__)); > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(n); > =A0#ifdef ICMPPRINTFS > =A0 =A0 =A0 =A0if (icmpprintfs) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0printf("icmp_error(%p, %x, %d)\n", oip, ty= pe, code); > @@ -336,6 +337,7 @@ > =A0 =A0 =A0 =A0void (*ctlfunc)(int, struct sockaddr *, void *); > =A0 =A0 =A0 =A0int fibnum; > > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 * Locate icmp structure in mbuf, and check > =A0 =A0 =A0 =A0 * that not corrupted and of at least minimum length. > @@ -866,6 +868,7 @@ > =A0 =A0 =A0 =A0register int hlen; > =A0 =A0 =A0 =A0register struct icmp *icp; > > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0hlen =3D ip->ip_hl << 2; > =A0 =A0 =A0 =A0m->m_data +=3D hlen; > =A0 =A0 =A0 =A0m->m_len -=3D hlen; > Index: sys/netinet/ip_input.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/netinet/ip_input.c,v > retrieving revision 1.393.2.3 > diff -u -r1.393.2.3 ip_input.c > --- sys/netinet/ip_input.c =A0 =A0 =A019 Mar 2012 20:49:16 -0000 =A0 =A0 = =A01.393.2.3 > +++ sys/netinet/ip_input.c =A0 =A0 =A022 May 2012 14:23:45 -0000 > @@ -385,6 +385,7 @@ > =A0 =A0 =A0 =A0struct in_addr odst; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0/* original dst address */ > > =A0 =A0 =A0 =A0M_ASSERTPKTHDR(m); > + =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m); > > =A0 =A0 =A0 =A0if (m->m_flags & M_FASTFWD_OURS) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > @@ -467,6 +468,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto bad; > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0ip->ip_off =3D ntohs(ip->ip_off); > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > > =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 * Check that the amount of data in the buffers > @@ -1371,6 +1373,7 @@ > =A0 =A0 =A0 =A0struct route ro; > =A0 =A0 =A0 =A0int error, type =3D 0, code =3D 0, mtu =3D 0; > > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0if (m->m_flags & (M_BCAST|M_MCAST) || in_canforward(ip->ip= _dst) =3D=3D 0) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0IPSTAT_INC(ips_cantforward); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0m_freem(m); > Index: sys/netinet/ip_ipsec.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/netinet/ip_ipsec.c,v > retrieving revision 1.28.2.1 > diff -u -r1.28.2.1 ip_ipsec.c > --- sys/netinet/ip_ipsec.c =A0 =A0 =A023 Sep 2011 00:51:37 -0000 =A0 =A0 = =A01.28.2.1 > +++ sys/netinet/ip_ipsec.c =A0 =A0 =A022 May 2012 14:25:41 -0000 > @@ -346,6 +346,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(*m)->m_pkthdr.csum_flags = &=3D ~CSUM_SCTP; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > =A0#endif > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(*m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_len =3D htons(ip->ip_len); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ip->ip_off =3D htons(ip->ip_off); > > Index: sys/netinet/ip_mroute.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/netinet/ip_mroute.c,v > retrieving revision 1.161.2.2 > diff -u -r1.161.2.2 ip_mroute.c > --- sys/netinet/ip_mroute.c =A0 =A0 28 Mar 2012 12:45:35 -0000 =A0 =A0 = =A01.161.2.2 > +++ sys/netinet/ip_mroute.c =A0 =A0 22 May 2012 14:32:54 -0000 > @@ -1496,6 +1496,7 @@ > =A0 =A0 vifi_t vifi; > =A0 =A0 int plen =3D ip->ip_len; > > + =A0 =A0ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 VIF_LOCK_ASSERT(); > > =A0 =A0 /* > @@ -2375,6 +2376,8 @@ > =A0 =A0 struct mbuf *mb_copy =3D NULL; > =A0 =A0 int mtu; > > + =A0 =A0ASSERT_HOST_BYTE_ORDER(m); > + > =A0 =A0 /* Take care of delayed checksums */ > =A0 =A0 if (m->m_pkthdr.csum_flags & CSUM_DELAY_DATA) { > =A0 =A0 =A0 =A0in_delayed_cksum(m); > Index: sys/netinet/ip_output.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/sys/netinet/ip_output.c,v > retrieving revision 1.329.2.2 > diff -u -r1.329.2.2 ip_output.c > --- sys/netinet/ip_output.c =A0 =A0 10 Nov 2011 20:28:30 -0000 =A0 =A0 = =A01.329.2.2 > +++ sys/netinet/ip_output.c =A0 =A0 22 May 2012 14:47:14 -0000 > @@ -133,6 +133,7 @@ > =A0 =A0 =A0 =A0int no_route_but_check_spd =3D 0; > =A0#endif > =A0 =A0 =A0 =A0M_ASSERTPKTHDR(m); > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > > =A0 =A0 =A0 =A0if (inp !=3D NULL) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0INP_LOCK_ASSERT(inp); > @@ -434,6 +435,8 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0} > > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > + > =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 * Verify that we have any chance at all of being able to = queue the > =A0 =A0 =A0 =A0 * packet or packet fragments, unless ALTQ is enabled on t= he given > @@ -505,6 +508,7 @@ > > =A0 =A0 =A0 =A0/* Run through list of hooks for output packets. */ > =A0 =A0 =A0 =A0odst.s_addr =3D ip->ip_dst.s_addr; > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0error =3D pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_= OUT, inp); > =A0 =A0 =A0 =A0if (error !=3D 0 || m =3D=3D NULL) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto done; > @@ -596,6 +600,7 @@ > =A0 =A0 =A0 =A0 * If small enough for interface, or the interface will ta= ke > =A0 =A0 =A0 =A0 * care of the fragmentation for us, we can just send dire= ctly. > =A0 =A0 =A0 =A0 */ > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m); > =A0 =A0 =A0 =A0if (ip->ip_len <=3D mtu || > =A0 =A0 =A0 =A0 =A0 =A0(m->m_pkthdr.csum_flags & ifp->if_hwassist & CSUM_= TSO) !=3D 0 || > =A0 =A0 =A0 =A0 =A0 =A0((ip->ip_off & IP_DF) =3D=3D 0 && (ifp->if_hwassis= t & CSUM_FRAGMENT))) { > @@ -628,6 +633,7 @@ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * to avoid confusing lower layers. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0m->m_flags &=3D ~(M_PROTOFLAGS); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 ASSERT_NET_BYTE_ORDER(m); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0error =3D (*ifp->if_output)(ifp, m, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(struct so= ckaddr *)dst, ro); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0goto done; > @@ -716,6 +722,8 @@ > =A0 =A0 =A0 =A0if (len < 8) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return EMSGSIZE; > > + =A0 =A0 =A0 ASSERT_HOST_BYTE_ORDER(m0); > + > =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 * If the interface will not calculate checksums on > =A0 =A0 =A0 =A0 * fragmented packets, then do it here. > _______________________________________________ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" --=20 Ermal From owner-freebsd-pf@FreeBSD.ORG Wed May 23 07:07:03 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7A22106566C; Wed, 23 May 2012 07:07:03 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 432F58FC08; Wed, 23 May 2012 07:07:03 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4N75ldJ069305; Wed, 23 May 2012 09:05:47 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4N75idq069301 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 23 May 2012 09:05:44 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Wed, 23 May 2012 09:05:44 +0200 (CEST) From: Joerg Pulz To: =?ISO-8859-15?Q?Ermal_Lu=E7i?= In-Reply-To: Message-ID: References: <201205221200.q4MC0Gtg085514@freefall.freebsd.org> <20120522150603.GF29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3469798045-1370558372-1337756746=:89783" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Wed, 23 May 2012 09:05:46 +0200 (CEST) Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 07:07:03 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-1370558372-1337756746=:89783 Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 22 May 2012, Ermal Luçi wrote: > iirc this is from fastforwarding being enabled. > Just from memory though, cause i remember seeing this panic as well. > > Again, from memory this is fastforwarding related, try disabling it. > If it was pf(4) surely in pfSense would have been seen more frequently > and in pfSense fastforwarding is not used but normal path.... Ermal, thanks for your reply to this. As i already stated in a previous mail, fastforwarding is not and was never used on this system. net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 0 net.inet6.ip6.forwarding: 0 Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvIxISPOsGF+KA+MRAmIUAJ4gth6QsTMXmHRCnKhsm4XQ2S0ibQCeOB8h W3C84aefIPrpu9O69pIrmEM= =/wga -----END PGP SIGNATURE----- --3469798045-1370558372-1337756746=:89783-- From owner-freebsd-pf@FreeBSD.ORG Wed May 23 13:33:44 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B29EE106566C for ; Wed, 23 May 2012 13:33:44 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-gh0-f182.google.com (mail-gh0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6652C8FC08 for ; Wed, 23 May 2012 13:33:44 +0000 (UTC) Received: by ghbz22 with SMTP id z22so1561725ghb.13 for ; Wed, 23 May 2012 06:33:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Ru7S0JSVPcDkKhrsH6IfOD1zgqlGd69jLWvuU9ak0FQ=; b=UQEEvrxO63HI35NCyU3S+jem8jmO5VVN/oirWlrmiiGDgqT1U6ZIv15zRmC8ci72/t BGotlPylrEyveHePjTi6ALzS4zh7YkKZ9otYKX0Rs3BMTU3Mdg6ZuDp+PH4sdN4KT/Uz jRk0VkF5tddlxZoRLYCtfMe2a+jIbLIySMaOQE9iW2n9rC4fdbZ5gKlHrZ1c901tOoQL PB9t0BZROGnOwsf0gDLJHdagD5JfisqDkcuXlJLRLMpnzEbHgBFiW4rDPxRlradn6Cjx kuNTWp0qW3T/SWmKXCqlX0KJq8LCVweiZHKXwHw86Fi2EsTtl7eu3lzEbscRJgQttos8 Y77Q== MIME-Version: 1.0 Received: by 10.42.77.9 with SMTP id g9mr1172411ick.4.1337780018165; Wed, 23 May 2012 06:33:38 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.231.35.202 with HTTP; Wed, 23 May 2012 06:33:37 -0700 (PDT) In-Reply-To: References: <201205221200.q4MC0Gtg085514@freefall.freebsd.org> <20120522150603.GF29536@insomnia.benzedrine.cx> Date: Wed, 23 May 2012 15:33:37 +0200 X-Google-Sender-Auth: VvmrMAjA29e_OY15Fdgc6eqq23M Message-ID: From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Joerg Pulz Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 13:33:44 -0000 On Wed, May 23, 2012 at 9:05 AM, Joerg Pulz wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > On Tue, 22 May 2012, Ermal Lu=E7i wrote: > >> iirc this is from fastforwarding being enabled. >> Just from memory though, cause i remember seeing this panic as well. >> >> Again, from memory this is fastforwarding related, try disabling it. >> If it was pf(4) surely in pfSense would have been seen more frequently >> and in pfSense fastforwarding is not used but normal path.... > > > Ermal, > > thanks for your reply to this. > As i already stated in a previous mail, fastforwarding is not and was nev= er > used on this system. > Heh i might have misread. Can you try with this patch https://github.com/bsdperimeter/pfsense-tools/blob/master/patches/RELENG_8_= 3/pf_route-to_fragemnts.RELENG_8.diff >From the commit message seems this is realted with your issue: commit 164f4705fe4474d264d5d561ac3e3d60a512d2f7 Author: Ermal Date: Sun Mar 21 19:01:34 2010 +0000 Add patch that fixes sending of fragmented packets with policy-routing. > > =A0net.inet.ip.forwarding: 1 > =A0net.inet.ip.fastforwarding: 0 > =A0net.inet6.ip6.forwarding: 0 > > Kind regards > Joerg > > - -- The beginning is the most important part of the work. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0-Plato > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.18 (FreeBSD) > > iD8DBQFPvIxISPOsGF+KA+MRAmIUAJ4gth6QsTMXmHRCnKhsm4XQ2S0ibQCeOB8h > W3C84aefIPrpu9O69pIrmEM=3D > =3D/wga > -----END PGP SIGNATURE----- --=20 Ermal From owner-freebsd-pf@FreeBSD.ORG Wed May 23 19:49:33 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 517BB106566C; Wed, 23 May 2012 19:49:33 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id B86698FC12; Wed, 23 May 2012 19:49:32 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4NJmBKx097085; Wed, 23 May 2012 21:48:11 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4NJm6RQ097073 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 23 May 2012 21:48:09 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Wed, 23 May 2012 21:48:03 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120522150603.GF29536@insomnia.benzedrine.cx> Message-ID: References: <201205221200.q4MC0Gtg085514@freefall.freebsd.org> <20120522150603.GF29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3469798045-1716925155-1337802345=:21881" Content-ID: X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Wed, 23 May 2012 21:48:09 +0200 (CEST) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 19:49:33 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-1716925155-1337802345=:21881 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: On Tue, 22 May 2012, Daniel Hartmeier wrote: > If you have the chance, please try the patch below. > > It adds byte order checks all over the place, hoping for a panic closer > to the source of the problem. Daniel, system was running for about a day with your patch with many users using it. It panic'ed some minutes ago. System configuration is still the same, no other patches, no changed interface settings or removed/changed kernel options. Here is the kgdb(1) output with "m" and "ifp" listed. I hope this helps to get closer to the source of the problem. Let me know if you need more output. Kind regards Joerg #### kgdb.out_assert GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: ASSERT_HOST_BYTE_ORDER cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 pfil_run_hooks() at pfil_run_hooks+0x159 ip_output() at ip_output+0x6de ip_forward() at ip_forward+0x19e ip_input() at ip_input+0x670 swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 585 out of 4077 MB:..3%..11%..22%..31%..41%..52%..61%..72%..82%..91% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff8074b325 in pfil_run_hooks (ph=0xfffffe000581f880, mp=0xffffff8000241978, ifp=0xfffffe0003002000, dir=2, inp=0x0) at /usr/src/sys/net/pfil.c:89 89 ASSERT_HOST_BYTE_ORDER(m); (kgdb) list 84 ASSERT_HOST_BYTE_ORDER(m); 85 rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, 86 inp); 87 if (rv != 0 || m == NULL) 88 break; 89 ASSERT_HOST_BYTE_ORDER(m); 90 } 91 } 92 PFIL_RUNLOCK(ph, &rmpt); 93 *mp = m; (kgdb) p *m $1 = {m_hdr = {mh_next = 0xfffffe000586bb00, mh_nextpkt = 0x0, mh_data = 0xfffffe010045c974 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 450, flowid = 0, csum_flags = 768, csum_data = 26073, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe000572c700}}, MH_dat = {MH_ext = { ext_buf = 0xc02c01fc0045
, ext_free = 0xc02c01c20045, ext_arg1 = 0x4d46cb4f398a0437, ext_arg2 = 0xc201004557b3bb81, ext_size = 21286, ref_cnt = 0x240119ac02079b0a, ext_type = 2059207427}, MH_databuf = "E\000ü\001,À\000\000E\000Â\001,À\000\0007\004\2129OËFM\201»³WE\000\001Â&S\000\000?\001\224\016\n\233\a\002¬\031\001$\003\003½z\000\000\000\000E\000\001¦åí\000\000>\021Ö\177¬\031\001$\n\233\a\002\0005ÿ_\001\222)Ûdh\201\200\000\001\000\003\000\b\000\bÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000Â\001\000\000\000\000\000\000\000\003\000\000Ùe\000\000\000\000\000\000ÞÀ­Þ\000Çr\005\000þÿÿE\000ü\001,À\000\000E\000Â\001,À\000\0007\004\2129OËFM\201»³WE\000\001Â&S\000\000?\001\224\016\n\233\a\002¬\031\001$\003\003½z\000\000\000\000E\000\001¦åí\000\000>\021Ö\177¬\031\001$\n\233\a\002\0005ÿ_\001\222)Ûdh\201\200\000\001\000\003\000\b\000\bÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p *ifp $2 = {if_softc = 0xffffff80007b1000, if_l2com = 0xfffffe000300ba40, if_vnet = 0x0, if_link = {tqe_next = 0xfffffe0003001000, tqe_prev = 0xfffffe0003001818}, if_xname = "bge1", '\0' , if_dname = 0xfffffe00028f07d8 "bge", if_dunit = 1, if_refcount = 1, if_addrhead = {tqh_first = 0xfffffe0003009800, tqh_last = 0xfffffe000591b4b8}, if_pcount = 0, if_carp = 0x0, if_bpf = 0xfffffe0005126900, if_index = 6, if_index_reserved = 0, if_vlantrunk = 0x0, if_flags = 34819, if_capabilities = 524443, if_capenable = 524443, if_linkmib = 0x0, if_linkmiblen = 0, if_data = { ifi_type = 6 '\006', ifi_physical = 0 '\0', ifi_addrlen = 6 '\006', ifi_hdrlen = 18 '\022', ifi_link_state = 2 '\002', ifi_spare_char1 = 0 '\0', ifi_spare_char2 = 0 '\0', ifi_datalen = 152 '\230', ifi_mtu = 1500, ifi_metric = 0, ifi_baudrate = 1000000000, ifi_ipackets = 1922972, ifi_ierrors = 0, ifi_opackets = 962786, ifi_oerrors = 0, ifi_collisions = 0, ifi_ibytes = 1150684321, ifi_obytes = 312161748, ifi_imcasts = 942443, ifi_omcasts = 0, ifi_iqdrops = 0, ifi_noproto = 0, ifi_hwassist = 3, ifi_epoch = 1, ifi_lastchange = {tv_sec = 1337714565, tv_usec = 347019}}, if_multiaddrs = {tqh_first = 0xfffffe0005915900, tqh_last = 0xfffffe0005a39100}, if_amcount = 0, if_output = 0xffffffff8073d805 , if_input = 0xffffffff8073cddb , if_start = 0xffffffff803c3087 , if_ioctl = 0xffffffff803c92ba , if_init = 0xffffffff803c9274 , if_resolvemulti = 0xffffffff8073c79d , if_qflush = 0xffffffff807355d2 , if_transmit = 0xffffffff8073549e , if_reassign = 0, if_home_vnet = 0x0, if_addr = 0xfffffe0003009800, if_llsoftc = 0x0, if_drv_flags = 64, if_snd = {ifq_head = 0x0, ifq_tail = 0x0, ifq_len = 0, ifq_maxlen = 511, ifq_drops = 0, ifq_mtx = {lock_object = { lo_name = 0xfffffe0003002028 "bge1", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006cf480}, mtx_lock = 4}, ifq_drv_head = 0x0, ifq_drv_tail = 0x0, ifq_drv_len = 0, ifq_drv_maxlen = 511, altq_type = 0, altq_flags = 1, altq_disc = 0x0, altq_ifp = 0xfffffe0003002000, altq_enqueue = 0, altq_dequeue = 0, altq_request = 0, altq_clfier = 0x0, altq_classify = 0, altq_tbr = 0x0, altq_cdnr = 0x0}, if_broadcastaddr = 0xffffffff80adb000 "ÿÿÿÿÿÿ", if_bridge = 0x0, if_label = 0x0, if_prefixhead = {tqh_first = 0x0, tqh_last = 0xfffffe0003002278}, if_afdata = {0x0, 0x0, 0xfffffe000581fa00, 0x0 , 0xfffffe0005814800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, if_afdata_initialized = 2, if_afdata_lock = { lock_object = {lo_name = 0xffffffff80ada29a "if_afdata", lo_flags = 69402624, lo_data = 0, lo_witness = 0xffffff80006cf400}, rw_lock = 1}, if_linktask = {ta_link = {stqe_next = 0x0}, ta_pending = 0, ta_priority = 0, ta_func = 0xffffffff80737a79 , ta_context = 0xfffffe0003002000}, if_addr_mtx = {lock_object = { lo_name = 0xffffffff80acc360 "if_addr_mtx", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006c8b80}, mtx_lock = 4}, if_clones = {le_next = 0x0, le_prev = 0x0}, if_groups = { tqh_first = 0xfffffe00050d3ae0, tqh_last = 0xfffffe00050d3ae8}, if_pf_kif = 0xfffffe0005889300, if_lagg = 0x0, if_description = 0x0, if_fib = 0, if_alloctype = 6 '\006', if_cspare = "\000\000", if_ispare = {0, 0, 0, 0}, if_pspare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}} (kgdb) #### kgdb.out_assert - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvT72SPOsGF+KA+MRAvxgAJ91uOe4RymMtaUOoZ7IK61/qHpoSQCZAbd0 /LVHK3BmvPKBUbd6e5rokUE= =9vPz -----END PGP SIGNATURE----- --3469798045-1716925155-1337802345=:21881-- From owner-freebsd-pf@FreeBSD.ORG Wed May 23 19:50:05 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1AEFB1065700 for ; Wed, 23 May 2012 19:50:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0464C8FC19 for ; Wed, 23 May 2012 19:50:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4NJo4Xl088702 for ; Wed, 23 May 2012 19:50:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4NJo4m1088701; Wed, 23 May 2012 19:50:04 GMT (envelope-from gnats) Date: Wed, 23 May 2012 19:50:04 GMT Message-Id: <201205231950.q4NJo4m1088701@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 19:50:05 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: =?ISO-8859-15?Q?Ermal_Lu=E7i?= , FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Wed, 23 May 2012 21:48:03 +0200 (CEST) This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-1716925155-1337802345=:21881 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: On Tue, 22 May 2012, Daniel Hartmeier wrote: > If you have the chance, please try the patch below. > > It adds byte order checks all over the place, hoping for a panic closer > to the source of the problem. Daniel, system was running for about a day with your patch with many users using it. It panic'ed some minutes ago. System configuration is still the same, no other patches, no changed interface settings or removed/changed kernel options. Here is the kgdb(1) output with "m" and "ifp" listed. I hope this helps to get closer to the source of the problem. Let me know if you need more output. Kind regards Joerg #### kgdb.out_assert GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: ASSERT_HOST_BYTE_ORDER cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 pfil_run_hooks() at pfil_run_hooks+0x159 ip_output() at ip_output+0x6de ip_forward() at ip_forward+0x19e ip_input() at ip_input+0x670 swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 585 out of 4077 MB:..3%..11%..22%..31%..41%..52%..61%..72%..82%..91% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff8074b325 in pfil_run_hooks (ph=0xfffffe000581f880, mp=0xffffff8000241978, ifp=0xfffffe0003002000, dir=2, inp=0x0) at /usr/src/sys/net/pfil.c:89 89 ASSERT_HOST_BYTE_ORDER(m); (kgdb) list 84 ASSERT_HOST_BYTE_ORDER(m); 85 rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, 86 inp); 87 if (rv != 0 || m == NULL) 88 break; 89 ASSERT_HOST_BYTE_ORDER(m); 90 } 91 } 92 PFIL_RUNLOCK(ph, &rmpt); 93 *mp = m; (kgdb) p *m $1 = {m_hdr = {mh_next = 0xfffffe000586bb00, mh_nextpkt = 0x0, mh_data = 0xfffffe010045c974 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 450, flowid = 0, csum_flags = 768, csum_data = 26073, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe000572c700}}, MH_dat = {MH_ext = { ext_buf = 0xc02c01fc0045
, ext_free = 0xc02c01c20045, ext_arg1 = 0x4d46cb4f398a0437, ext_arg2 = 0xc201004557b3bb81, ext_size = 21286, ref_cnt = 0x240119ac02079b0a, ext_type = 2059207427}, MH_databuf = "E\000ü\001,À\000\000E\000Â\001,À\000\0007\004\2129OËFM\201»³WE\000\001Â&S\000\000?\001\224\016\n\233\a\002¬\031\001$\003\003½z\000\000\000\000E\000\001¦åí\000\000>\021Ö\177¬\031\001$\n\233\a\002\0005ÿ_\001\222)Ûdh\201\200\000\001\000\003\000\b\000\bÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000Â\001\000\000\000\000\000\000\000\003\000\000Ùe\000\000\000\000\000\000ÞÀ­Þ\000Çr\005\000þÿÿE\000ü\001,À\000\000E\000Â\001,À\000\0007\004\2129OËFM\201»³WE\000\001Â&S\000\000?\001\224\016\n\233\a\002¬\031\001$\003\003½z\000\000\000\000E\000\001¦åí\000\000>\021Ö\177¬\031\001$\n\233\a\002\0005ÿ_\001\222)Ûdh\201\200\000\001\000\003\000\b\000\bÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p *ifp $2 = {if_softc = 0xffffff80007b1000, if_l2com = 0xfffffe000300ba40, if_vnet = 0x0, if_link = {tqe_next = 0xfffffe0003001000, tqe_prev = 0xfffffe0003001818}, if_xname = "bge1", '\0' , if_dname = 0xfffffe00028f07d8 "bge", if_dunit = 1, if_refcount = 1, if_addrhead = {tqh_first = 0xfffffe0003009800, tqh_last = 0xfffffe000591b4b8}, if_pcount = 0, if_carp = 0x0, if_bpf = 0xfffffe0005126900, if_index = 6, if_index_reserved = 0, if_vlantrunk = 0x0, if_flags = 34819, if_capabilities = 524443, if_capenable = 524443, if_linkmib = 0x0, if_linkmiblen = 0, if_data = { ifi_type = 6 '\006', ifi_physical = 0 '\0', ifi_addrlen = 6 '\006', ifi_hdrlen = 18 '\022', ifi_link_state = 2 '\002', ifi_spare_char1 = 0 '\0', ifi_spare_char2 = 0 '\0', ifi_datalen = 152 '\230', ifi_mtu = 1500, ifi_metric = 0, ifi_baudrate = 1000000000, ifi_ipackets = 1922972, ifi_ierrors = 0, ifi_opackets = 962786, ifi_oerrors = 0, ifi_collisions = 0, ifi_ibytes = 1150684321, ifi_obytes = 312161748, ifi_imcasts = 942443, ifi_omcasts = 0, ifi_iqdrops = 0, ifi_noproto = 0, ifi_hwassist = 3, ifi_epoch = 1, ifi_lastchange = {tv_sec = 1337714565, tv_usec = 347019}}, if_multiaddrs = {tqh_first = 0xfffffe0005915900, tqh_last = 0xfffffe0005a39100}, if_amcount = 0, if_output = 0xffffffff8073d805 , if_input = 0xffffffff8073cddb , if_start = 0xffffffff803c3087 , if_ioctl = 0xffffffff803c92ba , if_init = 0xffffffff803c9274 , if_resolvemulti = 0xffffffff8073c79d , if_qflush = 0xffffffff807355d2 , if_transmit = 0xffffffff8073549e , if_reassign = 0, if_home_vnet = 0x0, if_addr = 0xfffffe0003009800, if_llsoftc = 0x0, if_drv_flags = 64, if_snd = {ifq_head = 0x0, ifq_tail = 0x0, ifq_len = 0, ifq_maxlen = 511, ifq_drops = 0, ifq_mtx = {lock_object = { lo_name = 0xfffffe0003002028 "bge1", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006cf480}, mtx_lock = 4}, ifq_drv_head = 0x0, ifq_drv_tail = 0x0, ifq_drv_len = 0, ifq_drv_maxlen = 511, altq_type = 0, altq_flags = 1, altq_disc = 0x0, altq_ifp = 0xfffffe0003002000, altq_enqueue = 0, altq_dequeue = 0, altq_request = 0, altq_clfier = 0x0, altq_classify = 0, altq_tbr = 0x0, altq_cdnr = 0x0}, if_broadcastaddr = 0xffffffff80adb000 "ÿÿÿÿÿÿ", if_bridge = 0x0, if_label = 0x0, if_prefixhead = {tqh_first = 0x0, tqh_last = 0xfffffe0003002278}, if_afdata = {0x0, 0x0, 0xfffffe000581fa00, 0x0 , 0xfffffe0005814800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, if_afdata_initialized = 2, if_afdata_lock = { lock_object = {lo_name = 0xffffffff80ada29a "if_afdata", lo_flags = 69402624, lo_data = 0, lo_witness = 0xffffff80006cf400}, rw_lock = 1}, if_linktask = {ta_link = {stqe_next = 0x0}, ta_pending = 0, ta_priority = 0, ta_func = 0xffffffff80737a79 , ta_context = 0xfffffe0003002000}, if_addr_mtx = {lock_object = { lo_name = 0xffffffff80acc360 "if_addr_mtx", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006c8b80}, mtx_lock = 4}, if_clones = {le_next = 0x0, le_prev = 0x0}, if_groups = { tqh_first = 0xfffffe00050d3ae0, tqh_last = 0xfffffe00050d3ae8}, if_pf_kif = 0xfffffe0005889300, if_lagg = 0x0, if_description = 0x0, if_fib = 0, if_alloctype = 6 '\006', if_cspare = "\000\000", if_ispare = {0, 0, 0, 0}, if_pspare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}} (kgdb) #### kgdb.out_assert - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvT72SPOsGF+KA+MRAvxgAJ91uOe4RymMtaUOoZ7IK61/qHpoSQCZAbd0 /LVHK3BmvPKBUbd6e5rokUE= =9vPz -----END PGP SIGNATURE----- --3469798045-1716925155-1337802345=:21881-- From owner-freebsd-pf@FreeBSD.ORG Wed May 23 20:14:51 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 56981106564A for ; Wed, 23 May 2012 20:14:51 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id 220158FC15 for ; Wed, 23 May 2012 20:14:48 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4NKEg0q030854 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Wed, 23 May 2012 22:14:42 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4NKEgpk024784; Wed, 23 May 2012 22:14:42 +0200 (MEST) Date: Wed, 23 May 2012 22:14:42 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120523201442.GG29536@insomnia.benzedrine.cx> References: <201205231950.q4NJo4m1088701@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205231950.q4NJo4m1088701@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 20:14:51 -0000 On Wed, May 23, 2012 at 07:50:04PM +0000, Joerg Pulz wrote: > system was running for about a day with your patch with many users using > it. It panic'ed some minutes ago. > System configuration is still the same, no other patches, no changed > interface settings or removed/changed kernel options. > > Here is the kgdb(1) output with "m" and "ifp" listed. > I hope this helps to get closer to the source of the problem. > > Let me know if you need more output. Great, that should bring us closer to the cause! I'd say one of the pfil hooks is leaving the mbuf in the wrong byte order. You have ipfilter and ipfw compiled into the kernel, but are their modules loaded? I extended the patch to add more checks, in ipfilter and ipfw as well, if you can run this up to another panic, we might clearly identify the responsible hook. I'll study the trace in the meantime, maybe more can be deduced already :) Kind regards, Daniel Index: sys/sys/mbuf.h =================================================================== RCS file: /home/ncvs/src/sys/sys/mbuf.h,v retrieving revision 1.242.2.1 diff -u -r1.242.2.1 mbuf.h --- sys/sys/mbuf.h 23 Sep 2011 00:51:37 -0000 1.242.2.1 +++ sys/sys/mbuf.h 23 May 2012 06:50:14 -0000 @@ -824,6 +824,22 @@ /* Compatibility with 4.3. */ #define m_copy(m, o, l) m_copym((m), (o), (l), M_DONTWAIT) +#define ASSERT_NET_BYTE_ORDER(m) do { \ + struct ip *ip = mtod((m), struct ip *); \ + if (ip->ip_len != htons(ip->ip_len) && \ + ip->ip_len == (m)->m_pkthdr.len) \ + panic("%s:%d ASSERT_NET_BYTE_ORDER %d %d", __func__, \ + __LINE__, (int)ip->ip_len, (int)htons(ip->ip_len)); \ +} while(0) + +#define ASSERT_HOST_BYTE_ORDER(m) do { \ + struct ip *ip = mtod((m), struct ip *); \ + if (ip->ip_len != htons(ip->ip_len) && \ + ntohs(ip->ip_len) == (m)->m_pkthdr.len) \ + panic("%s:%d ASSERT_HOST_BYTE_ORDER %d %d", __func__, \ + __LINE__, (int)ip->ip_len, (int)htons(ip->ip_len)); \ +} while(0) + extern int max_datalen; /* MHLEN - max_hdr */ extern int max_hdr; /* Largest link + protocol header */ extern int max_linkhdr; /* Largest link-level header */ Index: sys/contrib/ipfilter/netinet/fil.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/ipfilter/netinet/fil.c,v retrieving revision 1.57.4.1 diff -u -r1.57.4.1 fil.c --- sys/contrib/ipfilter/netinet/fil.c 23 Sep 2011 00:51:37 -0000 1.57.4.1 +++ sys/contrib/ipfilter/netinet/fil.c 23 May 2012 13:49:39 -0000 @@ -2445,6 +2445,7 @@ fin->fin_qpi = qpi; # else /* MENTAT */ + ASSERT_HOST_BYTE_ORDER(*mp); m = *mp; # if defined(M_MCAST) @@ -2519,6 +2520,7 @@ #endif } + ASSERT_HOST_BYTE_ORDER(m); if (fr_makefrip(hlen, ip, fin) == -1) { pass = FR_BLOCK|FR_NOMATCH; goto finished; @@ -2784,6 +2786,8 @@ ip->ip_off = ntohs(ip->ip_off); } # endif + if (*mp != NULL) + ASSERT_HOST_BYTE_ORDER(*mp); return (FR_ISPASS(pass)) ? 0 : fin->fin_error; #else /* _KERNEL */ FR_VERBOSE(("fin_flx %#x pass %#x ", fin->fin_flx, pass)); @@ -2955,6 +2959,7 @@ #ifdef USE_INET6 if (IP_V(ip) == 4) { #endif + ASSERT_HOST_BYTE_ORDER(m); hlen = IP_HL(ip) << 2; slen = l3len - hlen; sum = htons((u_short)l4proto); Index: sys/contrib/ipfilter/netinet/ip_fil_freebsd.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/ipfilter/netinet/ip_fil_freebsd.c,v retrieving revision 1.20.4.1 diff -u -r1.20.4.1 ip_fil_freebsd.c --- sys/contrib/ipfilter/netinet/ip_fil_freebsd.c 23 Sep 2011 00:51:37 -0000 1.20.4.1 +++ sys/contrib/ipfilter/netinet/ip_fil_freebsd.c 23 May 2012 06:33:23 -0000 @@ -183,6 +183,7 @@ fr_check_wrapper(void *arg, struct mbuf **mp, struct ifnet *ifp, int dir) { struct ip *ip = mtod(*mp, struct ip *); + ASSERT_HOST_BYTE_ORDER(*mp); return fr_check(ip, ip->ip_hl << 2, ifp, (dir == PFIL_OUT), mp); } Index: sys/contrib/pf/net/pf.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/pf/net/pf.c,v retrieving revision 1.78.2.6 diff -u -r1.78.2.6 pf.c --- sys/contrib/pf/net/pf.c 29 Feb 2012 09:47:26 -0000 1.78.2.6 +++ sys/contrib/pf/net/pf.c 23 May 2012 09:22:10 -0000 @@ -2560,6 +2560,7 @@ case AF_INET: #ifdef __FreeBSD__ /* icmp_error() expects host byte ordering */ + ASSERT_NET_BYTE_ORDER(m0); ip = mtod(m0, struct ip *); NTOHS(ip->ip_len); NTOHS(ip->ip_off); @@ -5894,6 +5895,8 @@ (dir != PF_IN && dir != PF_OUT) || oifp == NULL) panic("pf_route: invalid parameters"); + ASSERT_NET_BYTE_ORDER(*m); + #ifdef __FreeBSD__ if (pd->pf_mtag->routed++ > 3) { #else @@ -5977,6 +5980,7 @@ if (oifp != ifp) { #ifdef __FreeBSD__ + ASSERT_NET_BYTE_ORDER(m0); PF_UNLOCK(); if (pf_test(PF_OUT, ifp, &m0, NULL, NULL) != PF_PASS) { PF_LOCK(); @@ -5998,6 +6002,7 @@ goto bad; } ip = mtod(m0, struct ip *); + ASSERT_NET_BYTE_ORDER(m0); } #ifdef __FreeBSD__ @@ -6008,6 +6013,7 @@ /* * XXX: in_delayed_cksum assumes HBO for ip->ip_len (at least) */ + ASSERT_NET_BYTE_ORDER(m0); NTOHS(ip->ip_len); NTOHS(ip->ip_off); /* XXX: needed? */ in_delayed_cksum(m0); @@ -6017,6 +6023,8 @@ } m0->m_pkthdr.csum_flags &= ifp->if_hwassist; + ASSERT_NET_BYTE_ORDER(m0); + if (ntohs(ip->ip_len) <= ifp->if_mtu || (ifp->if_hwassist & CSUM_FRAGMENT && ((ip->ip_off & htons(IP_DF)) == 0))) { @@ -6104,6 +6112,7 @@ if (r->rt != PF_DUPTO) { #ifdef __FreeBSD__ /* icmp_error() expects host byte ordering */ + ASSERT_NET_BYTE_ORDER(m0); NTOHS(ip->ip_len); NTOHS(ip->ip_off); PF_UNLOCK(); @@ -6124,6 +6133,7 @@ /* * XXX: is cheaper + less error prone than own function */ + ASSERT_NET_BYTE_ORDER(m0); NTOHS(ip->ip_len); NTOHS(ip->ip_off); error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); @@ -6672,6 +6682,8 @@ #endif /* DIAGNOSTIC */ #endif + ASSERT_NET_BYTE_ORDER(m); + if (m->m_pkthdr.len < (int)sizeof(*h)) { action = PF_DROP; REASON_SET(&reason, PFRES_SHORT); Index: sys/contrib/pf/net/pf_ioctl.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/pf/net/pf_ioctl.c,v retrieving revision 1.50.2.4 diff -u -r1.50.2.4 pf_ioctl.c --- sys/contrib/pf/net/pf_ioctl.c 29 Feb 2012 09:47:26 -0000 1.50.2.4 +++ sys/contrib/pf/net/pf_ioctl.c 22 May 2012 14:37:44 -0000 @@ -4121,6 +4121,7 @@ if ((*m)->m_pkthdr.len >= (int)sizeof(struct ip)) { /* if m_pkthdr.len is less than ip header, pf will handle. */ + ASSERT_HOST_BYTE_ORDER(*m); h = mtod(*m, struct ip *); HTONS(h->ip_len); HTONS(h->ip_off); @@ -4134,6 +4135,7 @@ } if (*m != NULL) { /* pf_test can change ip header location */ + ASSERT_NET_BYTE_ORDER(*m); h = mtod(*m, struct ip *); NTOHS(h->ip_len); NTOHS(h->ip_off); @@ -4163,6 +4165,7 @@ } if ((*m)->m_pkthdr.len >= (int)sizeof(*h)) { /* if m_pkthdr.len is less than ip header, pf will handle. */ + ASSERT_HOST_BYTE_ORDER(*m); h = mtod(*m, struct ip *); HTONS(h->ip_len); HTONS(h->ip_off); @@ -4176,6 +4179,7 @@ } if (*m != NULL) { /* pf_test can change ip header location */ + ASSERT_NET_BYTE_ORDER(*m); h = mtod(*m, struct ip *); NTOHS(h->ip_len); NTOHS(h->ip_off); Index: sys/contrib/pf/net/pf_norm.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/pf/net/pf_norm.c,v retrieving revision 1.21.2.2 diff -u -r1.21.2.2 pf_norm.c --- sys/contrib/pf/net/pf_norm.c 29 Feb 2012 09:47:26 -0000 1.21.2.2 +++ sys/contrib/pf/net/pf_norm.c 22 May 2012 14:41:02 -0000 @@ -1190,6 +1190,8 @@ if (hlen < (int)sizeof(struct ip)) goto drop; + ASSERT_NET_BYTE_ORDER(m); + if (hlen > ntohs(h->ip_len)) goto drop; Index: sys/net/if_bridge.c =================================================================== RCS file: /home/ncvs/src/sys/net/if_bridge.c,v retrieving revision 1.144.2.2 diff -u -r1.144.2.2 if_bridge.c --- sys/net/if_bridge.c 17 Mar 2012 12:11:53 -0000 1.144.2.2 +++ sys/net/if_bridge.c 22 May 2012 14:44:14 -0000 @@ -3142,6 +3142,7 @@ */ ip = mtod(*mp, struct ip *); + ASSERT_NET_BYTE_ORDER(*mp); ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); @@ -3195,6 +3196,7 @@ if (ip == NULL) goto bad; } + ASSERT_HOST_BYTE_ORDER(*mp); ip->ip_len = htons(ip->ip_len); ip->ip_off = htons(ip->ip_off); ip->ip_sum = 0; @@ -3332,6 +3334,7 @@ } /* Retrieve the packet length. */ + ASSERT_NET_BYTE_ORDER(m); len = ntohs(ip->ip_len); /* Index: sys/net/if_enc.c =================================================================== RCS file: /home/ncvs/src/sys/net/if_enc.c,v retrieving revision 1.17.2.1 diff -u -r1.17.2.1 if_enc.c --- sys/net/if_enc.c 23 Sep 2011 00:51:37 -0000 1.17.2.1 +++ sys/net/if_enc.c 22 May 2012 14:43:27 -0000 @@ -274,6 +274,7 @@ * before calling the firewall, swap fields the same as * IP does. here we assume the header is contiguous */ + ASSERT_NET_BYTE_ORDER(*mp); ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); @@ -284,6 +285,7 @@ break; /* restore byte ordering */ + ASSERT_HOST_BYTE_ORDER(*mp); ip = mtod(*mp, struct ip *); ip->ip_len = htons(ip->ip_len); ip->ip_off = htons(ip->ip_off); Index: sys/net/pfil.c =================================================================== RCS file: /home/ncvs/src/sys/net/pfil.c,v retrieving revision 1.19.2.1 diff -u -r1.19.2.1 pfil.c --- sys/net/pfil.c 23 Sep 2011 00:51:37 -0000 1.19.2.1 +++ sys/net/pfil.c 22 May 2012 14:49:24 -0000 @@ -46,6 +46,8 @@ #include #include +#include +#include static struct mtx pfil_global_lock; @@ -79,10 +81,12 @@ for (pfh = pfil_hook_get(dir, ph); pfh != NULL; pfh = TAILQ_NEXT(pfh, pfil_link)) { if (pfh->pfil_func != NULL) { + ASSERT_HOST_BYTE_ORDER(m); rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, inp); if (rv != 0 || m == NULL) break; + ASSERT_HOST_BYTE_ORDER(m); } } PFIL_RUNLOCK(ph, &rmpt); Index: sys/netgraph/ng_ipfw.c =================================================================== RCS file: /home/ncvs/src/sys/netgraph/ng_ipfw.c,v retrieving revision 1.21.2.1 diff -u -r1.21.2.1 ng_ipfw.c --- sys/netgraph/ng_ipfw.c 23 Sep 2011 00:51:37 -0000 1.21.2.1 +++ sys/netgraph/ng_ipfw.c 23 May 2012 13:57:52 -0000 @@ -268,6 +268,7 @@ switch (ip->ip_v) { #ifdef INET case IPVERSION: + ASSERT_NET_BYTE_ORDER(m); SET_HOST_IPLEN(ip); return (ip_output(m, NULL, NULL, IP_FORWARDING, NULL, NULL)); Index: sys/netinet/ip_divert.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_divert.c,v retrieving revision 1.173.2.1 diff -u -r1.173.2.1 ip_divert.c --- sys/netinet/ip_divert.c 23 Sep 2011 00:51:37 -0000 1.173.2.1 +++ sys/netinet/ip_divert.c 22 May 2012 14:27:15 -0000 @@ -207,6 +207,7 @@ (m = m_pullup(m, sizeof(struct ip))) == 0) return; ip = mtod(m, struct ip *); + ASSERT_NET_BYTE_ORDER(m); /* Delayed checksums are currently not compatible with divert. */ if (m->m_pkthdr.csum_flags & CSUM_DELAY_DATA) { @@ -396,6 +397,7 @@ /* Convert fields to host order for ip_output() */ ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); + ASSERT_HOST_BYTE_ORDER(m); break; #ifdef INET6 case IPV6_VERSION >> 4: Index: sys/netinet/ip_fastfwd.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_fastfwd.c,v retrieving revision 1.57.2.1 diff -u -r1.57.2.1 ip_fastfwd.c --- sys/netinet/ip_fastfwd.c 23 Sep 2011 00:51:37 -0000 1.57.2.1 +++ sys/netinet/ip_fastfwd.c 22 May 2012 14:46:00 -0000 @@ -179,6 +179,7 @@ M_ASSERTVALID(m); M_ASSERTPKTHDR(m); + ASSERT_NET_BYTE_ORDER(m); bzero(&ro, sizeof(ro)); @@ -343,6 +344,7 @@ /* * Convert to host representation */ + ASSERT_NET_BYTE_ORDER(m); ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); @@ -361,6 +363,7 @@ M_ASSERTVALID(m); M_ASSERTPKTHDR(m); + ASSERT_HOST_BYTE_ORDER(m); ip = mtod(m, struct ip *); /* m may have changed by pfil hook */ dest.s_addr = ip->ip_dst.s_addr; @@ -442,12 +445,14 @@ if (!PFIL_HOOKED(&V_inet_pfil_hook)) goto passout; + ASSERT_HOST_BYTE_ORDER(m); if (pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_OUT, NULL) || m == NULL) { goto drop; } M_ASSERTVALID(m); M_ASSERTPKTHDR(m); + ASSERT_HOST_BYTE_ORDER(m); ip = mtod(m, struct ip *); dest.s_addr = ip->ip_dst.s_addr; @@ -511,6 +516,7 @@ goto consumed; } + ASSERT_HOST_BYTE_ORDER(m); #ifndef ALTQ /* * Check if there is enough space in the interface queue Index: sys/netinet/ip_icmp.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_icmp.c,v retrieving revision 1.145.2.2 diff -u -r1.145.2.2 ip_icmp.c --- sys/netinet/ip_icmp.c 19 Mar 2012 20:49:16 -0000 1.145.2.2 +++ sys/netinet/ip_icmp.c 22 May 2012 14:31:17 -0000 @@ -185,6 +185,7 @@ unsigned icmplen, icmpelen, nlen; KASSERT((u_int)type <= ICMP_MAXTYPE, ("%s: illegal ICMP type", __func__)); + ASSERT_HOST_BYTE_ORDER(n); #ifdef ICMPPRINTFS if (icmpprintfs) printf("icmp_error(%p, %x, %d)\n", oip, type, code); @@ -336,6 +337,7 @@ void (*ctlfunc)(int, struct sockaddr *, void *); int fibnum; + ASSERT_HOST_BYTE_ORDER(m); /* * Locate icmp structure in mbuf, and check * that not corrupted and of at least minimum length. @@ -866,6 +868,7 @@ register int hlen; register struct icmp *icp; + ASSERT_HOST_BYTE_ORDER(m); hlen = ip->ip_hl << 2; m->m_data += hlen; m->m_len -= hlen; Index: sys/netinet/ip_input.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_input.c,v retrieving revision 1.393.2.3 diff -u -r1.393.2.3 ip_input.c --- sys/netinet/ip_input.c 19 Mar 2012 20:49:16 -0000 1.393.2.3 +++ sys/netinet/ip_input.c 22 May 2012 14:23:45 -0000 @@ -385,6 +385,7 @@ struct in_addr odst; /* original dst address */ M_ASSERTPKTHDR(m); + ASSERT_NET_BYTE_ORDER(m); if (m->m_flags & M_FASTFWD_OURS) { /* @@ -467,6 +468,7 @@ goto bad; } ip->ip_off = ntohs(ip->ip_off); + ASSERT_HOST_BYTE_ORDER(m); /* * Check that the amount of data in the buffers @@ -1371,6 +1373,7 @@ struct route ro; int error, type = 0, code = 0, mtu = 0; + ASSERT_HOST_BYTE_ORDER(m); if (m->m_flags & (M_BCAST|M_MCAST) || in_canforward(ip->ip_dst) == 0) { IPSTAT_INC(ips_cantforward); m_freem(m); Index: sys/netinet/ip_ipsec.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_ipsec.c,v retrieving revision 1.28.2.1 diff -u -r1.28.2.1 ip_ipsec.c --- sys/netinet/ip_ipsec.c 23 Sep 2011 00:51:37 -0000 1.28.2.1 +++ sys/netinet/ip_ipsec.c 22 May 2012 14:25:41 -0000 @@ -346,6 +346,7 @@ (*m)->m_pkthdr.csum_flags &= ~CSUM_SCTP; } #endif + ASSERT_HOST_BYTE_ORDER(*m); ip->ip_len = htons(ip->ip_len); ip->ip_off = htons(ip->ip_off); Index: sys/netinet/ip_mroute.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_mroute.c,v retrieving revision 1.161.2.2 diff -u -r1.161.2.2 ip_mroute.c --- sys/netinet/ip_mroute.c 28 Mar 2012 12:45:35 -0000 1.161.2.2 +++ sys/netinet/ip_mroute.c 22 May 2012 14:32:54 -0000 @@ -1496,6 +1496,7 @@ vifi_t vifi; int plen = ip->ip_len; + ASSERT_HOST_BYTE_ORDER(m); VIF_LOCK_ASSERT(); /* @@ -2375,6 +2376,8 @@ struct mbuf *mb_copy = NULL; int mtu; + ASSERT_HOST_BYTE_ORDER(m); + /* Take care of delayed checksums */ if (m->m_pkthdr.csum_flags & CSUM_DELAY_DATA) { in_delayed_cksum(m); Index: sys/netinet/ip_output.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ip_output.c,v retrieving revision 1.329.2.2 diff -u -r1.329.2.2 ip_output.c --- sys/netinet/ip_output.c 10 Nov 2011 20:28:30 -0000 1.329.2.2 +++ sys/netinet/ip_output.c 22 May 2012 14:47:14 -0000 @@ -133,6 +133,7 @@ int no_route_but_check_spd = 0; #endif M_ASSERTPKTHDR(m); + ASSERT_HOST_BYTE_ORDER(m); if (inp != NULL) { INP_LOCK_ASSERT(inp); @@ -434,6 +435,8 @@ } } + ASSERT_HOST_BYTE_ORDER(m); + /* * Verify that we have any chance at all of being able to queue the * packet or packet fragments, unless ALTQ is enabled on the given @@ -505,6 +508,7 @@ /* Run through list of hooks for output packets. */ odst.s_addr = ip->ip_dst.s_addr; + ASSERT_HOST_BYTE_ORDER(m); error = pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_OUT, inp); if (error != 0 || m == NULL) goto done; @@ -596,6 +600,7 @@ * If small enough for interface, or the interface will take * care of the fragmentation for us, we can just send directly. */ + ASSERT_HOST_BYTE_ORDER(m); if (ip->ip_len <= mtu || (m->m_pkthdr.csum_flags & ifp->if_hwassist & CSUM_TSO) != 0 || ((ip->ip_off & IP_DF) == 0 && (ifp->if_hwassist & CSUM_FRAGMENT))) { @@ -628,6 +633,7 @@ * to avoid confusing lower layers. */ m->m_flags &= ~(M_PROTOFLAGS); + ASSERT_NET_BYTE_ORDER(m); error = (*ifp->if_output)(ifp, m, (struct sockaddr *)dst, ro); goto done; @@ -716,6 +722,8 @@ if (len < 8) return EMSGSIZE; + ASSERT_HOST_BYTE_ORDER(m0); + /* * If the interface will not calculate checksums on * fragmented packets, then do it here. Index: sys/netinet/ipfw/ip_dn_io.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ipfw/ip_dn_io.c,v retrieving revision 1.14.2.1 diff -u -r1.14.2.1 ip_dn_io.c --- sys/netinet/ipfw/ip_dn_io.c 23 Sep 2011 00:51:37 -0000 1.14.2.1 +++ sys/netinet/ipfw/ip_dn_io.c 23 May 2012 06:26:56 -0000 @@ -650,6 +650,7 @@ tag->m_tag_id = 0; } + ASSERT_NET_BYTE_ORDER(m); switch (dst) { case DIR_OUT: SET_HOST_IPLEN(mtod(m, struct ip *)); Index: sys/netinet/ipfw/ip_fw2.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ipfw/ip_fw2.c,v retrieving revision 1.66.2.5 diff -u -r1.66.2.5 ip_fw2.c --- sys/netinet/ipfw/ip_fw2.c 23 Apr 2012 07:15:15 -0000 1.66.2.5 +++ sys/netinet/ipfw/ip_fw2.c 23 May 2012 06:26:04 -0000 @@ -942,6 +942,8 @@ if (m->m_flags & M_SKIP_FIREWALL || (! V_ipfw_vnet_ready)) return (IP_FW_PASS); /* accept */ + ASSERT_NET_BYTE_ORDER(m); + dst_ip.s_addr = 0; /* make sure it is initialized */ src_ip.s_addr = 0; /* make sure it is initialized */ pktlen = m->m_pkthdr.len; @@ -2411,6 +2413,7 @@ * ip_reass() expects len & off in host * byte order. */ + ASSERT_NET_BYTE_ORDER(m); SET_HOST_IPLEN(ip); args->m = m = ip_reass(m); @@ -2433,6 +2436,7 @@ ip->ip_sum = in_cksum(m, hlen); retval = IP_FW_REASS; set_match(args, f_pos, chain); + ASSERT_NET_BYTE_ORDER(m); } done = 1; /* exit outer loop */ break; Index: sys/netinet/ipfw/ip_fw_pfil.c =================================================================== RCS file: /home/ncvs/src/sys/netinet/ipfw/ip_fw_pfil.c,v retrieving revision 1.24.2.3 diff -u -r1.24.2.3 ip_fw_pfil.c --- sys/netinet/ipfw/ip_fw_pfil.c 6 Nov 2011 17:31:57 -0000 1.24.2.3 +++ sys/netinet/ipfw/ip_fw_pfil.c 23 May 2012 13:30:55 -0000 @@ -110,6 +110,7 @@ int ipfw; int ret; + ASSERT_HOST_BYTE_ORDER(*m0); /* all the processing now uses ip_len in net format */ if (mtod(*m0, struct ip *)->ip_v == 4) SET_NET_IPLEN(mtod(*m0, struct ip *)); @@ -119,6 +120,7 @@ bzero(&args, sizeof(args)); again: + ASSERT_NET_BYTE_ORDER(*m0); /* * extract and remove the tag if present. If we are left * with onepass, optimize the outgoing path. @@ -130,6 +132,7 @@ if (args.rule.info & IPFW_ONEPASS) { if (mtod(*m0, struct ip *)->ip_v == 4) SET_HOST_IPLEN(mtod(*m0, struct ip *)); + ASSERT_HOST_BYTE_ORDER(*m0); return (0); } } @@ -273,8 +276,10 @@ FREE_PKT(*m0); *m0 = NULL; } - if (*m0 && mtod(*m0, struct ip *)->ip_v == 4) + if (*m0 && mtod(*m0, struct ip *)->ip_v == 4) { SET_HOST_IPLEN(mtod(*m0, struct ip *)); + ASSERT_HOST_BYTE_ORDER(*m0); + } return ret; } @@ -292,6 +297,7 @@ struct ip *ip = mtod(*m0, struct ip *); struct m_tag *tag; + ASSERT_NET_BYTE_ORDER(*m0); /* Cloning needed for tee? */ if (tee == 0) { clone = *m0; /* use the original mbuf */ Index: sys/netipsec/ipsec_output.c =================================================================== RCS file: /home/ncvs/src/sys/netipsec/ipsec_output.c,v retrieving revision 1.33.2.2 diff -u -r1.33.2.2 ipsec_output.c --- sys/netipsec/ipsec_output.c 29 Feb 2012 09:47:26 -0000 1.33.2.2 +++ sys/netipsec/ipsec_output.c 23 May 2012 14:03:44 -0000 @@ -205,6 +205,7 @@ ip = mtod(m, struct ip *); ip->ip_len = ntohs(ip->ip_len); ip->ip_off = ntohs(ip->ip_off); + ASSERT_HOST_BYTE_ORDER(m); #ifdef IPSEC_NAT_T /* Index: sys/netipsec/xform_ah.c =================================================================== RCS file: /home/ncvs/src/sys/netipsec/xform_ah.c,v retrieving revision 1.28.2.1 diff -u -r1.28.2.1 xform_ah.c --- sys/netipsec/xform_ah.c 23 Sep 2011 00:51:37 -0000 1.28.2.1 +++ sys/netipsec/xform_ah.c 23 May 2012 14:05:17 -0000 @@ -322,6 +322,7 @@ else ip->ip_off = 0; } + ASSERT_NET_BYTE_ORDER(m); ptr = mtod(m, unsigned char *) + sizeof(struct ip); From owner-freebsd-pf@FreeBSD.ORG Wed May 23 20:22:04 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 27DB71065672 for ; Wed, 23 May 2012 20:22:04 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id 622708FC08 for ; Wed, 23 May 2012 20:22:02 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4NKM2pq031911 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Wed, 23 May 2012 22:22:03 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4NKM2Lw012795; Wed, 23 May 2012 22:22:02 +0200 (MEST) Date: Wed, 23 May 2012 22:22:02 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120523202202.GH29536@insomnia.benzedrine.cx> References: <201205231950.q4NJo4m1088701@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205231950.q4NJo4m1088701@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 20:22:04 -0000 On Wed, May 23, 2012 at 07:50:04PM +0000, Joerg Pulz wrote: > Let me know if you need more output. Oh, we can identify the pfil hook by printing *pfh, pfh->pfil_func and comparing the address to that of pf_check_out, fr_check_wrapper, and the one for ipfw, right? Daniel From owner-freebsd-pf@FreeBSD.ORG Wed May 23 22:08:19 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 11BD7106566B; Wed, 23 May 2012 22:08:19 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 607338FC08; Wed, 23 May 2012 22:08:18 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4NM72ab000246; Thu, 24 May 2012 00:07:02 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4NM6v87000242 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 24 May 2012 00:07:00 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Thu, 24 May 2012 00:06:54 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120523202202.GH29536@insomnia.benzedrine.cx> Message-ID: References: <201205231950.q4NJo4m1088701@freefall.freebsd.org> <20120523202202.GH29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3469798045-489758976-1337810639=:24195" Content-ID: X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Thu, 24 May 2012 00:07:00 +0200 (CEST) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 22:08:19 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-489758976-1337810639=:24195 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-15; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: On Wed, 23 May 2012, Daniel Hartmeier wrote: > On Wed, May 23, 2012 at 07:50:04PM +0000, Joerg Pulz wrote: > >> Let me know if you need more output. > > Oh, we can identify the pfil hook by printing *pfh, pfh->pfil_func and > comparing the address to that of pf_check_out, fr_check_wrapper, and the > one for ipfw, right? Danniel, here is what i could get out. I was unable to print *pfh and pfh->pfil_func, but i printed the other two and *ph, maybe this helps. Joerg #### kgdb.out_assert2 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: ASSERT_HOST_BYTE_ORDER cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 pfil_run_hooks() at pfil_run_hooks+0x159 ip_output() at ip_output+0x6de ip_forward() at ip_forward+0x19e ip_input() at ip_input+0x670 swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 585 out of 4077 MB:..3%..11%..22%..31%..41%..52%..61%..72%..82%..91% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff8074b325 in pfil_run_hooks (ph=0xfffffe000581f880, mp=0xffffff8000241978, ifp=0xfffffe0003002000, dir=2, inp=0x0) at /usr/src/sys/net/pfil.c:89 89 ASSERT_HOST_BYTE_ORDER(m); (kgdb) list 84 ASSERT_HOST_BYTE_ORDER(m); 85 rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, 86 inp); 87 if (rv != 0 || m == NULL) 88 break; 89 ASSERT_HOST_BYTE_ORDER(m); 90 } 91 } 92 PFIL_RUNLOCK(ph, &rmpt); 93 *mp = m; (kgdb) p *pfh (kgdb) p pfh->pfil_func (kgdb) p pf_check_out $1 = {int (void *, struct mbuf **, struct ifnet *, int, struct inpcb *)} 0xffffffff8032d17a (kgdb) p fr_check_wrapper $2 = {int (void *, struct mbuf **, struct ifnet *, int)} 0xffffffff802fae2d (kgdb) p *ph $3 = {ph_in = {tqh_first = 0xfffffe0003007b40, tqh_last = 0xfffffe000581fb00}, ph_out = {tqh_first = 0xffffffff80779733, tqh_last = 0x0}, ph_type = 92404512, ph_nhooks = -512, ph_lock = {lock_object = { lo_name = 0xfffffe0003007ae0 " ø\201\005", lo_flags = 2155321139, lo_data = 4294967295, lo_witness = 0x0}, rm_writecpus = {__bits = { -2198926812672}}, rm_activeReaders = {lh_first = 0xfffffe0005bf9b00}, _rm_lock = {_rm_lock_mtx = {lock_object = { lo_name = 0x1
, lo_flags = 0, lo_data = 0, lo_witness = 0xfffffe000589f800}, mtx_lock = 18446741874779218176}, _rm_lock_sx = {lock_object = { lo_name = 0x1
, lo_flags = 0, lo_data = 0, lo_witness = 0xfffffe000589f800}, sx_lock = 18446741874779218176}}}, ph_un = {phu_val = 0, phu_ptr = 0x0}, ph_list = {le_next = 0x0, le_prev = 0xfffffe000581f800}} (kgdb) #### kgdb.out_assert2 - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvV+BSPOsGF+KA+MRAudvAJ4kQQl4isOAkmVCvzcj1ipGEagbwACgkhhO Ib9Dfbm6bUJcUHS6yBcbrQU= =FnJL -----END PGP SIGNATURE----- --3469798045-489758976-1337810639=:24195-- From owner-freebsd-pf@FreeBSD.ORG Wed May 23 22:10:04 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9A41106566B for ; Wed, 23 May 2012 22:10:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A91A88FC0C for ; Wed, 23 May 2012 22:10:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4NMA48d058453 for ; Wed, 23 May 2012 22:10:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4NMA4PF058452; Wed, 23 May 2012 22:10:04 GMT (envelope-from gnats) Date: Wed, 23 May 2012 22:10:04 GMT Message-Id: <201205232210.q4NMA4PF058452@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 22:10:04 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Thu, 24 May 2012 00:06:54 +0200 (CEST) This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-489758976-1337810639=:24195 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-15; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: On Wed, 23 May 2012, Daniel Hartmeier wrote: > On Wed, May 23, 2012 at 07:50:04PM +0000, Joerg Pulz wrote: > >> Let me know if you need more output. > > Oh, we can identify the pfil hook by printing *pfh, pfh->pfil_func and > comparing the address to that of pf_check_out, fr_check_wrapper, and the > one for ipfw, right? Danniel, here is what i could get out. I was unable to print *pfh and pfh->pfil_func, but i printed the other two and *ph, maybe this helps. Joerg #### kgdb.out_assert2 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: ASSERT_HOST_BYTE_ORDER cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 pfil_run_hooks() at pfil_run_hooks+0x159 ip_output() at ip_output+0x6de ip_forward() at ip_forward+0x19e ip_input() at ip_input+0x670 swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 585 out of 4077 MB:..3%..11%..22%..31%..41%..52%..61%..72%..82%..91% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff8074b325 in pfil_run_hooks (ph=0xfffffe000581f880, mp=0xffffff8000241978, ifp=0xfffffe0003002000, dir=2, inp=0x0) at /usr/src/sys/net/pfil.c:89 89 ASSERT_HOST_BYTE_ORDER(m); (kgdb) list 84 ASSERT_HOST_BYTE_ORDER(m); 85 rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, 86 inp); 87 if (rv != 0 || m == NULL) 88 break; 89 ASSERT_HOST_BYTE_ORDER(m); 90 } 91 } 92 PFIL_RUNLOCK(ph, &rmpt); 93 *mp = m; (kgdb) p *pfh (kgdb) p pfh->pfil_func (kgdb) p pf_check_out $1 = {int (void *, struct mbuf **, struct ifnet *, int, struct inpcb *)} 0xffffffff8032d17a (kgdb) p fr_check_wrapper $2 = {int (void *, struct mbuf **, struct ifnet *, int)} 0xffffffff802fae2d (kgdb) p *ph $3 = {ph_in = {tqh_first = 0xfffffe0003007b40, tqh_last = 0xfffffe000581fb00}, ph_out = {tqh_first = 0xffffffff80779733, tqh_last = 0x0}, ph_type = 92404512, ph_nhooks = -512, ph_lock = {lock_object = { lo_name = 0xfffffe0003007ae0 " ø\201\005", lo_flags = 2155321139, lo_data = 4294967295, lo_witness = 0x0}, rm_writecpus = {__bits = { -2198926812672}}, rm_activeReaders = {lh_first = 0xfffffe0005bf9b00}, _rm_lock = {_rm_lock_mtx = {lock_object = { lo_name = 0x1
, lo_flags = 0, lo_data = 0, lo_witness = 0xfffffe000589f800}, mtx_lock = 18446741874779218176}, _rm_lock_sx = {lock_object = { lo_name = 0x1
, lo_flags = 0, lo_data = 0, lo_witness = 0xfffffe000589f800}, sx_lock = 18446741874779218176}}}, ph_un = {phu_val = 0, phu_ptr = 0x0}, ph_list = {le_next = 0x0, le_prev = 0xfffffe000581f800}} (kgdb) #### kgdb.out_assert2 - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvV+BSPOsGF+KA+MRAudvAJ4kQQl4isOAkmVCvzcj1ipGEagbwACgkhhO Ib9Dfbm6bUJcUHS6yBcbrQU= =FnJL -----END PGP SIGNATURE----- --3469798045-489758976-1337810639=:24195-- From owner-freebsd-pf@FreeBSD.ORG Thu May 24 02:42:02 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 273F9106566B; Thu, 24 May 2012 02:42:02 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id EDB028FC0A; Thu, 24 May 2012 02:42:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4O2g1rZ092596; Thu, 24 May 2012 02:42:01 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4O2g1wo092536; Thu, 24 May 2012 02:42:01 GMT (envelope-from linimon) Date: Thu, 24 May 2012 02:42:01 GMT Message-Id: <201205240242.q4O2g1wo092536@freefall.freebsd.org> To: Joerg.Pulz@frm2.tum.de, linimon@FreeBSD.org, gnats-admin@FreeBSD.org, freebsd-pf@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/168227: Re: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 02:42:02 -0000 Old Synopsis: Re: panic when using pf and route-to (maybe: bad fragment New Synopsis: Re: [pf] panic when using pf and route-to (maybe: bad fragment handling?) State-Changed-From-To: open->closed State-Changed-By: linimon State-Changed-When: Thu May 24 02:40:43 UTC 2012 State-Changed-Why: Misfiled followup to kern/168190; content migrated. Responsible-Changed-From-To: gnats-admin->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Thu May 24 02:40:43 UTC 2012 Responsible-Changed-Why: http://www.freebsd.org/cgi/query-pr.cgi?pr=168227 From owner-freebsd-pf@FreeBSD.ORG Thu May 24 06:36:26 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9BF91106566B for ; Thu, 24 May 2012 06:36:26 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id E4D068FC12 for ; Thu, 24 May 2012 06:36:24 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4O6SbMF026417 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Thu, 24 May 2012 08:28:37 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4O6SZJi024018; Thu, 24 May 2012 08:28:35 +0200 (MEST) Date: Thu, 24 May 2012 08:28:35 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120524062835.GI29536@insomnia.benzedrine.cx> References: <201205232210.q4NMA4PF058452@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205232210.q4NMA4PF058452@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 06:36:26 -0000 On Wed, May 23, 2012 at 10:10:04PM +0000, Joerg Pulz wrote: > here is what i could get out. > I was unable to print *pfh and pfh->pfil_func, but i printed the other > two and *ph, maybe this helps. That looks corrupted: ph_type = 92404512, ph_nhooks = -512 makes no sense to me. Can you go up one stack frame (to #11), which should be ip_output() 509 /* Run through list of hooks for output packets. */ 510 odst.s_addr = ip->ip_dst.s_addr; 511 ASSERT_HOST_BYTE_ORDER(m); 512 error = pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_OUT, inp); 513 if (error != 0 || m == NULL) 514 goto done; and there print V_inet_pfil_hook? Kind regards, Daniel From owner-freebsd-pf@FreeBSD.ORG Thu May 24 09:00:17 2012 Return-Path: Delivered-To: freebsd-pf@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD62B1065670; Thu, 24 May 2012 09:00:17 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 13A508FC1B; Thu, 24 May 2012 09:00:13 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4O8wwf3015357; Thu, 24 May 2012 10:58:58 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4O8wvVs015353 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 24 May 2012 10:58:57 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Thu, 24 May 2012 10:58:54 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120524062835.GI29536@insomnia.benzedrine.cx> Message-ID: References: <201205232210.q4NMA4PF058452@freefall.freebsd.org> <20120524062835.GI29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3469798045-664628730-1337849937=:89783" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Thu, 24 May 2012 10:58:57 +0200 (CEST) Cc: bug-followup@FreeBSD.org, freebsd-pf@FreeBSD.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 09:00:17 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-664628730-1337849937=:89783 Content-Type: TEXT/PLAIN; format=flowed; charset=ISO-8859-15 Content-Transfer-Encoding: 8BIT -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 24 May 2012, Daniel Hartmeier wrote: > On Wed, May 23, 2012 at 10:10:04PM +0000, Joerg Pulz wrote: > >> here is what i could get out. >> I was unable to print *pfh and pfh->pfil_func, but i printed the other >> two and *ph, maybe this helps. > > That looks corrupted: ph_type = 92404512, ph_nhooks = -512 makes no > sense to me. > > Can you go up one stack frame (to #11), which should be ip_output() > > 509 /* Run through list of hooks for output packets. */ > 510 odst.s_addr = ip->ip_dst.s_addr; > 511 ASSERT_HOST_BYTE_ORDER(m); > 512 error = pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_OUT, inp); > 513 if (error != 0 || m == NULL) > 514 goto done; > > and there print V_inet_pfil_hook? Daniel, i can't print V_inet_pfil_hook: No symbol "V_inet_pfil_hook" in current context. Meanwhile, the system was running over night with your second patch and panic'ed in the morning, about 3 hours ago. I was able to print *ifp, *pfh, pfh->pfil_func, pf_check_out, fr_check_wrapper and ipfw_check_hook. I couldn't print: *ph: Variable "ph" is not available. *m0: Cannot access memory at address 0xb000b0 Below is the output. Kind regards Joerg #### kgdb.out_assert_new GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: ipfw_check_hook:281 ASSERT_HOST_BYTE_ORDER 45056 176 cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 ipfw_check_hook() at ipfw_check_hook+0x511 pfil_run_hooks() at pfil_run_hooks+0xf1 ip_output() at ip_output+0x6de ip_forward() at ip_forward+0x19e ip_input() at ip_input+0x680 swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 559 out of 4077 MB:..3%..12%..21%..32%..41%..52%..61%..72%..81%..92% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff8077a144 in ipfw_check_hook (arg=) at /usr/src/sys/netinet/ipfw/ip_fw_pfil.c:281 281 ASSERT_HOST_BYTE_ORDER(*m0); (kgdb) list 276 FREE_PKT(*m0); 277 *m0 = NULL; 278 } 279 if (*m0 && mtod(*m0, struct ip *)->ip_v == 4) { 280 SET_HOST_IPLEN(mtod(*m0, struct ip *)); 281 ASSERT_HOST_BYTE_ORDER(*m0); 282 } 283 return ret; 284 } 285 (kgdb) p *ifp $1 = {if_softc = 0xffffff80007a9000, if_l2com = 0xfffffe000300b200, if_vnet = 0x0, if_link = {tqe_next = 0xfffffe0003002000, tqe_prev = 0xfffffe0003003818}, if_xname = "bge0", '\0' , if_dname = 0xfffffe00028f07d8 "bge", if_dunit = 0, if_refcount = 1, if_addrhead = {tqh_first = 0xfffffe000300a000, tqh_last = 0xfffffe000591a0b8}, if_pcount = 0, if_carp = 0x0, if_bpf = 0xfffffe00050d4680, if_index = 5, if_index_reserved = 0, if_vlantrunk = 0x0, if_flags = 34819, if_capabilities = 524443, if_capenable = 524443, if_linkmib = 0x0, if_linkmiblen = 0, if_data = { ifi_type = 6 '\006', ifi_physical = 0 '\0', ifi_addrlen = 6 '\006', ifi_hdrlen = 18 '\022', ifi_link_state = 2 '\002', ifi_spare_char1 = 0 '\0', ifi_spare_char2 = 0 '\0', ifi_datalen = 152 '\230', ifi_mtu = 1500, ifi_metric = 0, ifi_baudrate = 1000000000, ifi_ipackets = 221591, ifi_ierrors = 0, ifi_opackets = 3800, ifi_oerrors = 0, ifi_collisions = 0, ifi_ibytes = 18564820, ifi_obytes = 2351574, ifi_imcasts = 205582, ifi_omcasts = 0, ifi_iqdrops = 0, ifi_noproto = 0, ifi_hwassist = 3, ifi_epoch = 1, ifi_lastchange = {tv_sec = 1337811753, tv_usec = 642476}}, if_multiaddrs = {tqh_first = 0xfffffe0005915300, tqh_last = 0xfffffe00058d10c0}, if_amcount = 0, if_output = 0xffffffff8073da85 , if_input = 0xffffffff8073d05b , if_start = 0xffffffff803c32f7 , if_ioctl = 0xffffffff803c952a , if_init = 0xffffffff803c94e4 , if_resolvemulti = 0xffffffff8073ca1d , if_qflush = 0xffffffff80735842 , if_transmit = 0xffffffff8073570e , if_reassign = 0, if_home_vnet = 0x0, if_addr = 0xfffffe000300a000, if_llsoftc = 0x0, if_drv_flags = 64, if_snd = {ifq_head = 0x0, ifq_tail = 0x0, ifq_len = 0, ifq_maxlen = 511, ifq_drops = 0, ifq_mtx = {lock_object = { lo_name = 0xfffffe0003001828 "bge0", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006cf480}, mtx_lock = 4}, ifq_drv_head = 0x0, ifq_drv_tail = 0x0, ifq_drv_len = 0, ifq_drv_maxlen = 511, altq_type = 0, altq_flags = 1, altq_disc = 0x0, altq_ifp = 0xfffffe0003001800, altq_enqueue = 0, altq_dequeue = 0, altq_request = 0, altq_clfier = 0x0, altq_classify = 0, altq_tbr = 0x0, altq_cdnr = 0x0}, if_broadcastaddr = 0xffffffff80adb860 "ÿÿÿÿÿÿ", if_bridge = 0x0, if_label = 0x0, if_prefixhead = {tqh_first = 0x0, tqh_last = 0xfffffe0003001a78}, if_afdata = {0x0, 0x0, 0xfffffe0005821a20, 0x0 , 0xfffffe00058168c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, if_afdata_initialized = 2, if_afdata_lock = { lock_object = {lo_name = 0xffffffff80adaafa "if_afdata", lo_flags = 69402624, lo_data = 0, lo_witness = 0xffffff80006cf400}, rw_lock = 1}, if_linktask = {ta_link = {stqe_next = 0x0}, ta_pending = 0, ta_priority = 0, ta_func = 0xffffffff80737ce9 , ta_context = 0xfffffe0003001800}, if_addr_mtx = {lock_object = { lo_name = 0xffffffff80accbc0 "if_addr_mtx", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006c8b80}, mtx_lock = 4}, if_clones = {le_next = 0x0, le_prev = 0x0}, if_groups = { tqh_first = 0xfffffe0003007b20, tqh_last = 0xfffffe0003007b28}, if_pf_kif = 0xfffffe000588b400, if_lagg = 0x0, if_description = 0x0, if_fib = 0, if_alloctype = 6 '\006', if_cspare = "\000\000", if_ispare = {0, 0, 0, 0}, if_pspare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}} (kgdb) up #11 0xffffffff8074b53d in pfil_run_hooks (ph=) at /usr/src/sys/net/pfil.c:85 85 rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, (kgdb) list 80 KASSERT(ph->ph_nhooks >= 0, ("Pfil hook count dropped < 0")); 81 for (pfh = pfil_hook_get(dir, ph); pfh != NULL; 82 pfh = TAILQ_NEXT(pfh, pfil_link)) { 83 if (pfh->pfil_func != NULL) { 84 ASSERT_HOST_BYTE_ORDER(m); 85 rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, 86 inp); 87 if (rv != 0 || m == NULL) 88 break; 89 ASSERT_HOST_BYTE_ORDER(m); (kgdb) p *pfh $2 = {pfil_link = {tqe_next = 0xfffffe00058c5980, tqe_prev = 0xfffffe0005821b00}, pfil_func = 0xffffffff80779c33 , pfil_arg = 0x0} (kgdb) p pfh->pfil_func $3 = (int (*)(void *, struct mbuf **, struct ifnet *, int, struct inpcb *)) 0xffffffff80779c33 (kgdb) p pf_check_out $4 = {int (void *, struct mbuf **, struct ifnet *, int, struct inpcb *)} 0xffffffff8032d39a (kgdb) p fr_check_wrapper $5 = {int (void *, struct mbuf **, struct ifnet *, int)} 0xffffffff802fc303 (kgdb) p ipfw_check_hook $6 = {int (void *, struct mbuf **, struct ifnet *, int, struct inpcb *)} 0xffffffff80779c33 (kgdb) #### kgdb.out_assert_new - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvfhRSPOsGF+KA+MRAlM/AKClrSdzDyqSgechCL/RKRtj6KHpVQCfQtCL PQk+XB5xpajaVmaGba7wD7s= =J22z -----END PGP SIGNATURE----- --3469798045-664628730-1337849937=:89783-- From owner-freebsd-pf@FreeBSD.ORG Thu May 24 09:10:04 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A1B3D1065670 for ; Thu, 24 May 2012 09:10:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 89CDD8FC14 for ; Thu, 24 May 2012 09:10:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4O9A4p1044628 for ; Thu, 24 May 2012 09:10:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4O9A4rt044627; Thu, 24 May 2012 09:10:04 GMT (envelope-from gnats) Date: Thu, 24 May 2012 09:10:04 GMT Message-Id: <201205240910.q4O9A4rt044627@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 09:10:04 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: bug-followup@FreeBSD.org, freebsd-pf@FreeBSD.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Thu, 24 May 2012 10:58:54 +0200 (CEST) This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-664628730-1337849937=:89783 Content-Type: TEXT/PLAIN; format=flowed; charset=ISO-8859-15 Content-Transfer-Encoding: 8BIT -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 24 May 2012, Daniel Hartmeier wrote: > On Wed, May 23, 2012 at 10:10:04PM +0000, Joerg Pulz wrote: > >> here is what i could get out. >> I was unable to print *pfh and pfh->pfil_func, but i printed the other >> two and *ph, maybe this helps. > > That looks corrupted: ph_type = 92404512, ph_nhooks = -512 makes no > sense to me. > > Can you go up one stack frame (to #11), which should be ip_output() > > 509 /* Run through list of hooks for output packets. */ > 510 odst.s_addr = ip->ip_dst.s_addr; > 511 ASSERT_HOST_BYTE_ORDER(m); > 512 error = pfil_run_hooks(&V_inet_pfil_hook, &m, ifp, PFIL_OUT, inp); > 513 if (error != 0 || m == NULL) > 514 goto done; > > and there print V_inet_pfil_hook? Daniel, i can't print V_inet_pfil_hook: No symbol "V_inet_pfil_hook" in current context. Meanwhile, the system was running over night with your second patch and panic'ed in the morning, about 3 hours ago. I was able to print *ifp, *pfh, pfh->pfil_func, pf_check_out, fr_check_wrapper and ipfw_check_hook. I couldn't print: *ph: Variable "ph" is not available. *m0: Cannot access memory at address 0xb000b0 Below is the output. Kind regards Joerg #### kgdb.out_assert_new GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: ipfw_check_hook:281 ASSERT_HOST_BYTE_ORDER 45056 176 cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 ipfw_check_hook() at ipfw_check_hook+0x511 pfil_run_hooks() at pfil_run_hooks+0xf1 ip_output() at ip_output+0x6de ip_forward() at ip_forward+0x19e ip_input() at ip_input+0x680 swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 559 out of 4077 MB:..3%..12%..21%..32%..41%..52%..61%..72%..81%..92% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff8077a144 in ipfw_check_hook (arg=) at /usr/src/sys/netinet/ipfw/ip_fw_pfil.c:281 281 ASSERT_HOST_BYTE_ORDER(*m0); (kgdb) list 276 FREE_PKT(*m0); 277 *m0 = NULL; 278 } 279 if (*m0 && mtod(*m0, struct ip *)->ip_v == 4) { 280 SET_HOST_IPLEN(mtod(*m0, struct ip *)); 281 ASSERT_HOST_BYTE_ORDER(*m0); 282 } 283 return ret; 284 } 285 (kgdb) p *ifp $1 = {if_softc = 0xffffff80007a9000, if_l2com = 0xfffffe000300b200, if_vnet = 0x0, if_link = {tqe_next = 0xfffffe0003002000, tqe_prev = 0xfffffe0003003818}, if_xname = "bge0", '\0' , if_dname = 0xfffffe00028f07d8 "bge", if_dunit = 0, if_refcount = 1, if_addrhead = {tqh_first = 0xfffffe000300a000, tqh_last = 0xfffffe000591a0b8}, if_pcount = 0, if_carp = 0x0, if_bpf = 0xfffffe00050d4680, if_index = 5, if_index_reserved = 0, if_vlantrunk = 0x0, if_flags = 34819, if_capabilities = 524443, if_capenable = 524443, if_linkmib = 0x0, if_linkmiblen = 0, if_data = { ifi_type = 6 '\006', ifi_physical = 0 '\0', ifi_addrlen = 6 '\006', ifi_hdrlen = 18 '\022', ifi_link_state = 2 '\002', ifi_spare_char1 = 0 '\0', ifi_spare_char2 = 0 '\0', ifi_datalen = 152 '\230', ifi_mtu = 1500, ifi_metric = 0, ifi_baudrate = 1000000000, ifi_ipackets = 221591, ifi_ierrors = 0, ifi_opackets = 3800, ifi_oerrors = 0, ifi_collisions = 0, ifi_ibytes = 18564820, ifi_obytes = 2351574, ifi_imcasts = 205582, ifi_omcasts = 0, ifi_iqdrops = 0, ifi_noproto = 0, ifi_hwassist = 3, ifi_epoch = 1, ifi_lastchange = {tv_sec = 1337811753, tv_usec = 642476}}, if_multiaddrs = {tqh_first = 0xfffffe0005915300, tqh_last = 0xfffffe00058d10c0}, if_amcount = 0, if_output = 0xffffffff8073da85 , if_input = 0xffffffff8073d05b , if_start = 0xffffffff803c32f7 , if_ioctl = 0xffffffff803c952a , if_init = 0xffffffff803c94e4 , if_resolvemulti = 0xffffffff8073ca1d , if_qflush = 0xffffffff80735842 , if_transmit = 0xffffffff8073570e , if_reassign = 0, if_home_vnet = 0x0, if_addr = 0xfffffe000300a000, if_llsoftc = 0x0, if_drv_flags = 64, if_snd = {ifq_head = 0x0, ifq_tail = 0x0, ifq_len = 0, ifq_maxlen = 511, ifq_drops = 0, ifq_mtx = {lock_object = { lo_name = 0xfffffe0003001828 "bge0", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006cf480}, mtx_lock = 4}, ifq_drv_head = 0x0, ifq_drv_tail = 0x0, ifq_drv_len = 0, ifq_drv_maxlen = 511, altq_type = 0, altq_flags = 1, altq_disc = 0x0, altq_ifp = 0xfffffe0003001800, altq_enqueue = 0, altq_dequeue = 0, altq_request = 0, altq_clfier = 0x0, altq_classify = 0, altq_tbr = 0x0, altq_cdnr = 0x0}, if_broadcastaddr = 0xffffffff80adb860 "ÿÿÿÿÿÿ", if_bridge = 0x0, if_label = 0x0, if_prefixhead = {tqh_first = 0x0, tqh_last = 0xfffffe0003001a78}, if_afdata = {0x0, 0x0, 0xfffffe0005821a20, 0x0 , 0xfffffe00058168c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, if_afdata_initialized = 2, if_afdata_lock = { lock_object = {lo_name = 0xffffffff80adaafa "if_afdata", lo_flags = 69402624, lo_data = 0, lo_witness = 0xffffff80006cf400}, rw_lock = 1}, if_linktask = {ta_link = {stqe_next = 0x0}, ta_pending = 0, ta_priority = 0, ta_func = 0xffffffff80737ce9 , ta_context = 0xfffffe0003001800}, if_addr_mtx = {lock_object = { lo_name = 0xffffffff80accbc0 "if_addr_mtx", lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff80006c8b80}, mtx_lock = 4}, if_clones = {le_next = 0x0, le_prev = 0x0}, if_groups = { tqh_first = 0xfffffe0003007b20, tqh_last = 0xfffffe0003007b28}, if_pf_kif = 0xfffffe000588b400, if_lagg = 0x0, if_description = 0x0, if_fib = 0, if_alloctype = 6 '\006', if_cspare = "\000\000", if_ispare = {0, 0, 0, 0}, if_pspare = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}} (kgdb) up #11 0xffffffff8074b53d in pfil_run_hooks (ph=) at /usr/src/sys/net/pfil.c:85 85 rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, (kgdb) list 80 KASSERT(ph->ph_nhooks >= 0, ("Pfil hook count dropped < 0")); 81 for (pfh = pfil_hook_get(dir, ph); pfh != NULL; 82 pfh = TAILQ_NEXT(pfh, pfil_link)) { 83 if (pfh->pfil_func != NULL) { 84 ASSERT_HOST_BYTE_ORDER(m); 85 rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, 86 inp); 87 if (rv != 0 || m == NULL) 88 break; 89 ASSERT_HOST_BYTE_ORDER(m); (kgdb) p *pfh $2 = {pfil_link = {tqe_next = 0xfffffe00058c5980, tqe_prev = 0xfffffe0005821b00}, pfil_func = 0xffffffff80779c33 , pfil_arg = 0x0} (kgdb) p pfh->pfil_func $3 = (int (*)(void *, struct mbuf **, struct ifnet *, int, struct inpcb *)) 0xffffffff80779c33 (kgdb) p pf_check_out $4 = {int (void *, struct mbuf **, struct ifnet *, int, struct inpcb *)} 0xffffffff8032d39a (kgdb) p fr_check_wrapper $5 = {int (void *, struct mbuf **, struct ifnet *, int)} 0xffffffff802fc303 (kgdb) p ipfw_check_hook $6 = {int (void *, struct mbuf **, struct ifnet *, int, struct inpcb *)} 0xffffffff80779c33 (kgdb) #### kgdb.out_assert_new - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvfhRSPOsGF+KA+MRAlM/AKClrSdzDyqSgechCL/RKRtj6KHpVQCfQtCL PQk+XB5xpajaVmaGba7wD7s= =J22z -----END PGP SIGNATURE----- --3469798045-664628730-1337849937=:89783-- From owner-freebsd-pf@FreeBSD.ORG Thu May 24 09:43:56 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DB311065672 for ; Thu, 24 May 2012 09:43:56 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id DC00C8FC18 for ; Thu, 24 May 2012 09:43:55 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4O9hsKL026924 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Thu, 24 May 2012 11:43:54 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4O9hsXD000340; Thu, 24 May 2012 11:43:54 +0200 (MEST) Date: Thu, 24 May 2012 11:43:54 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120524094354.GK29536@insomnia.benzedrine.cx> References: <201205240910.q4O9A4rt044627@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205240910.q4O9A4rt044627@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 09:43:56 -0000 On Thu, May 24, 2012 at 09:10:04AM +0000, Joerg Pulz wrote: > panic: ipfw_check_hook:281 ASSERT_HOST_BYTE_ORDER 45056 176 > ipfw_check_hook() at ipfw_check_hook+0x511 > pfil_run_hooks() at pfil_run_hooks+0xf1 > ip_output() at ip_output+0x6de > ip_forward() at ip_forward+0x19e > ip_input() at ip_input+0x680 > swi_net() at swi_net+0x15a OK, this convinces me that the problem is in ipfw. You enabled it with options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=100 options IPFIREWALL_DEFAULT_TO_ACCEPT but say you're not using it? The above will actually enable ipfw's packet inspection with a default pass rule. And a non-trivial amount of code runs, unlike pf (and ipfilter), which must first be enabled (like with pfctl -e) first. Could you rebuild a kernel without the above options, just to confirm the theory that the problem is related to ipfw? We can try to find the problem within ipfw, maybe asking the ipfw developers for help. Daniel From owner-freebsd-pf@FreeBSD.ORG Thu May 24 13:51:45 2012 Return-Path: Delivered-To: freebsd-pf@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C03FA106566C; Thu, 24 May 2012 13:51:45 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 4758B8FC18; Thu, 24 May 2012 13:51:45 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4ODoTfl028884; Thu, 24 May 2012 15:50:29 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4ODo7jl028861 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 24 May 2012 15:50:07 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Thu, 24 May 2012 15:50:04 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120524094354.GK29536@insomnia.benzedrine.cx> Message-ID: References: <201205240910.q4O9A4rt044627@freefall.freebsd.org> <20120524094354.GK29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Thu, 24 May 2012 15:50:07 +0200 (CEST) Cc: bug-followup@FreeBSD.org, freebsd-pf@FreeBSD.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 13:51:45 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 24 May 2012, Daniel Hartmeier wrote: > On Thu, May 24, 2012 at 09:10:04AM +0000, Joerg Pulz wrote: > >> panic: ipfw_check_hook:281 ASSERT_HOST_BYTE_ORDER 45056 176 >> ipfw_check_hook() at ipfw_check_hook+0x511 >> pfil_run_hooks() at pfil_run_hooks+0xf1 >> ip_output() at ip_output+0x6de >> ip_forward() at ip_forward+0x19e >> ip_input() at ip_input+0x680 >> swi_net() at swi_net+0x15a > > OK, this convinces me that the problem is in ipfw. > > You enabled it with > > options IPFIREWALL > options IPFIREWALL_VERBOSE > options IPFIREWALL_VERBOSE_LIMIT=100 > options IPFIREWALL_DEFAULT_TO_ACCEPT > > but say you're not using it? > > The above will actually enable ipfw's packet inspection with a default > pass rule. And a non-trivial amount of code runs, unlike pf (and > ipfilter), which must first be enabled (like with pfctl -e) first. > > Could you rebuild a kernel without the above options, just to confirm > the theory that the problem is related to ipfw? > > We can try to find the problem within ipfw, maybe asking the ipfw > developers for help. Daniel, exactly, ipfw was enabled with the above kernel options but not configured to filter or do anything but the DEFAULT_TO_ACCEPT. I've rebuilt the kernel without IPFIREWALL options. The system is running now for about three and a half hours. Time will show if this solved our problem. I'm still wondering why these panics showed up in irregular unreproducable intervals. Thanks for writing to the ipfw list. I'm really interested in tracking this further down to fix it forever, so nobody will stumble over it again. Thanks for all your help. Feel free to contact me if you have new ideas or things i should try. Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvjyPSPOsGF+KA+MRAqgqAJ0Z8uuoOLHpbEcUTSrg1oXgNu7sowCfem2Z r8rPTyO39GMo9qJa10z+zzM= =pq7s -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Thu May 24 14:00:19 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C3C201065673 for ; Thu, 24 May 2012 14:00:19 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 93E208FC1D for ; Thu, 24 May 2012 14:00:19 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4OE0J3K001704 for ; Thu, 24 May 2012 14:00:19 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4OE0JIb001703; Thu, 24 May 2012 14:00:19 GMT (envelope-from gnats) Date: Thu, 24 May 2012 14:00:19 GMT Message-Id: <201205241400.q4OE0JIb001703@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 14:00:19 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: bug-followup@FreeBSD.org, freebsd-pf@FreeBSD.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Thu, 24 May 2012 15:50:04 +0200 (CEST) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 24 May 2012, Daniel Hartmeier wrote: > On Thu, May 24, 2012 at 09:10:04AM +0000, Joerg Pulz wrote: > >> panic: ipfw_check_hook:281 ASSERT_HOST_BYTE_ORDER 45056 176 >> ipfw_check_hook() at ipfw_check_hook+0x511 >> pfil_run_hooks() at pfil_run_hooks+0xf1 >> ip_output() at ip_output+0x6de >> ip_forward() at ip_forward+0x19e >> ip_input() at ip_input+0x680 >> swi_net() at swi_net+0x15a > > OK, this convinces me that the problem is in ipfw. > > You enabled it with > > options IPFIREWALL > options IPFIREWALL_VERBOSE > options IPFIREWALL_VERBOSE_LIMIT=100 > options IPFIREWALL_DEFAULT_TO_ACCEPT > > but say you're not using it? > > The above will actually enable ipfw's packet inspection with a default > pass rule. And a non-trivial amount of code runs, unlike pf (and > ipfilter), which must first be enabled (like with pfctl -e) first. > > Could you rebuild a kernel without the above options, just to confirm > the theory that the problem is related to ipfw? > > We can try to find the problem within ipfw, maybe asking the ipfw > developers for help. Daniel, exactly, ipfw was enabled with the above kernel options but not configured to filter or do anything but the DEFAULT_TO_ACCEPT. I've rebuilt the kernel without IPFIREWALL options. The system is running now for about three and a half hours. Time will show if this solved our problem. I'm still wondering why these panics showed up in irregular unreproducable intervals. Thanks for writing to the ipfw list. I'm really interested in tracking this further down to fix it forever, so nobody will stumble over it again. Thanks for all your help. Feel free to contact me if you have new ideas or things i should try. Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvjyPSPOsGF+KA+MRAqgqAJ0Z8uuoOLHpbEcUTSrg1oXgNu7sowCfem2Z r8rPTyO39GMo9qJa10z+zzM= =pq7s -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Fri May 25 07:27:05 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 55EB21065670; Fri, 25 May 2012 07:27:05 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id BF1658FC08; Fri, 25 May 2012 07:27:04 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4P7Pifr053141; Fri, 25 May 2012 09:25:44 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4P7Pfks053138 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 25 May 2012 09:25:42 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Fri, 25 May 2012 09:25:38 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: Message-ID: References: <201205240910.q4O9A4rt044627@freefall.freebsd.org> <20120524094354.GK29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Fri, 25 May 2012 09:25:42 +0200 (CEST) Cc: bug-followup@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 07:27:05 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 24 May 2012, Joerg Pulz wrote: > Daniel, > > exactly, ipfw was enabled with the above kernel options but not configured > to filter or do anything but the DEFAULT_TO_ACCEPT. > I've rebuilt the kernel without IPFIREWALL options. The system is running > now for about three and a half hours. > Time will show if this solved our problem. > I'm still wondering why these panics showed up in irregular unreproducable > intervals. > > Thanks for writing to the ipfw list. I'm really interested in tracking > this further down to fix it forever, so nobody will stumble over it again. > > Thanks for all your help. Feel free to contact me if you have new ideas or > things i should try. Daniel, the system is still running without panic, but i found the following log entries from last night: May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) Do you think that this may be related to the panics? I've found this error message two times in contrib/pf/net/pf.c. I can't say which of them or both have printed the message. Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvzP1SPOsGF+KA+MRAngoAJ4wk4PSjEtYvpCak2H8Qze8GaUbfwCgg2dq 2sQgy+3qWttRKxCj/WctPvY= =ejhQ -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Fri May 25 07:30:17 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D80CA106564A for ; Fri, 25 May 2012 07:30:16 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 80A1D8FC08 for ; Fri, 25 May 2012 07:30:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4P7UGYF006040 for ; Fri, 25 May 2012 07:30:16 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4P7UGu0006036; Fri, 25 May 2012 07:30:16 GMT (envelope-from gnats) Date: Fri, 25 May 2012 07:30:16 GMT Message-Id: <201205250730.q4P7UGu0006036@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 07:30:17 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: bug-followup@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Fri, 25 May 2012 09:25:38 +0200 (CEST) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 24 May 2012, Joerg Pulz wrote: > Daniel, > > exactly, ipfw was enabled with the above kernel options but not configured > to filter or do anything but the DEFAULT_TO_ACCEPT. > I've rebuilt the kernel without IPFIREWALL options. The system is running > now for about three and a half hours. > Time will show if this solved our problem. > I'm still wondering why these panics showed up in irregular unreproducable > intervals. > > Thanks for writing to the ipfw list. I'm really interested in tracking > this further down to fix it forever, so nobody will stumble over it again. > > Thanks for all your help. Feel free to contact me if you have new ideas or > things i should try. Daniel, the system is still running without panic, but i found the following log entries from last night: May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) Do you think that this may be related to the panics? I've found this error message two times in contrib/pf/net/pf.c. I can't say which of them or both have printed the message. Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPvzP1SPOsGF+KA+MRAngoAJ4wk4PSjEtYvpCak2H8Qze8GaUbfwCgg2dq 2sQgy+3qWttRKxCj/WctPvY= =ejhQ -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Fri May 25 09:16:37 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57B9F106566B for ; Fri, 25 May 2012 09:16:37 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id 934188FC12 for ; Fri, 25 May 2012 09:16:36 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q4P9GSio014711 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Fri, 25 May 2012 11:16:28 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q4P9GRDx023127; Fri, 25 May 2012 11:16:28 +0200 (MEST) Date: Fri, 25 May 2012 11:16:27 +0200 From: Daniel Hartmeier To: Joerg Pulz Message-ID: <20120525091627.GA27514@insomnia.benzedrine.cx> References: <201205250730.q4P7UGu0006036@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201205250730.q4P7UGu0006036@freefall.freebsd.org> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 09:16:37 -0000 On Fri, May 25, 2012 at 07:30:16AM +0000, Joerg Pulz wrote: > the system is still running without panic, but i found the following log > entries from last night: > > May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) > May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) > > Do you think that this may be related to the panics? > I've found this error message two times in contrib/pf/net/pf.c. > I can't say which of them or both have printed the message. Yes, this could possibly explain it. All pfil consumers assume that the IP header is one continuous memory region in the mbuf, without verifying this or correcting it (with m_pullup() or such) if wrong. pf: pf_check_in() in sys/contrib/pf/net/pf_ioctl.c h = mtod(*m, struct ip *); access h->ip_len ipfw: ipfw_check_hook() in sys/netinet/ipfw/ip_fw_pfil.c SET_NET_IPLEN(mtod(*m0, struct ip *)); ipfilter: fr_check_wrapper() in sys/contrib/ipfilter/netinet/ip_fil_freebsd.c struct ip *ip = mtod(*mp, struct ip *); access ip->ip_hl Hence, the caller of pfil_run_hooks() must ensure this before the call. ip_input() does, but there are operations that might violate the condition again. If the IP header is not continuous in the first mbuf, doing struct ip *ip = mtod(m, struct ip *); ip->ip_len = ntohs(ip->ip_len); will read (and write) unrelated memory. If, later, something does call m_pullup(), ip_len will get 'replaced' again. This could lead to some byte order swaps getting undone. I'm not sure what the proper action is here, i.e. should we be surprised that an mbuf with such a small m_len is found, and track down how it was produced, or should the pfil code simply expect this? I'd probably add a sanity check to pfil_run_hooks(), like --- sys/net/pfil.c 23 Sep 2011 00:51:37 -0000 1.19.2.1 +++ sys/net/pfil.c 25 May 2012 09:10:27 -0000 @@ -46,6 +46,8 @@ #include #include +#include +#include static struct mtx pfil_global_lock; @@ -74,15 +76,21 @@ struct mbuf *m = *mp; int rv = 0; + if (m->m_pkthdr.len < sizeof(struct ip) || + m->m_len < sizeof(struct ip)) + panic("pfil_run_hooks: m->m_pkthdr.len %d, m->m_len %d", + (int)m->m_pkthdr.len, (int)m->m_len); PFIL_RLOCK(ph, &rmpt); KASSERT(ph->ph_nhooks >= 0, ("Pfil hook count dropped < 0")); for (pfh = pfil_hook_get(dir, ph); pfh != NULL; pfh = TAILQ_NEXT(pfh, pfil_link)) { if (pfh->pfil_func != NULL) { + ASSERT_HOST_BYTE_ORDER(m); rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, inp); if (rv != 0 || m == NULL) break; + ASSERT_HOST_BYTE_ORDER(m); } } PFIL_RUNLOCK(ph, &rmpt); Then when it will panic (instead of just the pf_route() message), the stack trace could help. This might require several iterations, adding such checks all around the place. The cause might be some mbuf operation done in ipsec, for some edge case, explaining why it occurs so rarely... If I could easily reproduce it locally, I'd probably do it, but it's your machine that crashes all the time, so it's your call :) Daniel From owner-freebsd-pf@FreeBSD.ORG Fri May 25 11:26:54 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E2B94106564A; Fri, 25 May 2012 11:26:54 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 48A8D8FC0A; Fri, 25 May 2012 11:26:54 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4PBPc8X062293; Fri, 25 May 2012 13:25:38 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4PBPZfe062289 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 25 May 2012 13:25:35 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Fri, 25 May 2012 13:25:32 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120525091627.GA27514@insomnia.benzedrine.cx> Message-ID: References: <201205250730.q4P7UGu0006036@freefall.freebsd.org> <20120525091627.GA27514@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Fri, 25 May 2012 13:25:35 +0200 (CEST) Cc: bug-followup@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 11:26:55 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 25 May 2012, Daniel Hartmeier wrote: > On Fri, May 25, 2012 at 07:30:16AM +0000, Joerg Pulz wrote: > >> the system is still running without panic, but i found the following log >> entries from last night: >> >> May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) >> May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) >> >> Do you think that this may be related to the panics? >> I've found this error message two times in contrib/pf/net/pf.c. >> I can't say which of them or both have printed the message. > > Yes, this could possibly explain it. > > All pfil consumers assume that the IP header is one continuous memory > region in the mbuf, without verifying this or correcting it (with > m_pullup() or such) if wrong. > > pf: pf_check_in() in sys/contrib/pf/net/pf_ioctl.c > h = mtod(*m, struct ip *); > access h->ip_len > > ipfw: ipfw_check_hook() in sys/netinet/ipfw/ip_fw_pfil.c > SET_NET_IPLEN(mtod(*m0, struct ip *)); > > ipfilter: fr_check_wrapper() in sys/contrib/ipfilter/netinet/ip_fil_freebsd.c > struct ip *ip = mtod(*mp, struct ip *); > access ip->ip_hl > > Hence, the caller of pfil_run_hooks() must ensure this before the call. > ip_input() does, but there are operations that might violate the > condition again. > > If the IP header is not continuous in the first mbuf, doing > > struct ip *ip = mtod(m, struct ip *); > ip->ip_len = ntohs(ip->ip_len); > > will read (and write) unrelated memory. > > If, later, something does call m_pullup(), ip_len will get 'replaced' > again. This could lead to some byte order swaps getting undone. > > I'm not sure what the proper action is here, i.e. should we be surprised > that an mbuf with such a small m_len is found, and track down how it > was produced, or should the pfil code simply expect this? > > I'd probably add a sanity check to pfil_run_hooks(), like > > --- sys/net/pfil.c 23 Sep 2011 00:51:37 -0000 1.19.2.1 > +++ sys/net/pfil.c 25 May 2012 09:10:27 -0000 > @@ -46,6 +46,8 @@ > > #include > #include > +#include > +#include > > static struct mtx pfil_global_lock; > > @@ -74,15 +76,21 @@ > struct mbuf *m = *mp; > int rv = 0; > > + if (m->m_pkthdr.len < sizeof(struct ip) || > + m->m_len < sizeof(struct ip)) > + panic("pfil_run_hooks: m->m_pkthdr.len %d, m->m_len %d", > + (int)m->m_pkthdr.len, (int)m->m_len); > PFIL_RLOCK(ph, &rmpt); > KASSERT(ph->ph_nhooks >= 0, ("Pfil hook count dropped < 0")); > for (pfh = pfil_hook_get(dir, ph); pfh != NULL; > pfh = TAILQ_NEXT(pfh, pfil_link)) { > if (pfh->pfil_func != NULL) { > + ASSERT_HOST_BYTE_ORDER(m); > rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, > inp); > if (rv != 0 || m == NULL) > break; > + ASSERT_HOST_BYTE_ORDER(m); > } > } > PFIL_RUNLOCK(ph, &rmpt); > > Then when it will panic (instead of just the pf_route() message), the > stack trace could help. > > This might require several iterations, adding such checks all around the > place. The cause might be some mbuf operation done in ipsec, for some > edge case, explaining why it occurs so rarely... > > If I could easily reproduce it locally, I'd probably do it, but it's > your machine that crashes all the time, so it's your call :) Daniel, thanks for your explaining. As i said, i will do everything to track this down to the real bottom to get it fixed forever. Your proposed changes are in and the system is running with the rebuilt kernel. Now it's time to wait and see what happens. Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPv2wvSPOsGF+KA+MRAp1DAKCp2KZdPKBdsR5PUbK3ixqXqFdi9ACfbfvd vez+bq9X5lMjbdysCXy+stU= =tvtF -----END PGP SIGNATURE----- From owner-freebsd-pf@FreeBSD.ORG Fri May 25 11:30:11 2012 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5EDFE1065674 for ; Fri, 25 May 2012 11:30:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 599838FC15 for ; Fri, 25 May 2012 11:30:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q4PBU65Y079017 for ; Fri, 25 May 2012 11:30:06 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q4PBU64x079016; Fri, 25 May 2012 11:30:06 GMT (envelope-from gnats) Date: Fri, 25 May 2012 11:30:06 GMT Message-Id: <201205251130.q4PBU64x079016@freefall.freebsd.org> To: freebsd-pf@FreeBSD.org From: Joerg Pulz Cc: Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joerg Pulz List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 11:30:11 -0000 The following reply was made to PR kern/168190; it has been noted by GNATS. From: Joerg Pulz To: Daniel Hartmeier Cc: bug-followup@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) Date: Fri, 25 May 2012 13:25:32 +0200 (CEST) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 25 May 2012, Daniel Hartmeier wrote: > On Fri, May 25, 2012 at 07:30:16AM +0000, Joerg Pulz wrote: > >> the system is still running without panic, but i found the following log >> entries from last night: >> >> May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) >> May 24 23:28:57 charon kernel: pf_route: m0->m_len < sizeof(struct ip) >> >> Do you think that this may be related to the panics? >> I've found this error message two times in contrib/pf/net/pf.c. >> I can't say which of them or both have printed the message. > > Yes, this could possibly explain it. > > All pfil consumers assume that the IP header is one continuous memory > region in the mbuf, without verifying this or correcting it (with > m_pullup() or such) if wrong. > > pf: pf_check_in() in sys/contrib/pf/net/pf_ioctl.c > h = mtod(*m, struct ip *); > access h->ip_len > > ipfw: ipfw_check_hook() in sys/netinet/ipfw/ip_fw_pfil.c > SET_NET_IPLEN(mtod(*m0, struct ip *)); > > ipfilter: fr_check_wrapper() in sys/contrib/ipfilter/netinet/ip_fil_freebsd.c > struct ip *ip = mtod(*mp, struct ip *); > access ip->ip_hl > > Hence, the caller of pfil_run_hooks() must ensure this before the call. > ip_input() does, but there are operations that might violate the > condition again. > > If the IP header is not continuous in the first mbuf, doing > > struct ip *ip = mtod(m, struct ip *); > ip->ip_len = ntohs(ip->ip_len); > > will read (and write) unrelated memory. > > If, later, something does call m_pullup(), ip_len will get 'replaced' > again. This could lead to some byte order swaps getting undone. > > I'm not sure what the proper action is here, i.e. should we be surprised > that an mbuf with such a small m_len is found, and track down how it > was produced, or should the pfil code simply expect this? > > I'd probably add a sanity check to pfil_run_hooks(), like > > --- sys/net/pfil.c 23 Sep 2011 00:51:37 -0000 1.19.2.1 > +++ sys/net/pfil.c 25 May 2012 09:10:27 -0000 > @@ -46,6 +46,8 @@ > > #include > #include > +#include > +#include > > static struct mtx pfil_global_lock; > > @@ -74,15 +76,21 @@ > struct mbuf *m = *mp; > int rv = 0; > > + if (m->m_pkthdr.len < sizeof(struct ip) || > + m->m_len < sizeof(struct ip)) > + panic("pfil_run_hooks: m->m_pkthdr.len %d, m->m_len %d", > + (int)m->m_pkthdr.len, (int)m->m_len); > PFIL_RLOCK(ph, &rmpt); > KASSERT(ph->ph_nhooks >= 0, ("Pfil hook count dropped < 0")); > for (pfh = pfil_hook_get(dir, ph); pfh != NULL; > pfh = TAILQ_NEXT(pfh, pfil_link)) { > if (pfh->pfil_func != NULL) { > + ASSERT_HOST_BYTE_ORDER(m); > rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, > inp); > if (rv != 0 || m == NULL) > break; > + ASSERT_HOST_BYTE_ORDER(m); > } > } > PFIL_RUNLOCK(ph, &rmpt); > > Then when it will panic (instead of just the pf_route() message), the > stack trace could help. > > This might require several iterations, adding such checks all around the > place. The cause might be some mbuf operation done in ipsec, for some > edge case, explaining why it occurs so rarely... > > If I could easily reproduce it locally, I'd probably do it, but it's > your machine that crashes all the time, so it's your call :) Daniel, thanks for your explaining. As i said, i will do everything to track this down to the real bottom to get it fixed forever. Your proposed changes are in and the system is running with the rebuilt kernel. Now it's time to wait and see what happens. Kind regards Joerg - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPv2wvSPOsGF+KA+MRAp1DAKCp2KZdPKBdsR5PUbK3ixqXqFdi9ACfbfvd vez+bq9X5lMjbdysCXy+stU= =tvtF -----END PGP SIGNATURE-----