From owner-freebsd-pf@FreeBSD.ORG Mon May 21 18:49:36 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E53F1065670; Mon, 21 May 2012 18:49:36 +0000 (UTC) (envelope-from Joerg.Pulz@frm2.tum.de) Received: from mailhost.frm2.tum.de (mailhost.frm2.tum.de [129.187.179.12]) by mx1.freebsd.org (Postfix) with ESMTP id 95EFC8FC14; Mon, 21 May 2012 18:49:35 +0000 (UTC) Received: from mailhost.frm2.tum.de (localhost [127.0.0.1]) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4LImJxP008552; Mon, 21 May 2012 20:48:19 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) X-Virus-Scanned: at mailhost.frm2.tum.de Received: from hades.admin.frm2 (hades.admin.frm2 [172.25.1.10]) (authenticated bits=0) by mailhost.frm2.tum.de (8.14.4/8.14.4) with ESMTP id q4LImF4a008547 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 21 May 2012 20:48:18 +0200 (CEST) (envelope-from Joerg.Pulz@frm2.tum.de) Date: Mon, 21 May 2012 20:48:12 +0200 (CEST) From: Joerg Pulz To: Daniel Hartmeier In-Reply-To: <20120521165037.GA29536@insomnia.benzedrine.cx> Message-ID: References: <201205211420.q4LEK4ds039516@freefall.freebsd.org> <20120521165037.GA29536@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3469798045-1197118405-1337625847=:74709" Content-ID: X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (mailhost.frm2.tum.de [129.187.179.12]); Mon, 21 May 2012 20:48:18 +0200 (CEST) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-pf@freebsd.org Subject: Re: kern/168190: [pf] panic when using pf and route-to (maybe: bad fragment handling?) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 18:49:36 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3469798045-1197118405-1337625847=:74709 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT Content-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: On Mon, 21 May 2012, Daniel Hartmeier wrote: > On Mon, May 21, 2012 at 02:20:04PM +0000, Joerg Pulz wrote: > >> ext_if="bge0" >> int_if="bge1" >> vpn_net="10.1.1.0/24" >> srv_net="172.16.1.0/24" >> gw_addr="172.16.1.254" >> >> scrub in all >> >> pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state >> pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > > So something from $vpn_net comes in, gets routed to the default gateway > (on $ext_if side), attempts to pass out on $ext_if, matches the first > rule, route-to applies, packet gets re-routed to $gw_addr, passes out > on $int_if, matches the second rule, double route-to. > > All you need to do is prevent the second rule from applying for packets > where the first rule matched, like with tags: > > pass out on $ext_if route-to ($int_if $gw_addr) from $vpn_net to any keep state tag from_vpn > pass out on $int_if route-to ($int_if $gw_addr) from $vpn_net to $srv_net keep state > pass out on $int_if from $vpn_net to $srv_net keep state tagged from_vpn > > i.e. you add 'tag from_vpn' to the first rule, so packets matching it > get tagged, then you add a third rule without route-to that applies to > tagged packets, which wins last-match for such packets. > > Or, instead of adding a third rule, add '! tagged from_vpn' to the > second rule, if tagged packets can still pass out on $int_if by another > rule. Daniel, thanks again. There is one main question left for me. How could a packet which is directed to a host in $srv_net (and $srv_net is the same broadcast domain as $int_if) ever leave through $ext_if and make the first route-to match and therefor lead to a double route-to? Regarding your other mail with the reproduce instructions > BTW, if the theory is correct, you should be able to reproduce the > problem by sending a packet in from $vpn_net to a host in $srv_net > of size 334 (=306+28), e.g. with > > $ ping -s 306 172.16.1.1 I tried this, but it did no harm to the system at all. Packets where flowing as they should and answers came back. PF ruleset is still unchanged, no tagging. Meanwhile the system panic'ed again, and this time there is no double route-to involved at all. kgdb(1) output below. If your assumption in your first response is right: > It looks like the byte order of ip_len is wrong, htons(334) = 19969, > triggering fragmentation (334 < if_mtu, but 19969 > if_mtu). Then it should read htons(175) = 44800 (175 < if_mtu, but 44800 > if_mtu") But where is it coming from if double route-to can't be the culprit in this case? Any thoughts on this? Kind regards Joerg ### kgdb.out2 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: m_copym, offset > size of mbuf chain cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x182 m_copym() at m_copym+0x280 ip_fragment() at ip_fragment+0x1e5 pf_route() at pf_route+0x75c pf_test() at pf_test+0xc29 pf_check_out() at pf_check_out+0x3a pfil_run_hooks() at pfil_run_hooks+0xd2 ip_output() at ip_output+0x655 ip_forward() at ip_forward+0x175 ip_input() at ip_input+0x5fd swi_net() at swi_net+0x15a intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xaf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe - --- trap 0, rip = 0, rsp = 0xffffff8000241d00, rbp = 0 --- KDB: enter: panic Dumping 628 out of 4077 MB:..3%..11%..21%..31%..41%..51%..62%..72%..82%..92% Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko #0 doadump (textdump=0) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) up 10 #10 0xffffffff806e9079 in m_copym (m=0x0, off0=1500, len=1480, wait=1) at /usr/src/sys/kern/uipc_mbuf.c:541 541 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); (kgdb) list 536 KASSERT(len >= 0, ("m_copym, negative len %d", len)); 537 MBUF_CHECKSLEEP(wait); 538 if (off == 0 && m->m_flags & M_PKTHDR) 539 copyhdr = 1; 540 while (off > 0) { 541 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); 542 if (off < m->m_len) 543 break; 544 off -= m->m_len; 545 m = m->m_next; (kgdb) p off $1 = 1325 (kgdb) up #11 0xffffffff8077fe1f in ip_fragment (ip=0xfffffe0173b1fc74, m_frag=0xffffff8000241658, mtu=) at /usr/src/sys/netinet/ip_output.c:816 816 m->m_next = m_copym(m0, off, len, M_DONTWAIT); (kgdb) list 811 len = ip->ip_len - off; 812 m->m_flags |= M_LASTFRAG; 813 } else 814 mhip->ip_off |= IP_MF; 815 mhip->ip_len = htons((u_short)(len + mhlen)); 816 m->m_next = m_copym(m0, off, len, M_DONTWAIT); 817 if (m->m_next == NULL) { /* copy failed */ 818 m_free(m); 819 error = ENOBUFS; /* ??? */ 820 IPSTAT_INC(ips_odropped); (kgdb) p *m0 $2 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p off $3 = 1500 (kgdb) p len $4 = 1480 (kgdb) p hlen $5 = 20 (kgdb) up #12 0xffffffff8032842a in pf_route (m=0xffffff80002418e8, r=0xfffffe0005d87750, dir=) at /usr/src/sys/contrib/pf/net/pf.c:6138 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); (kgdb) list 6133 /* 6134 * XXX: is cheaper + less error prone than own function 6135 */ 6136 NTOHS(ip->ip_len); 6137 NTOHS(ip->ip_off); 6138 error = ip_fragment(ip, &m0, ifp->if_mtu, ifp->if_hwassist, sw_csum); 6139 #else 6140 error = ip_fragment(m0, ifp, ifp->if_mtu); 6141 #endif 6142 if (error) { (kgdb) p *ip $6 = {ip_hl = 5 '\005', ip_v = 4 '\004', ip_tos = 0 '\0', ip_len = 44800, ip_id = 11615, ip_off = 0, ip_ttl = 127 '\177', ip_p = 1 '\001', ip_sum = 18204, ip_src = {s_addr = 34052874}, ip_dst = {s_addr = 604051884}} (kgdb) p *m0 $7 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) p *m1 $8 = {m_hdr = {mh_next = 0xfffffe0005b70000, mh_nextpkt = 0x0, mh_data = 0xfffffe0173b1fc74 "E", mh_len = 60, mh_flags = 66, mh_type = 1, pad = "­ÞÞÀ­Þ"}, M_dat = {MH = {MH_pkthdr = {rcvif = 0xfffffe0003001800, header = 0x0, len = 175, flowid = 0, csum_flags = 1, csum_data = 59883, tso_segsz = 0, PH_vt = {vt_vtag = 0, vt_nrecs = 0}, tags = {slh_first = 0xfffffe00058c3840}}, MH_dat = {MH_ext = { ext_buf = 0xd29300ec0045
, ext_free = 0xd29300af0045, ext_arg1 = 0xba41965002280437, ext_arg2 = 0xaf00004557b3bb81, ext_size = 11615, ref_cnt = 0x240119ac02079b0a, ext_type = 1740374787}, MH_databuf = "E\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"}}, M_databuf = "\000\030\000\003\000þÿÿ\000\000\000\000\000\000\000\000¯\000\000\000\000\000\000\000\001\000\000\000ëé\000\000\000\000\000\000ÞÀ­Þ@8\214\005\000þÿÿE\000ì\000\223Ò\000\000E\000¯\000\223Ò\000\0007\004(\002P\226Aº\201»³WE\000\000¯_-\000\000\177\001\034G\n\233\a\002¬\031\001$\003\003¼g\000\000\000\000E\000\000\223\215:\000\000>\0210F¬\031\001$\n\233\a\002\0005äv\000\177\vC\200\035\201\200\000\001\000\002\000\002\000\000ÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­ÞÞÀ­Þ"...}} (kgdb) ### kgdb.out2 - -- The beginning is the most important part of the work. -Plato -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPuo3vSPOsGF+KA+MRAu+AAKDGT0lDqaIcYO4Q6Lx37oUX64GeCgCffGhf AtfXrgD94GTXHsX7roaKfAI= =wEqQ -----END PGP SIGNATURE----- --3469798045-1197118405-1337625847=:74709--