Date: Fri, 13 Apr 2012 01:29:31 +0300 From: Theodor-Iulian Ciobanu <thciobanu@nth.ro> To: freebsd-pf@freebsd.org Subject: Re: Panic in packet filter Message-ID: <20120413012931.00006832@unknown> In-Reply-To: <CAPBZQG352UtXQ%2B4STkHNwF0f3e4z_2q=0zoRSBDCTZFMAJRAQg@mail.gmail.com> References: <CAOxY2CotiKHHcw%2Bjv2pAi6CbZ7oM3V7ohMrwHY0XhrwTAaRz1w@mail.gmail.com> <CAPBZQG16nbu09Bj8rKYUQbuRvCFikvF28REcM41t2urVzn7c1w@mail.gmail.com> <CAOxY2Cph2rt-1wnoQRBdsr%2BmLCHyBaMAYW2o8Z08W%2B3Dz-_7iw@mail.gmail.com> <D25EB30E-9241-4B81-A312-E37861DA5017@apple-park.kiev.ua> <20120412141632.00007c72@unknown> <CAPBZQG352UtXQ%2B4STkHNwF0f3e4z_2q=0zoRSBDCTZFMAJRAQg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 12 Apr 2012 15:01:46 +0200 Ermal Lu=E7i <eri@freebsd.org> wrote: > Hello, >=20 > On Thu, Apr 12, 2012 at 1:16 PM, Theodor-Iulian Ciobanu > <thciobanu@nth.ro> wrote: > > Hello, > > > > I came across this same issue yesterday on a system I have just set > > up. I'm currently using the default kernel: > > > > FreeBSD changeme 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan =A03 > > 07:46:30 UTC 2012 > > root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 > > > > with pf obviously loaded as a module. Even with kern.smp.disabled=3D1 > > pf will crash as soon as it matches a rule that contains tables with > > counters (I added such a table with just three addresses). > > > > I'll have this machine around for testing for about a week or so > > and am willing to try out any available patches to help fix the > > issue. > > >=20 > Try this patch > http://people.freebsd.org/~eri/pf_table_counter_fix.diff. It should > fix the issue for you. >=20 > Seems there is a forgotten pool initialization for this, my fault! >=20 > Though looking at it the whole thing seems a microoptimization that is > still present on latest OpenBSD code, > that saves about 16bytes! >=20 > Anyway see if it fixes the issue to get this committed. Great use of 16b, as it doesn't seem to crash anymore, at least in a simple synthetic test (uploading C:\Windows from 2 systems at once through ftp, 10 transfer connections each). Thank you! > > On Fri Feb 24 14:47:53 2012 > > iskander at apple-park.kiev.ua (Alexander Vyrlanovich) wrote: > > > >> > >> On 24 Feb 2012, at 11:10, Ali Mdidech wrote: > >> > >> > Hi Ermal, > >> > > >> > 2012/2/24 Ermal Lu?i <eri@freebsd.org>: > >> >> On Thu, Feb 23, 2012 at 8:44 AM, Ali Mdidech <ali@moua7.com> > >> >> wrote: > >> >>> Hi List, > >> >>> > >> >>> I've a box that panics multiple times randomly since a year > >> >>> whatever the release is (8 or 9) > >> >>> The crash dump shows that the problem is related to pf. > >> >>> Is this some sort of identified bug? > >> >>> Below some info and my pf.conf file. > >> >>> > >> >>> Thank you very much for your help. > >> >>> > >> >> > >> >> Can you try do disable SMP through sysctl and see if you still > >> >> get this? > >> >> What are you doing to get the panic? > >> > > >> > Well, I'm able now to avoid or reproduce the panic. > >> > Disabling counters in <ssh_brute> table makes the server stable > >> > enough and no panic for 48 hours. > >> > Restoring the counters and adding a host in the table by hand > >> > (pfctl -t ssh_brute -T add someip) provokes the panic within few > >> > seconds. I've disabled smp (adding kern.smp.disabled=3D1 in > >> > loader.conf and rebooting) =3D> kernel still panics. > >> > > >> > FreeBSD somehost 9.0-RELEASE FreeBSD 9.0-RELEASE #1: Sat Jan 21 > >> > 09:31:30 CET 2012 =A0 =A0 root@somehost:/usr/obj/usr/src/sys/DDX3KRNL > >> > i386 > >> I can confirm that problem with counters in pf tables persist > >> at last on i386 and amd64. My systems is: > >> > >> FreeBSD gw 9.0-RELEASE FreeBSD 9.0-RELEASE #1: Tue Jan =A03 15:55:41 > >> EET 2012 > >> root@gw:/usr/obj/usr/src/sys/GW3 =A0amd64 > >> > >> FreeBSD gw2 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Wed Jan 25 13:52:48 > >> EET 2012 > >> root@gw2:/usr/obj/usr/src/sys/GWS90 =A0i386 > >> > >> pf + altq compiled in kernel > >> > >> Same result: kernel panic. Without counters systems is rock solid. > >> > >> >> Also its very helpful to know the `uname -a` command output. > >> >> > >> >>> panic: page fault > >> >>> > >> >>> GNU gdb 6.1.1 [FreeBSD] > >> >>> Copyright 2004 Free Software Foundation, Inc. > >> >>> GDB is free software, covered by the GNU General Public > >> >>> License, and you are > >> >>> welcome to change it and/or distribute copies of it under > >> >>> certain conditions. > >> >>> Type "show copying" to see the conditions. > >> >>> There is absolutely no warranty for GDB. =A0Type "show warranty" > >> >>> for details. > >> >>> This GDB was configured as "i386-marcel-freebsd"... > >> >>> > >> >>> Unread portion of the kernel message buffer: > >> >>> > >> >>> > >> >>> Fatal trap 12: page fault while in kernel mode > >> >>> cpuid =3D 0; apic id =3D 00 > >> >>> fault virtual address =A0 =3D 0x6c > >> >>> fault code =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D supervisor read, page no= t present > >> >>> instruction pointer =A0 =A0 =3D 0x20:0xc0a25dc0 > >> >>> stack pointer =A0 =A0 =A0 =A0 =A0 =3D 0x28:0xc4df5910 > >> >>> frame pointer =A0 =A0 =A0 =A0 =A0 =3D 0x28:0xc4df5954 > >> >>> code segment =A0 =A0 =A0 =A0 =A0 =A0=3D base 0x0, limit 0xfffff, t= ype 0x1b > >> >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D DPL 0, pres 1, = def32 1, gran 1 > >> >>> processor eflags =A0 =A0 =A0 =A0=3D interrupt enabled, resume, IOP= L =3D 0 > >> >>> current process =A0 =A0 =A0 =A0 =3D 12 (irq256: em0:rx 0) > >> >>> trap number =A0 =A0 =A0 =A0 =A0 =A0 =3D 12 > >> >>> panic: page fault > >> >>> cpuid =3D 0 > >> >>> KDB: stack backtrace: > >> >>> #0 0xc08380b7 at kdb_backtrace+0x47 > >> >>> #1 0xc0805617 at panic+0x117 > >> >>> #2 0xc0aebcc3 at trap_fatal+0x323 > >> >>> #3 0xc0aec802 at trap+0x182 > >> >>> #4 0xc0ad5f8c at calltrap+0x6 > >> >>> #5 0xc589f7cc at pfr_update_stats+0x1cc > >> >>> #6 0xc588de21 at pf_test+0x981 > >> >>> #7 0xc5895e79 at pf_check_in+0x39 > >> >>> #8 0xc08c3c68 at pfil_run_hooks+0x78 > >> >>> #9 0xc08e18ae at ip_input+0x24e > >> >>> #10 0xc08c2d9f at netisr_dispatch_src+0x8f > >> >>> #11 0xc08c3040 at netisr_dispatch+0x20 > >> >>> #12 0xc08b9721 at ether_demux+0x171 > >> >>> #13 0xc08b9b6f at ether_nh_input+0x37f > >> >>> #14 0xc08c2d9f at netisr_dispatch_src+0x8f > >> >>> #15 0xc08c3040 at netisr_dispatch+0x20 > >> >>> #16 0xc08b9269 at ether_input+0x19 > >> >>> #17 0xc05b383f at em_rxeof+0x30f > >> >>> Uptime: 1h45m44s > >> >>> Physical memory: 2002 MB > >> >>> Dumping 185 MB: 170 154 138 122 106 90 74 58 42 26 10 > >> >>> > >> >>> Reading symbols from /boot/kernel/pf.ko...Reading symbols from > >> >>> /boot/kernel/pf.ko.symbols... > >> >>> done. > >> >>> done. > >> >>> Loaded symbols for /boot/kernel/pf.ko > >> >>> #0 =A0doadump (textdump=3D1) at pcpu.h:244 > >> >>> 244 =A0 =A0 pcpu.h: No such file or directory. > >> >>> =A0 =A0 =A0 =A0in pcpu.h > >> >>> (kgdb) #0 =A0doadump (textdump=3D1) at pcpu.h:244 > >> >>> #1 =A00xc08053ba in kern_reboot (howto=3D260) > >> >>> =A0 =A0at /usr/src/sys/kern/kern_shutdown.c:442 > >> >>> #2 =A00xc0805651 in panic (fmt=3DVariable "fmt" is not available. > >> >>> ) at /usr/src/sys/kern/kern_shutdown.c:607 > >> >>> #3 =A00xc0aebcc3 in trap_fatal (frame=3D0xc4df58d0, eva=3D108) > >> >>> =A0 =A0at /usr/src/sys/i386/i386/trap.c:975 > >> >>> #4 =A00xc0aec802 in trap (frame=3D0xc4df58d0) at /usr/src/sys/i386/ > >> >>> i386/trap.c:352 > >> >>> #5 =A00xc0ad5f8c in calltrap () at /usr/src/sys/i386/i386/ > >> >>> exception.s:168 > >> >>> #6 =A00xc0a25dc0 in uma_zalloc_arg (zone=3D0x0, udata=3D0x0, > >> >>> flags=3D257) at pcpu.h:244 > >> >>> #7 =A00xc589f7cc in pfr_update_stats (kt=3D0xc58d44d8, > >> >>> a=3D0xc56aa01a, af=3D2 '\002', > >> >>> =A0 =A0len=3D52, dir_out=3D0, op_pass=3D0, notrule=3D0) at uma.h:3= 05 > >> >>> #8 =A00xc588de21 in pf_test (dir=3D1, ifp=3D0xc5253c00, > >> >>> m0=3D0xc4df5acc, eh=3D0x0, > >> >>> =A0 =A0inp=3D0x0) > >> >>> at /usr/src/sys/modules/pf/../../contrib/pf/net/pf.c: 7057 > >> >>> #9 =A00xc5895e79 in pf_check_in (arg=3D0x0, m=3D0xc4df5acc, > >> >>> ifp=3D0xc5253c00, dir=3D1, > >> >>> =A0 =A0inp=3D0x0) at /usr/src/sys/modules/pf/../../contrib/pf/net/ > >> >>> pf_ioctl.c:4139 > >> >>> #10 0xc08c3c68 in pfil_run_hooks (ph=3D0xc0d685e0, mp=3D0xc4df5b24, > >> >>> =A0 =A0ifp=3D0xc5253c00, dir=3D1, inp=3D0x0) > >> >>> at /usr/src/sys/net/pfil.c:82 #11 0xc08e18ae in ip_input > >> >>> (m=3D0xc567db00) at /usr/src/sys/netinet/ip_input.c:510 > >> >>> #12 0xc08c2d9f in netisr_dispatch_src (proto=3D1, source=3D0, > >> >>> m=3D0xc567db00) > >> >>> =A0 =A0at /usr/src/sys/net/netisr.c:1013 > >> >>> #13 0xc08c3040 in netisr_dispatch (proto=3D1, m=3D0xc567db00) > >> >>> =A0 =A0at /usr/src/sys/net/netisr.c:1104 > >> >>> #14 0xc08b9721 in ether_demux (ifp=3D0xc5253c00, m=3D0xc567db00) > >> >>> =A0 =A0at /usr/src/sys/net/if_ethersubr.c:937 > >> >>> #15 0xc08b9b6f in ether_nh_input (m=3D0xc567db00) > >> >>> =A0 =A0at /usr/src/sys/net/if_ethersubr.c:756 > >> >>> #16 0xc08c2d9f in netisr_dispatch_src (proto=3D9, source=3D0, > >> >>> m=3D0xc567db00) > >> >>> =A0 =A0at /usr/src/sys/net/netisr.c:1013 > >> >>> #17 0xc08c3040 in netisr_dispatch (proto=3D9, m=3D0xc567db00) > >> >>> =A0 =A0at /usr/src/sys/net/netisr.c:1104 > >> >>> #18 0xc08b9269 in ether_input (ifp=3D0xc5253c00, m=3D0xc567db00) > >> >>> =A0 =A0at /usr/src/sys/net/if_ethersubr.c:797 > >> >>> #19 0xc05b383f in em_rxeof (rxr=3D0xc520bc00, count=3D99, done=3D0= x0) > >> >>> =A0 =A0at /usr/src/sys/dev/e1000/if_em.c:4340 > >> >>> #20 0xc05b3a06 in em_msix_rx (arg=3D0xc520bc00) > >> >>> =A0 =A0at /usr/src/sys/dev/e1000/if_em.c:1577 > >> >>> #21 0xc07da6eb in intr_event_execute_handlers (p=3D0xc5157588, > >> >>> ie=3D0xc5241680) > >> >>> =A0 =A0at /usr/src/sys/kern/kern_intr.c:1257 > >> >>> #22 0xc07dbeaa in ithread_loop (arg=3D0xc52506e0) > >> >>> =A0 =A0at /usr/src/sys/kern/kern_intr.c:1270 > >> >>> #23 0xc07d78f7 in fork_exit (callout=3D0xc07dbe30 <ithread_loop>, > >> >>> =A0 =A0arg=3D0xc52506e0, frame=3D0xc4df5d28) at /usr/src/sys/kern/ > >> >>> kern_fork.c:995 > >> >>> #24 0xc0ad6004 in fork_trampoline () at /usr/src/sys/i386/i386/ > >> >>> exception.s:275 > >> >>> (kgdb) > >> >>> > >> >>> > >> >>> ################## pf.conf ################## > >> >>> ext_if =3D "em0" > >> >>> > >> >>> public_tcp_ports =3D "{21,25,53,80,143,443,873,993,50021:50121}" > >> >>> public_udp_ports =3D "53" > >> >>> > >> >>> table <secure> {someip} > >> >>> table <ssh_brute> persist counters > >> >>> > >> >>> ### Redirection for SMTP > >> >>> rdr on $ext_if proto tcp from any to $ext_if port 225 -> > >> >>> $ext_if port 25 > >> >>> > >> >>> ### Block everything in an pass everything out > >> >>> pass out on $ext_if all modulate state > >> >>> block in on $ext_if all > >> >>> > >> >>> ### secure users > >> >>> pass in quick on $ext_if proto tcp from <secure> to any flags > >> >>> S/SA \ modulate state > >> >>> > >> >>> ### public tcp/udp ports rules > >> >>> pass in on $ext_if proto udp to $ext_if port $public_udp_ports > >> >>> pass in on $ext_if proto tcp to $ext_if port $public_tcp_ports > >> >>> flags S/SA \ > >> >>> modulate state > >> >>> > >> >>> ### block ssh bruteforce > >> >>> block in quick from <ssh_brute> > >> >>> pass in quick on $ext_if proto tcp to $ext_if port 22 flags > >> >>> S/SA modulate state \ > >> >>> (max-src-conn 5, max-src-conn-rate 10/60, overload <ssh_brute> > >> >>> flush global) > >> >>> > >> >>> ### block icmp timestamp request/response > >> >>> block in quick on $ext_if inet proto icmp all icmp-type {13, > >> >>> 14} pass in quick on $ext_if proto icmp all > >> >>> > >> >>> ############ end pf.conf ############## > >> >>> > >> >>> -- > >> >>> Ali Mdidech > >> >>> _______________________________________________ > >> >>> freebsd-pf@freebsd.org mailing list > >> >>> http://lists.freebsd.org/mailman/listinfo/freebsd-pf > >> >>> To unsubscribe, send any mail to "freebsd-pf- > >> >>> unsubscribe@freebsd.org" > >> >> > >> >> > >> >> > >> >> -- > >> >> Ermal > >> > > >> > -- > >> > Ali Mdidech > >> > _______________________________________________ > >> > freebsd-pf@freebsd.org mailing list > >> > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > >> > To unsubscribe, send any mail to > >> > "freebsd-pf-unsubscribe@freebsd.org" > >> > >> ????????? ?????????? > >> -------------------------- > >> ????????? ????????????? > >> ??? "???" > > > > -- > > Theo > > _______________________________________________ > > freebsd-pf@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > > To unsubscribe, send any mail to > > "freebsd-pf-unsubscribe@freebsd.org" --=20 Theo
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120413012931.00006832>