Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 12 Apr 2012 15:01:46 +0200
From:      =?ISO-8859-1?Q?Ermal_Lu=E7i?= <eri@freebsd.org>
To:        Theodor-Iulian Ciobanu <thciobanu@nth.ro>
Cc:        freebsd-pf@freebsd.org
Subject:   Re: Panic in packet filter
Message-ID:  <CAPBZQG352UtXQ%2B4STkHNwF0f3e4z_2q=0zoRSBDCTZFMAJRAQg@mail.gmail.com>
In-Reply-To: <20120412141632.00007c72@unknown>
References:  <CAOxY2CotiKHHcw%2Bjv2pAi6CbZ7oM3V7ohMrwHY0XhrwTAaRz1w@mail.gmail.com> <CAPBZQG16nbu09Bj8rKYUQbuRvCFikvF28REcM41t2urVzn7c1w@mail.gmail.com> <CAOxY2Cph2rt-1wnoQRBdsr%2BmLCHyBaMAYW2o8Z08W%2B3Dz-_7iw@mail.gmail.com> <D25EB30E-9241-4B81-A312-E37861DA5017@apple-park.kiev.ua> <20120412141632.00007c72@unknown>

next in thread | previous in thread | raw e-mail | index | archive | help
Hello,

On Thu, Apr 12, 2012 at 1:16 PM, Theodor-Iulian Ciobanu
<thciobanu@nth.ro> wrote:
> Hello,
>
> I came across this same issue yesterday on a system I have just set up.
> I'm currently using the default kernel:
>
> FreeBSD changeme 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan =A03 07:46:3=
0
> UTC 2012 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
> amd64
>
> with pf obviously loaded as a module. Even with kern.smp.disabled=3D1 pf
> will crash as soon as it matches a rule that contains tables with
> counters (I added such a table with just three addresses).
>
> I'll have this machine around for testing for about a week or so and am
> willing to try out any available patches to help fix the issue.
>

Try this patch http://people.freebsd.org/~eri/pf_table_counter_fix.diff.
It should fix the issue for you.

Seems there is a forgotten pool initialization for this, my fault!

Though looking at it the whole thing seems a microoptimization that is
still present on latest OpenBSD code,
that saves about 16bytes!

Anyway see if it fixes the issue to get this committed.

> On Fri Feb 24 14:47:53 2012
> iskander at apple-park.kiev.ua (Alexander Vyrlanovich) wrote:
>
>>
>> On 24 Feb 2012, at 11:10, Ali Mdidech wrote:
>>
>> > Hi Ermal,
>> >
>> > 2012/2/24 Ermal Lu?i <eri@freebsd.org>:
>> >> On Thu, Feb 23, 2012 at 8:44 AM, Ali Mdidech <ali@moua7.com> wrote:
>> >>> Hi List,
>> >>>
>> >>> I've a box that panics multiple times randomly since a year
>> >>> whatever the release is (8 or 9)
>> >>> The crash dump shows that the problem is related to pf.
>> >>> Is this some sort of identified bug?
>> >>> Below some info and my pf.conf file.
>> >>>
>> >>> Thank you very much for your help.
>> >>>
>> >>
>> >> Can you try do disable SMP through sysctl and see if you still
>> >> get this?
>> >> What are you doing to get the panic?
>> >
>> > Well, I'm able now to avoid or reproduce the panic.
>> > Disabling counters in <ssh_brute> table makes the server stable
>> > enough and no panic for 48 hours.
>> > Restoring the counters and adding a host in the table by hand (pfctl
>> > -t ssh_brute -T add someip) provokes the panic within few seconds.
>> > I've disabled smp (adding kern.smp.disabled=3D1 in loader.conf and
>> > rebooting) =3D> kernel still panics.
>> >
>> > FreeBSD somehost 9.0-RELEASE FreeBSD 9.0-RELEASE #1: Sat Jan 21
>> > 09:31:30 CET 2012 =A0 =A0 root@somehost:/usr/obj/usr/src/sys/DDX3KRNL
>> > i386
>> I can confirm that problem with counters in pf tables persist
>> at last on i386 and amd64. My systems is:
>>
>> FreeBSD gw 9.0-RELEASE FreeBSD 9.0-RELEASE #1: Tue Jan =A03 15:55:41
>> EET 2012
>> root@gw:/usr/obj/usr/src/sys/GW3 =A0amd64
>>
>> FreeBSD gw2 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Wed Jan 25 13:52:48
>> EET 2012
>> root@gw2:/usr/obj/usr/src/sys/GWS90 =A0i386
>>
>> pf + altq compiled in kernel
>>
>> Same result: kernel panic. Without counters systems is rock solid.
>>
>> >> Also its very helpful to know the `uname -a` command output.
>> >>
>> >>> panic: page fault
>> >>>
>> >>> GNU gdb 6.1.1 [FreeBSD]
>> >>> Copyright 2004 Free Software Foundation, Inc.
>> >>> GDB is free software, covered by the GNU General Public License,
>> >>> and you are
>> >>> welcome to change it and/or distribute copies of it under
>> >>> certain conditions.
>> >>> Type "show copying" to see the conditions.
>> >>> There is absolutely no warranty for GDB. =A0Type "show warranty"
>> >>> for details.
>> >>> This GDB was configured as "i386-marcel-freebsd"...
>> >>>
>> >>> Unread portion of the kernel message buffer:
>> >>>
>> >>>
>> >>> Fatal trap 12: page fault while in kernel mode
>> >>> cpuid =3D 0; apic id =3D 00
>> >>> fault virtual address =A0 =3D 0x6c
>> >>> fault code =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D supervisor read, page not =
present
>> >>> instruction pointer =A0 =A0 =3D 0x20:0xc0a25dc0
>> >>> stack pointer =A0 =A0 =A0 =A0 =A0 =3D 0x28:0xc4df5910
>> >>> frame pointer =A0 =A0 =A0 =A0 =A0 =3D 0x28:0xc4df5954
>> >>> code segment =A0 =A0 =A0 =A0 =A0 =A0=3D base 0x0, limit 0xfffff, typ=
e 0x1b
>> >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D DPL 0, pres 1, de=
f32 1, gran 1
>> >>> processor eflags =A0 =A0 =A0 =A0=3D interrupt enabled, resume, IOPL =
=3D 0
>> >>> current process =A0 =A0 =A0 =A0 =3D 12 (irq256: em0:rx 0)
>> >>> trap number =A0 =A0 =A0 =A0 =A0 =A0 =3D 12
>> >>> panic: page fault
>> >>> cpuid =3D 0
>> >>> KDB: stack backtrace:
>> >>> #0 0xc08380b7 at kdb_backtrace+0x47
>> >>> #1 0xc0805617 at panic+0x117
>> >>> #2 0xc0aebcc3 at trap_fatal+0x323
>> >>> #3 0xc0aec802 at trap+0x182
>> >>> #4 0xc0ad5f8c at calltrap+0x6
>> >>> #5 0xc589f7cc at pfr_update_stats+0x1cc
>> >>> #6 0xc588de21 at pf_test+0x981
>> >>> #7 0xc5895e79 at pf_check_in+0x39
>> >>> #8 0xc08c3c68 at pfil_run_hooks+0x78
>> >>> #9 0xc08e18ae at ip_input+0x24e
>> >>> #10 0xc08c2d9f at netisr_dispatch_src+0x8f
>> >>> #11 0xc08c3040 at netisr_dispatch+0x20
>> >>> #12 0xc08b9721 at ether_demux+0x171
>> >>> #13 0xc08b9b6f at ether_nh_input+0x37f
>> >>> #14 0xc08c2d9f at netisr_dispatch_src+0x8f
>> >>> #15 0xc08c3040 at netisr_dispatch+0x20
>> >>> #16 0xc08b9269 at ether_input+0x19
>> >>> #17 0xc05b383f at em_rxeof+0x30f
>> >>> Uptime: 1h45m44s
>> >>> Physical memory: 2002 MB
>> >>> Dumping 185 MB: 170 154 138 122 106 90 74 58 42 26 10
>> >>>
>> >>> Reading symbols from /boot/kernel/pf.ko...Reading symbols from
>> >>> /boot/kernel/pf.ko.symbols...
>> >>> done.
>> >>> done.
>> >>> Loaded symbols for /boot/kernel/pf.ko
>> >>> #0 =A0doadump (textdump=3D1) at pcpu.h:244
>> >>> 244 =A0 =A0 pcpu.h: No such file or directory.
>> >>> =A0 =A0 =A0 =A0in pcpu.h
>> >>> (kgdb) #0 =A0doadump (textdump=3D1) at pcpu.h:244
>> >>> #1 =A00xc08053ba in kern_reboot (howto=3D260)
>> >>> =A0 =A0at /usr/src/sys/kern/kern_shutdown.c:442
>> >>> #2 =A00xc0805651 in panic (fmt=3DVariable "fmt" is not available.
>> >>> ) at /usr/src/sys/kern/kern_shutdown.c:607
>> >>> #3 =A00xc0aebcc3 in trap_fatal (frame=3D0xc4df58d0, eva=3D108)
>> >>> =A0 =A0at /usr/src/sys/i386/i386/trap.c:975
>> >>> #4 =A00xc0aec802 in trap (frame=3D0xc4df58d0) at /usr/src/sys/i386/
>> >>> i386/trap.c:352
>> >>> #5 =A00xc0ad5f8c in calltrap () at /usr/src/sys/i386/i386/
>> >>> exception.s:168
>> >>> #6 =A00xc0a25dc0 in uma_zalloc_arg (zone=3D0x0, udata=3D0x0, flags=
=3D257)
>> >>> =A0 =A0at pcpu.h:244
>> >>> #7 =A00xc589f7cc in pfr_update_stats (kt=3D0xc58d44d8, a=3D0xc56aa01=
a,
>> >>> af=3D2 '\002',
>> >>> =A0 =A0len=3D52, dir_out=3D0, op_pass=3D0, notrule=3D0) at uma.h:305
>> >>> #8 =A00xc588de21 in pf_test (dir=3D1, ifp=3D0xc5253c00, m0=3D0xc4df5=
acc,
>> >>> eh=3D0x0,
>> >>> =A0 =A0inp=3D0x0) at /usr/src/sys/modules/pf/../../contrib/pf/net/pf=
.c:
>> >>> 7057
>> >>> #9 =A00xc5895e79 in pf_check_in (arg=3D0x0, m=3D0xc4df5acc,
>> >>> ifp=3D0xc5253c00, dir=3D1,
>> >>> =A0 =A0inp=3D0x0) at /usr/src/sys/modules/pf/../../contrib/pf/net/
>> >>> pf_ioctl.c:4139
>> >>> #10 0xc08c3c68 in pfil_run_hooks (ph=3D0xc0d685e0, mp=3D0xc4df5b24,
>> >>> =A0 =A0ifp=3D0xc5253c00, dir=3D1, inp=3D0x0) at /usr/src/sys/net/pfi=
l.c:82
>> >>> #11 0xc08e18ae in ip_input (m=3D0xc567db00)
>> >>> =A0 =A0at /usr/src/sys/netinet/ip_input.c:510
>> >>> #12 0xc08c2d9f in netisr_dispatch_src (proto=3D1, source=3D0,
>> >>> m=3D0xc567db00)
>> >>> =A0 =A0at /usr/src/sys/net/netisr.c:1013
>> >>> #13 0xc08c3040 in netisr_dispatch (proto=3D1, m=3D0xc567db00)
>> >>> =A0 =A0at /usr/src/sys/net/netisr.c:1104
>> >>> #14 0xc08b9721 in ether_demux (ifp=3D0xc5253c00, m=3D0xc567db00)
>> >>> =A0 =A0at /usr/src/sys/net/if_ethersubr.c:937
>> >>> #15 0xc08b9b6f in ether_nh_input (m=3D0xc567db00)
>> >>> =A0 =A0at /usr/src/sys/net/if_ethersubr.c:756
>> >>> #16 0xc08c2d9f in netisr_dispatch_src (proto=3D9, source=3D0,
>> >>> m=3D0xc567db00)
>> >>> =A0 =A0at /usr/src/sys/net/netisr.c:1013
>> >>> #17 0xc08c3040 in netisr_dispatch (proto=3D9, m=3D0xc567db00)
>> >>> =A0 =A0at /usr/src/sys/net/netisr.c:1104
>> >>> #18 0xc08b9269 in ether_input (ifp=3D0xc5253c00, m=3D0xc567db00)
>> >>> =A0 =A0at /usr/src/sys/net/if_ethersubr.c:797
>> >>> #19 0xc05b383f in em_rxeof (rxr=3D0xc520bc00, count=3D99, done=3D0x0=
)
>> >>> =A0 =A0at /usr/src/sys/dev/e1000/if_em.c:4340
>> >>> #20 0xc05b3a06 in em_msix_rx (arg=3D0xc520bc00)
>> >>> =A0 =A0at /usr/src/sys/dev/e1000/if_em.c:1577
>> >>> #21 0xc07da6eb in intr_event_execute_handlers (p=3D0xc5157588,
>> >>> ie=3D0xc5241680)
>> >>> =A0 =A0at /usr/src/sys/kern/kern_intr.c:1257
>> >>> #22 0xc07dbeaa in ithread_loop (arg=3D0xc52506e0)
>> >>> =A0 =A0at /usr/src/sys/kern/kern_intr.c:1270
>> >>> #23 0xc07d78f7 in fork_exit (callout=3D0xc07dbe30 <ithread_loop>,
>> >>> =A0 =A0arg=3D0xc52506e0, frame=3D0xc4df5d28) at /usr/src/sys/kern/
>> >>> kern_fork.c:995
>> >>> #24 0xc0ad6004 in fork_trampoline () at /usr/src/sys/i386/i386/
>> >>> exception.s:275
>> >>> (kgdb)
>> >>>
>> >>>
>> >>> ################## pf.conf ##################
>> >>> ext_if =3D "em0"
>> >>>
>> >>> public_tcp_ports =3D "{21,25,53,80,143,443,873,993,50021:50121}"
>> >>> public_udp_ports =3D "53"
>> >>>
>> >>> table <secure> {someip}
>> >>> table <ssh_brute> persist counters
>> >>>
>> >>> ### Redirection for SMTP
>> >>> rdr on $ext_if proto tcp from any to $ext_if port 225 -> $ext_if
>> >>> port 25
>> >>>
>> >>> ### Block everything in an pass everything out
>> >>> pass out on $ext_if all modulate state
>> >>> block in on $ext_if all
>> >>>
>> >>> ### secure users
>> >>> pass in quick on $ext_if proto tcp from <secure> to any flags
>> >>> S/SA \ modulate state
>> >>>
>> >>> ### public tcp/udp ports rules
>> >>> pass in on $ext_if proto udp to $ext_if port $public_udp_ports
>> >>> pass in on $ext_if proto tcp to $ext_if port $public_tcp_ports
>> >>> flags S/SA \
>> >>> modulate state
>> >>>
>> >>> ### block ssh bruteforce
>> >>> block in quick from <ssh_brute>
>> >>> pass in quick on $ext_if proto tcp to $ext_if port 22 flags S/SA
>> >>> modulate state \
>> >>> (max-src-conn 5, max-src-conn-rate 10/60, overload <ssh_brute>
>> >>> flush global)
>> >>>
>> >>> ### block icmp timestamp request/response
>> >>> block in quick on $ext_if inet proto icmp all icmp-type {13, 14}
>> >>> pass in quick on $ext_if proto icmp all
>> >>>
>> >>> ############ end pf.conf ##############
>> >>>
>> >>> --
>> >>> Ali Mdidech
>> >>> _______________________________________________
>> >>> freebsd-pf@freebsd.org mailing list
>> >>> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
>> >>> To unsubscribe, send any mail to "freebsd-pf-
>> >>> unsubscribe@freebsd.org"
>> >>
>> >>
>> >>
>> >> --
>> >> Ermal
>> >
>> > --
>> > Ali Mdidech
>> > _______________________________________________
>> > freebsd-pf@freebsd.org mailing list
>> > http://lists.freebsd.org/mailman/listinfo/freebsd-pf
>> > To unsubscribe, send any mail to
>> > "freebsd-pf-unsubscribe@freebsd.org"
>>
>> ????????? ??????????
>> --------------------------
>> ????????? ?????????????
>> ??? "???"
>
> --
> Theo
> _______________________________________________
> freebsd-pf@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org"



--=20
Ermal



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPBZQG352UtXQ%2B4STkHNwF0f3e4z_2q=0zoRSBDCTZFMAJRAQg>