Date: Wed, 01 Apr 2009 11:45:43 +0200 From: Sebastiaan van Erk <sebster@sebster.com> To: freebsd-pf@freebsd.org Subject: Re: state mismatch/connection issues Message-ID: <49D337C7.9020707@sebster.com> In-Reply-To: <49C9F27F.3010505@sebster.com> References: <49C9F27F.3010505@sebster.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Hi,
I upgrade to the latest FreeBSD-7.0 release using freebsd-update, with
kernel 7.0-RELEASE-p11.
I still get massive amounts of state mismatches and intermittent
connection problems (connection refused, operation not permitted) with
outging connections....
My firewall rules are unchanged (see below), the stats are now:
Status: Enabled for 3 days 21:29:15 Debug: Urgent
State Table Total Rate
current entries 1994
searches 33567431 99.7/s
inserts 4611322 13.7/s
removals 4609328 13.7/s
Counters
match 6170429 18.3/s
bad-offset 0 0.0/s
fragment 0 0.0/s
short 0 0.0/s
normalize 1 0.0/s
memory 1516667 4.5/s
bad-timestamp 0 0.0/s
congestion 0 0.0/s
ip-option 0 0.0/s
proto-cksum 247 0.0/s
state-mismatch 1438892 4.3/s
state-insert 0 0.0/s
state-limit 0 0.0/s
src-limit 0 0.0/s
synproxy 0 0.0/s
Does anybody have *any* clue what's going on, and how I can go about
fixing it?
Thanks in advance,
Sebastiaan
Sebastiaan van Erk wrote:
> Hi,
>
> I'm running FreeBSD-7.0 RELEASE with the following patch to the kernel
> (I know it's integrated in the latest patchlevels which you get when you
> do freebsd-update, but since I'm still getting state-mismatches WITH the
> patch I'm holding off on the upgrade until I have more information as to
> the nature of the problem):
>
> *** net/pf.c 2007/09/07 21:34:10 1.558
> --- net/pf.c 2007/09/18 19:45:59 1.559
> *************** pf_test_state_tcp(struct pf_state **state, int directi
> *** 3730,3735 ****
> --- 3730,3751 ----
> REASON_SET(reason, PFRES_SYNPROXY);
> return (PF_SYNPROXY_DROP);
> }
> + }
> +
> + if (((th->th_flags & (TH_SYN|TH_ACK)) == TH_SYN) &&
> + dst->state >= TCPS_FIN_WAIT_2 &&
> + src->state >= TCPS_FIN_WAIT_2) {
> + if (pf_status.debug >= PF_DEBUG_MISC) {
> + printf("pf: state reuse ");
> + pf_print_state(*state);
> + pf_print_flags(th->th_flags);
> + printf("\n");
> + }
> + /* XXX make sure it's the same direction ?? */
> + (*state)->src.state = (*state)->dst.state = TCPS_CLOSED;
> + pf_unlink_state(*state);
> + *state = NULL;
> + return (PF_DROP);
> }
>
> if (src->wscale && dst->wscale && !(th->th_flags & TH_SYN)) {
>
>
> The problem I'm having is that I get intermittent connection
> refused/operation not permitted to another machine on the local network.
> When I do pfctl -s info I see *huge* numbers of state mismatches:
>
> Status: Enabled for 94 days 01:27:40 Debug: Urgent
>
> State Table Total Rate
> current entries 398
> searches 986228319 121.4/s
> inserts 104049508 12.8/s
> removals 104049110 12.8/s
> Counters
> match 107482262 13.2/s
> bad-offset 0 0.0/s
> fragment 0 0.0/s
> short 0 0.0/s
> normalize 42 0.0/s
> memory 3125235 0.4/s
> bad-timestamp 0 0.0/s
> congestion 0 0.0/s
> ip-option 0 0.0/s
> proto-cksum 13919 0.0/s
> state-mismatch 3039814 0.4/s
> state-insert 0 0.0/s
> state-limit 0 0.0/s
> src-limit 0 0.0/s
> synproxy 0 0.0/s
>
> This is causing serious problems at them moment. It seems that the state
> problems occur in certain small time windows (my nagios starts reporting
> that every service is connection refused/operation not permitted, which
> is about 20 services). Then I get 20 recovery messages.
>
> The firewall rules are trivially simple, $ext_if has 2 ips and $int_if
> has one:
>
> interfaces = "{" $ext_if "," $int_if "}"
>
> scrub in all
> set skip on lo0
> antispoof for $interfaces inet
> block out log quick on $ext_if from !$ext_ip1 to any
> block in quick on $ext_if from any to 255.255.255.255
> block log all
>
> pass in quick inet proto icmp all icmp-type $icmp_types
>
> pass in quick on $int_if from $int_net to any
> pass out quick on $int_if from any to $int_net
>
> pass out on $ext_if proto tcp all
> pass out on $ext_if proto { udp, icmp } all
> pass in on $ext_if proto tcp from any to $ext_ip1 port $tcp_services1
> pass in on $ext_if proto tcp from any to $ext_ip2 port $tcp_services2
>
> Does anybody have any idea what's going on and where I can look? This is
> a production server so it's seriously influencing the quality of the
> hosted services. :-(
>
>
> Regards,
> Sebastiaan
[-- Attachment #2 --]
0 *H
01 0 + 0 *H
Q00lS|
6$1-~j0
*H
0b1 0 UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA0
080630135157Z
090630135157Z0h10Uvan Erk10U*
Sebastiaan10USebastiaan van Erk1"0 *H
sebster@sebster.com0"0
*H
0
Va\bEnݚa<M8ʄ^tv>x73bohi2oqS_¶Bm^p* I x"9 pt!j ar#)n)^? 'z<).+Ѐ4ig R'UP*\Ւ ,?.;?f BܯTzM IDվCK*3Yŧ
mcaztxʐsq/ 00.0 U 0sebster@sebster.com0 U 0 0
*H
KT4W6ӽq]
tS` %f1 G:H b zJj$EjE'JV~-VbVnJZ E/`@@04!+T: c پf`$Z=1#|oG[OB RG00lS|
6$1-~j0
*H
0b1 0 UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA0
080630135157Z
090630135157Z0h10Uvan Erk10U*
Sebastiaan10USebastiaan van Erk1"0 *H
sebster@sebster.com0"0
*H
0
Va\bEnݚa<M8ʄ^tv>x73bohi2oqS_¶Bm^p* I x"9 pt!j ar#)n)^? 'z<).+Ѐ4ig R'UP*\Ւ ,?.;?f BܯTzM IDվCK*3Yŧ
mcaztxʐsq/ 00.0 U 0sebster@sebster.com0 U 0 0
*H
KT4W6ӽq]
tS` %f1 G:H b zJj$EjE'JV~-VbVnJZ E/`@@04!+T: c پf`$Z=1#|oG[OB RG0?0
0
*H
01 0 UZA10U Western Cape10U Cape Town10U
Thawte Consulting1(0&U Certification Services Division1$0"UThawte Personal Freemail CA1+0) *H
personal-freemail@thawte.com0
030717000000Z
130716235959Z0b1 0 UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA00
*H
0 Ħ<UsUNʙZ hup [v :aQP
0cZ,p+Z?qV˯< 6$*+w=+>@dקe*TH<a@ dr` 00U 0 0CU <0:08642http://crl.thawte.com/ThawtePersonalFreemailCA.crl0 U 0)U "0 0 10UPrivateLabel2-1380
*H
HP .
fgCL!6 -6/ P p<ab:~ t%Pb'qW%ݩ9 Oe_N4[5MwV!x!5$ F]_eO1q0m0v0b1 0 UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS|
6$1-~j0 + 0 *H
1 *H
0 *H
1
090401094543Z0# *H
1+k^,BR 00_ *H
1R0P0 `He0
*H
0*H
0
*H
@0+0
*H
(0 +71x0v0b1 0 UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS|
6$1-~j0 *H
1xv0b1 0 UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS|
6$1-~j0
*H
S^
Xhm ;
ЛEn+NGS},j҃Ȟ (! ɼl]Nt0v֥?*Ir c{۰nE?yˉۑky Hm712j=1h8LO
+d7+l'ġkeձuDܕ =$ 4>BYJ[4S!4#c\b(b ϾyʜpHO *x7X VVx 7]^
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?49D337C7.9020707>