Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 25 Mar 2009 09:59:43 +0100
From:      Sebastiaan van Erk <sebster@sebster.com>
To:        freebsd-pf@freebsd.org
Subject:   state mismatch/connection issues
Message-ID:  <49C9F27F.3010505@sebster.com>

next in thread | raw e-mail | index | archive | help

[-- Attachment #1 --]
Hi,

I'm running FreeBSD-7.0 RELEASE with the following patch to the kernel 
(I know it's integrated in the latest patchlevels which you get when you 
do freebsd-update, but since I'm still getting state-mismatches WITH the 
patch I'm holding off on the upgrade until I have more information as to 
the nature of the problem):

*** net/pf.c    2007/09/07 21:34:10     1.558
--- net/pf.c    2007/09/18 19:45:59     1.559
*************** pf_test_state_tcp(struct pf_state **state, int directi
*** 3730,3735 ****
--- 3730,3751 ----
                         REASON_SET(reason, PFRES_SYNPROXY);
                         return (PF_SYNPROXY_DROP);
                 }
+       }
+
+       if (((th->th_flags & (TH_SYN|TH_ACK)) == TH_SYN) &&
+           dst->state >= TCPS_FIN_WAIT_2 &&
+           src->state >= TCPS_FIN_WAIT_2) {
+               if (pf_status.debug >= PF_DEBUG_MISC) {
+                       printf("pf: state reuse ");
+                       pf_print_state(*state);
+                       pf_print_flags(th->th_flags);
+                       printf("\n");
+               }
+               /* XXX make sure it's the same direction ?? */
+               (*state)->src.state = (*state)->dst.state = TCPS_CLOSED;
+               pf_unlink_state(*state);
+               *state = NULL;
+               return (PF_DROP);
         }

         if (src->wscale && dst->wscale && !(th->th_flags & TH_SYN)) {


The problem I'm having is that I get intermittent connection 
refused/operation not permitted to another machine on the local network. 
When I do pfctl -s info I see *huge* numbers of state mismatches:

Status: Enabled for 94 days 01:27:40          Debug: Urgent

State Table                          Total             Rate
   current entries                      398
   searches                       986228319          121.4/s
   inserts                        104049508           12.8/s
   removals                       104049110           12.8/s
Counters
   match                          107482262           13.2/s
   bad-offset                             0            0.0/s
   fragment                               0            0.0/s
   short                                  0            0.0/s
   normalize                             42            0.0/s
   memory                           3125235            0.4/s
   bad-timestamp                          0            0.0/s
   congestion                             0            0.0/s
   ip-option                              0            0.0/s
   proto-cksum                        13919            0.0/s
   state-mismatch                   3039814            0.4/s
   state-insert                           0            0.0/s
   state-limit                            0            0.0/s
   src-limit                              0            0.0/s
   synproxy                               0            0.0/s

This is causing serious problems at them moment. It seems that the state 
problems occur in certain small time windows (my nagios starts reporting 
that every service is connection refused/operation not permitted, which 
is about 20 services). Then I get 20 recovery messages.

The firewall rules are trivially simple, $ext_if has 2 ips and $int_if 
has one:

interfaces = "{" $ext_if "," $int_if "}"

scrub in all
set skip on lo0
antispoof for $interfaces inet
block out log quick on $ext_if from !$ext_ip1 to any
block in quick on $ext_if from any to 255.255.255.255
block log all

pass in quick inet proto icmp all icmp-type $icmp_types

pass in quick on $int_if from $int_net to any
pass out quick on $int_if from any to $int_net

pass out on $ext_if proto tcp all
pass out on $ext_if proto { udp, icmp } all
pass in on $ext_if proto tcp from any to $ext_ip1 port $tcp_services1
pass in on $ext_if proto tcp from any to $ext_ip2 port $tcp_services2

Does anybody have any idea what's going on and where I can look? This is 
a production server so it's seriously influencing the quality of the 
hosted services. :-(


Regards,
Sebastiaan

[-- Attachment #2 --]
0	*H
010	+0	*H
	Q00lS|
6$1-~j0
	*H
0b10	UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA0
080630135157Z
090630135157Z0h10Uvan Erk10U*
Sebastiaan10USebastiaan van Erk1"0 	*H
	sebster@sebster.com0"0
	*H
0
Va\bEnݚa<M8ʄ^tv>x73bohi2oqS_¶Bm^p*I	x"9pt!jar#)n)^?'z<).+Ѐ4igR'UP*\Ւ,?.;?fBܯTzM IDվCK*3Yŧ
mcaztxʐsq/00.0U0sebster@sebster.com0U00
	*H
KT4W6ӽq]
tS` %f1G:HbzJj$EjE'JV~-VbVnJZE/`@@04!+T:c	پf`$Z=1#|oG[OBRG00lS|
6$1-~j0
	*H
0b10	UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA0
080630135157Z
090630135157Z0h10Uvan Erk10U*
Sebastiaan10USebastiaan van Erk1"0 	*H
	sebster@sebster.com0"0
	*H
0
Va\bEnݚa<M8ʄ^tv>x73bohi2oqS_¶Bm^p*I	x"9pt!jar#)n)^?'z<).+Ѐ4igR'UP*\Ւ,?.;?fBܯTzM IDվCK*3Yŧ
mcaztxʐsq/00.0U0sebster@sebster.com0U00
	*H
KT4W6ӽq]
tS` %f1G:HbzJj$EjE'JV~-VbVnJZE/`@@04!+T:c	پf`$Z=1#|oG[OBRG0?0
0
	*H
010	UZA10UWestern Cape10U	Cape Town10U
Thawte Consulting1(0&UCertification Services Division1$0"UThawte Personal Freemail CA1+0)	*H
	personal-freemail@thawte.com0
030717000000Z
130716235959Z0b10	UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA00
	*H
0Ħ<UsUNʙZhup[v:aQP
0cZ,p+Z?qV˯<6$*+w=+>@dקe*TH<a@dr`00U00CU<0:08642http://crl.thawte.com/ThawtePersonalFreemailCA.crl0U0)U"0 010UPrivateLabel2-1380
	*H
HP.
fgCL!6-6/P p<ab:~t%Pb'qW%ݩ9 Oe_N4[5MwV!x!5$F]_eO1q0m0v0b10	UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS|
6$1-~j0	+0	*H
	1	*H
0	*H
	1
090325085943Z0#	*H
	1k"Woq$RyZζSfC0_	*H
	1R0P0	`He0
*H
0*H
0
*H
@0+0
*H
(0	+71x0v0b10	UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS|
6$1-~j0*H
	1xv0b10	UZA1%0#U
Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS|
6$1-~j0
	*H
 +]RcoSY2tF^
C6x.ʚUUcJK)89HfSZC	,q#7KI"p7τBJ\1R{ӔZЬ߯̌}E	:olk G~bE` K7's80i*~Gȳ_*E&gQ:26CpmG4Zl*_cO߰?i>f\mm

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?49C9F27F.3010505>