Date: Thu, 28 May 2009 12:02:40 -0400 From: Martin Turgeon <freebsd@optiksecurite.com> To: freebsd-pf@freebsd.org Subject: State Mismatch and tcp.closed Message-ID: <4A1EB5A0.7030206@optiksecurite.com>
next in thread | raw e-mail | index | archive | help
Hi list! I had a problem with state mismatch on my DB server that I solved by lowering the tcp.closed timeout. I setted it to 2 instead of 90. I now have what looks like the same problem on the front-end web server. However, when I tried to apply the same fix, I got connection problem with the back-end DB, but the state mismatch disappearred. On the front-end web server, the state mismatch occurs on the external interface, only on port 80. I enabled misc debugging and got this in /var/log/messages on the front-end web server: May 28 05:02:19 francis kernel: pf: BAD state: TCP 127.0.0.25:80 206.125.166.65:80 98.207.239.10:54737 [lo=820536733 high=820603340 win=65535 modulator=0 wscale=0] [lo=2871317100 high=2871375106 win=8326 modulator=0 wscale=3] 7:4 R seq=820536733 (820536732) ack=2871317100 len=0 ackskew=0 pkts=43:69 dir=in,fwd May 28 05:02:19 francis kernel: pf: State failure on: | May 28 05:02:19 francis kernel: pf: BAD state: TCP 127.0.0.25:80 206.125.166.65:80 98.207.239.10:54733 [lo=374985971 high=375052578 win=65535 modulator=0 wscale=0] [lo=2999164748 high=2999229169 win=8326 modulator=0 wscale=3] 7:4 R seq=374985971 (374985970) ack=2999164748 len=0 ackskew=0 pkts=40:54 dir=in,fwd May 28 05:02:19 francis kernel: pf: State failure on: | May 28 05:03:06 francis kernel: pf: BAD state: TCP 127.0.0.20:80 206.125.166.80:80 123.116.84.41:59776 [lo=3407758259 high=3407823796 win=4096 modulator=0 wscale=2] [lo=374200006 high=374216390 win=8192 modulator=0 wscale=3] 4:2 A seq=3407758259 (3407758260) ack=2320196160 len=0 ackskew=-1945996154 pkts=1:1 dir=in,fwd May 28 05:03:06 francis kernel: pf: State failure on: 3 | May 28 05:03:06 francis kernel: pf: BAD state: TCP 127.0.0.20:80 206.125.166.80:80 123.116.84.41:59776 [lo=3407758259 high=3407823796 win=4096 modulator=0 wscale=2] [lo=374200006 high=374216390 win=8192 modulator=0 wscale=3] 4:2 RA seq=3407758259 (3407758260) ack=2320196160 len=0 ackskew=-1945996154 pkts=1:1 dir=in,fwd This server has been up for 12 days and already got almost 600000 state mismatch! I tried to lower tcp.finwait, no result. I tried to set optimization to aggressive, no result. I tried to disable port randomization via sysctl, no result either. I tcpdumped and there is only a few RST so I don't understand why tcp.closed would solve my problem. If it's a problem with source port reuse, tcp.finwait should be the timeout that would help, not tcp.closed, right? How can a lower tcp.closed on the front-end cause mysql connection problem with the back-end? I tcpdumped while there is a connection problem with the DB and there is nothing that seems wrong, no RST at all! The front-end web server tries to connect to the DB, wait 3 sec and if it fails to establish a connection, it then tries to connect to a read-only backup DB, on another server, which never fails to connect. The only thing I'm sure is that it's the tcp.closed that cause the DB connection problem. As soon as I remove it, the state mismatch comes back on the external interface but there's no DB connection problem anymore. What am I missing? Martin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4A1EB5A0.7030206>