Date: Mon, 16 Sep 2002 08:41:54 +1000 From: Peter Jeremy <peter.jeremy@alcatel.com.au> To: freebsd-security@freebsd.org Subject: Unexpected keep state behaviour in ipfw Message-ID: <20020915224154.GD495@gsmx07.alcatel.com.au>
next in thread | raw e-mail | index | archive | help
I've been putting together a firewall and bumped into an oddity in the
ipfw keep-state behaviour. In particular, the TCP state information
for connections through the firewall appears to be being lost so that
the connection disappears. I've used keep-state in the past and I'm
sure it has worked as expected. Also, my ssh connection from an
internal host to the firewall host remains up even after extended
periods of inactivity (like overnight). Either ipfw doesn't like a
back-to-back keep-state configuration or I've done something silly but
I can't see what.
I was initially using ipfw1 in -STABLE from about a week ago. When I
switched to ipfw2 in -STABLE as of last Friday, the problem remained.
(In both cases, I did a complete build and installworld).
Configuration:
+-------+ +-------+ +-------+
| | internal | | Internet | |
| hosta |------------| fwall |------------| hostb |
| | ed0| |ed1 | |
+-------+ +-------+ +-------+
fwall is using ipnat for address translation and was initially using
both IPFilter and ipfw rules, but the problem remained after I changed
IPFilter to do nothing (pass {in,out} all). [I wanted to use ipfw
because I am more familiar with it and I wanted to use ipnat because
the firewall box is relatively underpowered and I didn't want all
the traffic to have to go through the kernel/userland interface to
natd(8)].
My ipfw rules were initially:
00100 allow ip from any to any via lo0
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00600 check-state
00650 deny log tcp from any to any established
01200 allow ip from internal-net/24 to any in recv ed0 keep-state
01300 skipto 30000 ip from any to internal-net/24 out xmit ed0
11000 allow tcp from me to any out xmit ed1 keep-state
11005 allow udp from me to any dst-port 53 out xmit ed1 keep-state
11005 allow udp from me to any dst-port 123 out xmit ed1
11005 allow udp from me 68 to any dst-port 67 out xmit ed1 keep-state
11005 allow icmp from me to any out xmit ed1 icmptypes 0,3,4,8,11,12
11005 allow icmp from any to me in recv ed1 icmptypes 0,3,4,8,11,12
11010 allow udp from any 123 to me in recv ed1
13000 deny log ip from any to any
30020 allow udp from me to any dst-port 123
30030 allow icmp from me to any icmptypes 0,3,4,8,11,12
31000 deny log ip from any to any
65535 deny ip from any to any
I initially thought there might be a problem with duplicate FIN
packets and added the following rule but it made no difference:
00610 allow tcp from any to any tcpflags fin
The following is an example of the problem showing up on an FTP
control connection. 'fwall-ed1' is the internet-visible address.
/var/log/security:
07:08:02 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:08:03 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:08:05 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:08:10 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:08:20 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:08:40 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:09:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:10:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:11:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:12:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:13:20 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:14:14 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1
07:14:20 fwall /kernel: ipfw: 650 Deny TCP hostb:21 hosta:1112 in via ed1
fwall# tcpdump -i ed0
07:04:01.863247 hosta.1112 > hostb.ftp: P 147:157(10) ack 662 win 57964 <nop,nop,timestamp 25730182 274805222> (DF) [tos 0x10]
07:04:02.104378 hostb.ftp > hosta.1112: P 662:715(53) ack 157 win 17524 <nop,nop,timestamp 274805270 25730182>
07:04:02.198438 hosta.1112 > hostb.ftp: . ack 715 win 57964 <nop,nop,timestamp 25730216 274805270> (DF) [tos 0x10]
07:04:02.428996 hostb.ftp > hosta.1112: P 715:745(30) ack 157 win 17524 <nop,nop,timestamp 274805302 25730216>
07:04:02.528507 hosta.1112 > hostb.ftp: . ack 745 win 57964 <nop,nop,timestamp 25730249 274805302> (DF) [tos 0x10]
07:08:02.190741 hostb.ftp > hosta.1112: P 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274829273 25730249>
07:08:02.291580 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25754220 274829273> (DF) [tos 0x10]
07:08:03.417151 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274829396 25730249>
07:08:03.417888 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25754332 274829396> (DF) [tos 0x10]
07:08:05.877990 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274829642 25730249>
07:08:05.878703 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25754578 274829642> (DF) [tos 0x10]
07:08:10.798902 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274830134 25730249>
07:08:10.799621 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25755070 274830134> (DF) [tos 0x10]
07:08:20.651400 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274831119 25730249>
07:08:20.652125 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25756055 274831119> (DF) [tos 0x10]
07:08:40.366279 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274833090 25730249>
07:08:40.367120 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25758026 274833090> (DF) [tos 0x10]
07:09:19.796673 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274837032 25730249>
07:09:19.797487 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25761968 274837032> (DF) [tos 0x10]
07:10:19.808705 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274843032 25730249>
07:10:19.809477 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25767968 274843032> (DF) [tos 0x10]
07:11:19.821934 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274849032 25730249>
07:11:19.822731 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25773968 274849032> (DF) [tos 0x10]
07:12:19.835383 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274855032 25730249>
07:12:19.836158 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25779968 274855032> (DF) [tos 0x10]
07:13:19.849184 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274861032 25730249>
07:13:19.850012 hosta.1112 > hostb.ftp: . ack 782 win 57927 <nop,nop,timestamp 25785968 274861032> (DF) [tos 0x10]
07:14:13.879800 hosta.1112 > hostb.ftp: P 157:163(6) ack 782 win 57927 <nop,nop,timestamp 25791370 274861032> (DF) [tos 0x10]
07:14:13.879993 hosta.1112 > hostb.ftp: F 163:163(0) ack 782 win 57964 <nop,nop,timestamp 25791370 274861032> (DF) [tos 0x10]
07:14:14.113211 hostb.ftp > hosta.1112: R 2194364317:2194364317(0) win 0 (DF)
fwall# tcpdump -i ed1
07:04:02.103450 hostb.ftp > fwall-ed1.1112: P 662:715(53) ack 157 win 17524 <nop,nop,timestamp 274805270 25730182>
07:04:02.122875 hostb.59915 > fwall-ed1.1115: P 1:223(222) ack 1 win 17524 <nop,nop,timestamp 274805272 25730182>
07:04:02.122972 hostb.59915 > fwall-ed1.1115: F 223:223(0) ack 1 win 17524 <nop,nop,timestamp 274805272 25730182>
07:04:02.125752 fwall-ed1.1115 > hostb.59915: . ack 224 win 57964 <nop,nop,timestamp 25730208 274805272> (DF) [tos 0x8]
07:04:02.140270 fwall-ed1.1115 > hostb.59915: F 1:1(0) ack 224 win 57964 <nop,nop,timestamp 25730210 274805272> (DF) [tos 0x8]
07:04:02.199236 fwall-ed1.1112 > hostb.ftp: . ack 715 win 57964 <nop,nop,timestamp 25730216 274805270> (DF) [tos 0x10]
07:04:02.369724 hostb.59915 > fwall-ed1.1115: . ack 2 win 17524 <nop,nop,timestamp 274805296 25730210>
07:04:02.428304 hostb.ftp > fwall-ed1.1112: P 715:745(30) ack 157 win 17524 <nop,nop,timestamp 274805302 25730216>
07:04:02.529223 fwall-ed1.1112 > hostb.ftp: . ack 745 win 57964 <nop,nop,timestamp 25730249 274805302> (DF) [tos 0x10]
07:08:02.189388 hostb.ftp > fwall-ed1.1112: P 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274829273 25730249>
07:08:03.416245 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274829396 25730249>
07:08:05.876702 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274829642 25730249>
07:08:10.797991 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274830134 25730249>
07:08:20.650392 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274831119 25730249>
07:08:40.364907 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274833090 25730249>
07:09:19.795268 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274837032 25730249>
07:10:19.807132 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274843032 25730249>
07:11:19.820499 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274849032 25730249>
07:12:19.834029 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274855032 25730249>
07:13:19.847717 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274861032 25730249>
07:14:13.882752 fwall-ed1.1112 > hostb.ftp: F 163:163(0) ack 782 win 57964 <nop,nop,timestamp 25791370 274861032> (DF) [tos 0x10]
07:14:14.112270 hostb.ftp > fwall-ed1.1112: R 2194364317:2194364317(0) win 0 (DF)
07:14:19.861397 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274867032 25730249>
07:15:19.875126 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 <nop,nop,timestamp 274873032 25730249>
Why does ipfw close the outgoing path from hosta to hostb after 4
minutes of inactivity?
According to the ipfw man page, ipfw2 will generate keep-alive packets
when a keep-state rule is about to expire. Why don't I see any in the
tcpdump traces?
Peter
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-security" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020915224154.GD495>