Date: Tue, 17 Sep 2002 02:55:36 -0400 From: dfolkins <dfolkins@comcast.net> To: freebsd-security@freebsd.org Subject: Re: Unexpected keep state behaviour in ipfw Message-ID: <001a01c25e17$39edcde0$0a00a8c0@groovy3xp> References: <20020915224154.GD495@gsmx07.alcatel.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
> I've been putting together a firewall and bumped into an oddity in the
> ipfw keep-state behaviour. In particular, the TCP state information
> for connections through the firewall appears to be being lost so that
> the connection disappears. I've used keep-state in the past and I'm
> sure it has worked as expected. Also, my ssh connection from an
> internal host to the firewall host remains up even after extended
> periods of inactivity (like overnight). Either ipfw doesn't like a
> back-to-back keep-state configuration or I've done something silly but
> I can't see what.
>
> I was initially using ipfw1 in -STABLE from about a week ago. When I
> switched to ipfw2 in -STABLE as of last Friday, the problem remained.
> (In both cases, I did a complete build and installworld).
>
please note right away that i have no experience with ipfw2 and its
automatic keep-alive, my response is targeted to ipfw only.
first, your "idle time" for standard tcp connections is controlled by a
sysctl variable named net.inet.ip.fw.dyn_ack_lifetime. if it is set too
short (default is 300 seconds, i think) you can always just reset it to a
longer value, either from command prompt or from sysctl.conf. just set it
to a number you think appropriate for idle established connections to remain
active. e.g. if you want your idle ftp connections to stay alive for 10
minutes, set that variable to 600. there are a bunch of related variables.
to see them all just do a "sysctl -a |grep dyn".
as to why your ssh connection stays alive even through the night - i suspect
that is because your ssh server on your firewall has a configuration setting
that makes it send keep-alives, i.e. your clientaliveinterval in sshd_config
is set to some value which is less than your net.inet.ip.fw.dyn_ack_lifetime
value. that might be keeping your idle ssh connections up.
sorry, cant tell you anything about ipfw2's keepalive packets.
--
dfolkins
>
> Configuration:
>
> +-------+ +-------+ +-------+
> | | internal | | Internet | |
> | hosta |------------| fwall |------------| hostb |
> | | ed0| |ed1 | |
> +-------+ +-------+ +-------+
>
> fwall is using ipnat for address translation and was initially using
> both IPFilter and ipfw rules, but the problem remained after I changed
> IPFilter to do nothing (pass {in,out} all). [I wanted to use ipfw
> because I am more familiar with it and I wanted to use ipnat because
> the firewall box is relatively underpowered and I didn't want all
> the traffic to have to go through the kernel/userland interface to
> natd(8)].
>
> My ipfw rules were initially:
> 00100 allow ip from any to any via lo0
> 00200 deny ip from any to 127.0.0.0/8
> 00300 deny ip from 127.0.0.0/8 to any
> 00600 check-state
> 00650 deny log tcp from any to any established
> 01200 allow ip from internal-net/24 to any in recv ed0 keep-state
> 01300 skipto 30000 ip from any to internal-net/24 out xmit ed0
> 11000 allow tcp from me to any out xmit ed1 keep-state
> 11005 allow udp from me to any dst-port 53 out xmit ed1 keep-state
> 11005 allow udp from me to any dst-port 123 out xmit ed1
> 11005 allow udp from me 68 to any dst-port 67 out xmit ed1 keep-state
> 11005 allow icmp from me to any out xmit ed1 icmptypes 0,3,4,8,11,12
> 11005 allow icmp from any to me in recv ed1 icmptypes 0,3,4,8,11,12
> 11010 allow udp from any 123 to me in recv ed1
> 13000 deny log ip from any to any
> 30020 allow udp from me to any dst-port 123
> 30030 allow icmp from me to any icmptypes 0,3,4,8,11,12
> 31000 deny log ip from any to any
> 65535 deny ip from any to any
>
> I initially thought there might be a problem with duplicate FIN
> packets and added the following rule but it made no difference:
> 00610 allow tcp from any to any tcpflags fin
>
> The following is an example of the problem showing up on an FTP
> control connection. 'fwall-ed1' is the internet-visible address.
>
> /var/log/security:
> 07:08:02 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:08:03 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:08:05 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:08:10 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:08:20 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:08:40 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:09:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:10:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:11:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:12:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:13:20 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:14:14 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via
ed1
> 07:14:20 fwall /kernel: ipfw: 650 Deny TCP hostb:21 hosta:1112 in via ed1
>
> fwall# tcpdump -i ed0
> 07:04:01.863247 hosta.1112 > hostb.ftp: P 147:157(10) ack 662 win 57964
<nop,nop,timestamp 25730182 274805222> (DF) [tos 0x10]
> 07:04:02.104378 hostb.ftp > hosta.1112: P 662:715(53) ack 157 win 17524
<nop,nop,timestamp 274805270 25730182>
> 07:04:02.198438 hosta.1112 > hostb.ftp: . ack 715 win 57964
<nop,nop,timestamp 25730216 274805270> (DF) [tos 0x10]
> 07:04:02.428996 hostb.ftp > hosta.1112: P 715:745(30) ack 157 win 17524
<nop,nop,timestamp 274805302 25730216>
> 07:04:02.528507 hosta.1112 > hostb.ftp: . ack 745 win 57964
<nop,nop,timestamp 25730249 274805302> (DF) [tos 0x10]
> 07:08:02.190741 hostb.ftp > hosta.1112: P 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274829273 25730249>
> 07:08:02.291580 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25754220 274829273> (DF) [tos 0x10]
> 07:08:03.417151 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274829396 25730249>
> 07:08:03.417888 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25754332 274829396> (DF) [tos 0x10]
> 07:08:05.877990 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274829642 25730249>
> 07:08:05.878703 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25754578 274829642> (DF) [tos 0x10]
> 07:08:10.798902 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274830134 25730249>
> 07:08:10.799621 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25755070 274830134> (DF) [tos 0x10]
> 07:08:20.651400 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274831119 25730249>
> 07:08:20.652125 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25756055 274831119> (DF) [tos 0x10]
> 07:08:40.366279 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274833090 25730249>
> 07:08:40.367120 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25758026 274833090> (DF) [tos 0x10]
> 07:09:19.796673 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274837032 25730249>
> 07:09:19.797487 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25761968 274837032> (DF) [tos 0x10]
> 07:10:19.808705 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274843032 25730249>
> 07:10:19.809477 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25767968 274843032> (DF) [tos 0x10]
> 07:11:19.821934 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274849032 25730249>
> 07:11:19.822731 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25773968 274849032> (DF) [tos 0x10]
> 07:12:19.835383 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524
<nop,nop,timestamp 274855032 25730249>
> 07:12:19.836158 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25779968 274855032> (DF) [tos 0x10]
> 07:13:19.849184 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 <n
op,nop,timestamp 274861032 25730249>
> 07:13:19.850012 hosta.1112 > hostb.ftp: . ack 782 win 57927
<nop,nop,timestamp 25785968 274861032> (DF) [tos 0x10]
> 07:14:13.879800 hosta.1112 > hostb.ftp: P 157:163(6) ack 782 win 57927
<nop,nop,timestamp 25791370 274861032> (DF) [tos 0x10]
> 07:14:13.879993 hosta.1112 > hostb.ftp: F 163:163(0) ack 782 win 57964
<nop,nop,timestamp 25791370 274861032> (DF) [tos 0x10]
> 07:14:14.113211 hostb.ftp > hosta.1112: R 2194364317:2194364317(0) win 0
(DF)
>
> fwall# tcpdump -i ed1
> 07:04:02.103450 hostb.ftp > fwall-ed1.1112: P 662:715(53) ack 157 win
17524 <nop,nop,timestamp 274805270 25730182>
> 07:04:02.122875 hostb.59915 > fwall-ed1.1115: P 1:223(222) ack 1 win 17524
<nop,nop,timestamp 274805272 25730182>
> 07:04:02.122972 hostb.59915 > fwall-ed1.1115: F 223:223(0) ack 1 win 17524
<nop,nop,timestamp 274805272 25730182>
> 07:04:02.125752 fwall-ed1.1115 > hostb.59915: . ack 224 win 57964
<nop,nop,timestamp 25730208 274805272> (DF) [tos 0x8]
> 07:04:02.140270 fwall-ed1.1115 > hostb.59915: F 1:1(0) ack 224 win 57964
<nop,nop,timestamp 25730210 274805272> (DF) [tos 0x8]
> 07:04:02.199236 fwall-ed1.1112 > hostb.ftp: . ack 715 win 57964
<nop,nop,timestamp 25730216 274805270> (DF) [tos 0x10]
> 07:04:02.369724 hostb.59915 > fwall-ed1.1115: . ack 2 win 17524
<nop,nop,timestamp 274805296 25730210>
> 07:04:02.428304 hostb.ftp > fwall-ed1.1112: P 715:745(30) ack 157 win
17524 <nop,nop,timestamp 274805302 25730216>
> 07:04:02.529223 fwall-ed1.1112 > hostb.ftp: . ack 745 win 57964
<nop,nop,timestamp 25730249 274805302> (DF) [tos 0x10]
> 07:08:02.189388 hostb.ftp > fwall-ed1.1112: P 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274829273 25730249>
> 07:08:03.416245 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274829396 25730249>
> 07:08:05.876702 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274829642 25730249>
> 07:08:10.797991 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274830134 25730249>
> 07:08:20.650392 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274831119 25730249>
> 07:08:40.364907 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274833090 25730249>
> 07:09:19.795268 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274837032 25730249>
> 07:10:19.807132 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274843032 25730249>
> 07:11:19.820499 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274849032 25730249>
> 07:12:19.834029 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274855032 25730249>
> 07:13:19.847717 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274861032 25730249>
> 07:14:13.882752 fwall-ed1.1112 > hostb.ftp: F 163:163(0) ack 782 win 57964
<nop,nop,timestamp 25791370 274861032> (DF) [tos 0x10]
> 07:14:14.112270 hostb.ftp > fwall-ed1.1112: R 2194364317:2194364317(0) win
0 (DF)
> 07:14:19.861397 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274867032 25730249>
> 07:15:19.875126 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win
17524 <nop,nop,timestamp 274873032 25730249>
>
> Why does ipfw close the outgoing path from hosta to hostb after 4
> minutes of inactivity?
>
> According to the ipfw man page, ipfw2 will generate keep-alive packets
> when a keep-state rule is about to expire. Why don't I see any in the
> tcpdump traces?
>
> Peter
>
> To Unsubscribe: send mail to majordomo@FreeBSD.org
> with "unsubscribe freebsd-security" in the body of the message
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-security" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?001a01c25e17$39edcde0$0a00a8c0>