From owner-freebsd-security Mon Sep 16 23:55:53 2002 Delivered-To: freebsd-security@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8B94C37B400 for ; Mon, 16 Sep 2002 23:55:46 -0700 (PDT) Received: from smtp.comcast.net (smtp.comcast.net [24.153.64.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id E490743E3B for ; Mon, 16 Sep 2002 23:55:45 -0700 (PDT) (envelope-from dfolkins@comcast.net) Disposition-notification-to: dfolkins@comcast.net Received: from groovy3xp (pcp01731796pcs.selrsv01.pa.comcast.net [68.83.131.193]) by mtaout01.icomcast.net (iPlanet Messaging Server 5.1 HotFix 1.4 (built Aug 5 2002)) with SMTP id <0H2K00A4DLWWFK@mtaout01.icomcast.net> for freebsd-security@freebsd.org; Tue, 17 Sep 2002 02:55:45 -0400 (EDT) Date: Tue, 17 Sep 2002 02:55:36 -0400 From: dfolkins Subject: Re: Unexpected keep state behaviour in ipfw To: freebsd-security@freebsd.org Message-id: <001a01c25e17$39edcde0$0a00a8c0@groovy3xp> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-Mailer: Microsoft Outlook Express 6.00.2600.0000 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7BIT X-Priority: 3 X-MSMail-priority: Normal References: <20020915224154.GD495@gsmx07.alcatel.com.au> Sender: owner-freebsd-security@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > I've been putting together a firewall and bumped into an oddity in the > ipfw keep-state behaviour. In particular, the TCP state information > for connections through the firewall appears to be being lost so that > the connection disappears. I've used keep-state in the past and I'm > sure it has worked as expected. Also, my ssh connection from an > internal host to the firewall host remains up even after extended > periods of inactivity (like overnight). Either ipfw doesn't like a > back-to-back keep-state configuration or I've done something silly but > I can't see what. > > I was initially using ipfw1 in -STABLE from about a week ago. When I > switched to ipfw2 in -STABLE as of last Friday, the problem remained. > (In both cases, I did a complete build and installworld). > please note right away that i have no experience with ipfw2 and its automatic keep-alive, my response is targeted to ipfw only. first, your "idle time" for standard tcp connections is controlled by a sysctl variable named net.inet.ip.fw.dyn_ack_lifetime. if it is set too short (default is 300 seconds, i think) you can always just reset it to a longer value, either from command prompt or from sysctl.conf. just set it to a number you think appropriate for idle established connections to remain active. e.g. if you want your idle ftp connections to stay alive for 10 minutes, set that variable to 600. there are a bunch of related variables. to see them all just do a "sysctl -a |grep dyn". as to why your ssh connection stays alive even through the night - i suspect that is because your ssh server on your firewall has a configuration setting that makes it send keep-alives, i.e. your clientaliveinterval in sshd_config is set to some value which is less than your net.inet.ip.fw.dyn_ack_lifetime value. that might be keeping your idle ssh connections up. sorry, cant tell you anything about ipfw2's keepalive packets. -- dfolkins > > Configuration: > > +-------+ +-------+ +-------+ > | | internal | | Internet | | > | hosta |------------| fwall |------------| hostb | > | | ed0| |ed1 | | > +-------+ +-------+ +-------+ > > fwall is using ipnat for address translation and was initially using > both IPFilter and ipfw rules, but the problem remained after I changed > IPFilter to do nothing (pass {in,out} all). [I wanted to use ipfw > because I am more familiar with it and I wanted to use ipnat because > the firewall box is relatively underpowered and I didn't want all > the traffic to have to go through the kernel/userland interface to > natd(8)]. > > My ipfw rules were initially: > 00100 allow ip from any to any via lo0 > 00200 deny ip from any to 127.0.0.0/8 > 00300 deny ip from 127.0.0.0/8 to any > 00600 check-state > 00650 deny log tcp from any to any established > 01200 allow ip from internal-net/24 to any in recv ed0 keep-state > 01300 skipto 30000 ip from any to internal-net/24 out xmit ed0 > 11000 allow tcp from me to any out xmit ed1 keep-state > 11005 allow udp from me to any dst-port 53 out xmit ed1 keep-state > 11005 allow udp from me to any dst-port 123 out xmit ed1 > 11005 allow udp from me 68 to any dst-port 67 out xmit ed1 keep-state > 11005 allow icmp from me to any out xmit ed1 icmptypes 0,3,4,8,11,12 > 11005 allow icmp from any to me in recv ed1 icmptypes 0,3,4,8,11,12 > 11010 allow udp from any 123 to me in recv ed1 > 13000 deny log ip from any to any > 30020 allow udp from me to any dst-port 123 > 30030 allow icmp from me to any icmptypes 0,3,4,8,11,12 > 31000 deny log ip from any to any > 65535 deny ip from any to any > > I initially thought there might be a problem with duplicate FIN > packets and added the following rule but it made no difference: > 00610 allow tcp from any to any tcpflags fin > > The following is an example of the problem showing up on an FTP > control connection. 'fwall-ed1' is the internet-visible address. > > /var/log/security: > 07:08:02 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:08:03 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:08:05 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:08:10 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:08:20 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:08:40 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:09:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:10:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:11:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:12:19 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:13:20 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:14:14 fwall /kernel: ipfw: 650 Deny TCP fwall-ed1:1112 hostb:21 out via ed1 > 07:14:20 fwall /kernel: ipfw: 650 Deny TCP hostb:21 hosta:1112 in via ed1 > > fwall# tcpdump -i ed0 > 07:04:01.863247 hosta.1112 > hostb.ftp: P 147:157(10) ack 662 win 57964 (DF) [tos 0x10] > 07:04:02.104378 hostb.ftp > hosta.1112: P 662:715(53) ack 157 win 17524 > 07:04:02.198438 hosta.1112 > hostb.ftp: . ack 715 win 57964 (DF) [tos 0x10] > 07:04:02.428996 hostb.ftp > hosta.1112: P 715:745(30) ack 157 win 17524 > 07:04:02.528507 hosta.1112 > hostb.ftp: . ack 745 win 57964 (DF) [tos 0x10] > 07:08:02.190741 hostb.ftp > hosta.1112: P 745:782(37) ack 157 win 17524 > 07:08:02.291580 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:08:03.417151 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:08:03.417888 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:08:05.877990 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:08:05.878703 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:08:10.798902 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:08:10.799621 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:08:20.651400 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:08:20.652125 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:08:40.366279 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:08:40.367120 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:09:19.796673 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:09:19.797487 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:10:19.808705 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:10:19.809477 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:11:19.821934 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:11:19.822731 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:12:19.835383 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:12:19.836158 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:13:19.849184 hostb.ftp > hosta.1112: . 745:782(37) ack 157 win 17524 > 07:13:19.850012 hosta.1112 > hostb.ftp: . ack 782 win 57927 (DF) [tos 0x10] > 07:14:13.879800 hosta.1112 > hostb.ftp: P 157:163(6) ack 782 win 57927 (DF) [tos 0x10] > 07:14:13.879993 hosta.1112 > hostb.ftp: F 163:163(0) ack 782 win 57964 (DF) [tos 0x10] > 07:14:14.113211 hostb.ftp > hosta.1112: R 2194364317:2194364317(0) win 0 (DF) > > fwall# tcpdump -i ed1 > 07:04:02.103450 hostb.ftp > fwall-ed1.1112: P 662:715(53) ack 157 win 17524 > 07:04:02.122875 hostb.59915 > fwall-ed1.1115: P 1:223(222) ack 1 win 17524 > 07:04:02.122972 hostb.59915 > fwall-ed1.1115: F 223:223(0) ack 1 win 17524 > 07:04:02.125752 fwall-ed1.1115 > hostb.59915: . ack 224 win 57964 (DF) [tos 0x8] > 07:04:02.140270 fwall-ed1.1115 > hostb.59915: F 1:1(0) ack 224 win 57964 (DF) [tos 0x8] > 07:04:02.199236 fwall-ed1.1112 > hostb.ftp: . ack 715 win 57964 (DF) [tos 0x10] > 07:04:02.369724 hostb.59915 > fwall-ed1.1115: . ack 2 win 17524 > 07:04:02.428304 hostb.ftp > fwall-ed1.1112: P 715:745(30) ack 157 win 17524 > 07:04:02.529223 fwall-ed1.1112 > hostb.ftp: . ack 745 win 57964 (DF) [tos 0x10] > 07:08:02.189388 hostb.ftp > fwall-ed1.1112: P 745:782(37) ack 157 win 17524 > 07:08:03.416245 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:08:05.876702 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:08:10.797991 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:08:20.650392 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:08:40.364907 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:09:19.795268 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:10:19.807132 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:11:19.820499 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:12:19.834029 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:13:19.847717 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:14:13.882752 fwall-ed1.1112 > hostb.ftp: F 163:163(0) ack 782 win 57964 (DF) [tos 0x10] > 07:14:14.112270 hostb.ftp > fwall-ed1.1112: R 2194364317:2194364317(0) win 0 (DF) > 07:14:19.861397 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > 07:15:19.875126 hostb.ftp > fwall-ed1.1112: . 745:782(37) ack 157 win 17524 > > Why does ipfw close the outgoing path from hosta to hostb after 4 > minutes of inactivity? > > According to the ipfw man page, ipfw2 will generate keep-alive packets > when a keep-state rule is about to expire. Why don't I see any in the > tcpdump traces? > > Peter > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-security" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-security" in the body of the message