From owner-freebsd-current@freebsd.org Fri May 20 14:21:03 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57363B42519 for ; Fri, 20 May 2016 14:21:03 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0A3971A9C for ; Fri, 20 May 2016 14:21:02 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (envelope-from ) id <1b3lIS-001QoO-DK>; Fri, 20 May 2016 16:21:00 +0200 Received: from x4e3482fd.dyn.telefonica.de ([78.52.130.253] helo=thor.walstatt.dynvpn.de) by inpost2.zedat.fu-berlin.de (Exim 4.85) with esmtpsa (TLSv1.2:AES128-GCM-SHA256:128) (envelope-from ) id <1b3lIS-002GMC-2H>; Fri, 20 May 2016 16:21:00 +0200 Date: Fri, 20 May 2016 16:23:01 +0200 From: "O. Hartmann" To: Jan Bramkamp Cc: Vladimir Zakharov , freebsd-current@freebsd.org Subject: Re: CURRENT: ipfw: problems with timeouts and worse network performance Message-ID: <20160520162301.4ebbd679.ohartman@zedat.fu-berlin.de> In-Reply-To: <0429473a-6bb6-70cb-05ca-fe0e83c173a0@rlwinm.de> References: <20160520140152.3ab6fe44@hermann> <20160520125401.GC2371@vzakharov> <38888426-4d91-56ed-9ab3-0d516b0b8d46@rlwinm.de> <20160520135109.GD2371@vzakharov> <0429473a-6bb6-70cb-05ca-fe0e83c173a0@rlwinm.de> Organization: FU Berlin X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/dOSOhwJ+p8QGDvNvwO2xDoS"; protocol="application/pgp-signature" X-Originating-IP: 78.52.130.253 X-ZEDAT-Hint: A X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2016 14:21:03 -0000 --Sig_/dOSOhwJ+p8QGDvNvwO2xDoS Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Am Fri, 20 May 2016 16:01:17 +0200 Jan Bramkamp schrieb: > On 20/05/16 15:51, Vladimir Zakharov wrote: > > On Fri, May 20, 2016, Jan Bramkamp wrote: =20 > >> On 20/05/16 14:54, Vladimir Zakharov wrote: =20 > >>> Hello > >>> > >>> On Fri, May 20, 2016, O. Hartmann wrote: =20 > >>>> I reported earlier about broken pipes in ssh sessions to remote host= s, > >>>> which occur on an erratic basis. i'm investigating this problem now = and > >>>> it seems that it is also ipfw-related, but I'm not sure. This problem > >>>> is present since a couple of weeks now. =20 > >>> > >>> Maybe this could help... > >>> > >>> I've also experienced problems with broken pipes in ssh sessions some > >>> time ago. Setting in sysctl.conf > >>> > >>> net.inet.ip.fw.dyn_ack_lifetime=3D3600 > >>> > >>> fixed problem for me. I didn't experiment with the value though. So, > >>> possibly, changing default value (300s) to 1 hour is overkill :). =20 > >> > >> By default the OpenSSH SSH client is configured to use TCP keepalives. > >> Those should produce enough packets at a short enough interval to keep > >> the dynamic IPFW state established. > >> > >> Does your traffic pass through libalias? =20 > > I guess not. How can I be sure? =20 >=20 > Libalias is used by ipfw and the old userland natd to implement IPv4=20 > NAT. It requires unmodified access to all packets including their=20 > headers. LRO and TSO coalesce packets to reduce save CPU time but the=20 > process is loses some of the information required by libalias. Unless=20 > your ruleset uses ipfw in-kernel NAT or diverts traffic to natd you=20 > don't have to worry about libalias. >=20 > Use `kldstat -v | grep libalias` to check for libalias in the running=20 > kernel and `pgrep natd` to search for running natd instances. As I replied earlier, in my case it is manyfolded - I use In-kernel-NAT as = well as straight forward filtering. The problem of broken pipes in ssh occured simultanously on ALL CURRENT box= es (a bunch of different age, CPU types, NICs and configs, but all most recent CURRENT) at= the same time. Those massive dropouts and timeouts I whitnessed now occured after we updat= ed from ~CURRENT 300005 to CURRENT 300158. On some experimental servers, the config= , especially that of ipfw, has not changed over half a year by now - but they suffer als= o from the problem I described and the problem can be solved by disabling IPFW. The problem is simpel to trigger: have firewall type "WORKSTATION" configur= ed, IPFW active (I have IPFW statically in-kernel-compiled, no modules so far). Have= /usr/src as a svn+https repository and try a svn update of the source tree. I haven't checked so far whether the problem occurs also with non-SSL conne= ctions since all connections I see the suffering are somehow encrypted. Regards, Oliver --Sig_/dOSOhwJ+p8QGDvNvwO2xDoS Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJXPx3FAAoJEOgBcD7A/5N8hWYIALOrBHGgH/IiaK/a43Vd9IyL I4/cwPX/sKCqQ5V/73ZlVkXPvp5Jqa09PuuxuxCOUOef+hg+4lIFOUu69Jgzb9yL Cu9OMXl7bgSehv+J66ONtaSlj98t+ALDqKnjZRZuOgLYQkaL9AWHuv5nTvV0bNjd Mpodpr3U7HNMNKiNkfBIsDqh2ebTYCdX0+e+O/Ddu3MRtE1fzyCawiwEWuNcVlG9 /WKulzTBF95TA2a3FersAjreqgnSXqV+VjCqLS31IdNZkG6OYmVYanD35zM6ld2a nVDcTEOGnp0jni/vuT/dwtDqmVTeUdUKI+G9p+S1bqTSSYcUvzEPyG1Vjk/DDy8= =zM3+ -----END PGP SIGNATURE----- --Sig_/dOSOhwJ+p8QGDvNvwO2xDoS--