Date: Thu, 11 Feb 2021 17:08:43 +0200 From: Andriy Gapon <avg@FreeBSD.org> To: net@FreeBSD.org Cc: "Andrey V. Elsukov" <ae@freebsd.org> Subject: ipfw stateful rules and quick port re-use Message-ID: <5ccab312-085c-b764-97c9-4c2bc846cd22@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
Recently we encountered an interesting issue at work. By accident our software started to quickly re-use a source TCP port when connecting to a remote system. That is, after a graceful shutdown of a connection (two FINs, etc), the software would quickly establish an identical connection by re-using the same local port and connecting to the same remote end-point. That did not work well for the application :) We saw problems where packets from the second connection would be dropped by ipfw. That happened because there would be no dynamic rule to let the packets through even though the first connection worked without any issues. >From a quick glance at the code it seems that the TCP protocol state kept by ipfw for dynamic rules is "append-only". That is, bits can be set in it but never cleared. So, when the first connection is closed the dynamic has "both syn" and "both fin" bits. When the second connection is established before the rule is expired, the rule is re-used for it, but its state remains the same. And its expiry time remains dyn_fin_lifetime. I think that that opens a race between the expiry timer (running every second) and the connection's packets given the short lifetime. Maybe I misanalyzed the situation and it's probably very rare. But still it's a valid use of TCP, so maybe ipfw could support it better (e.g., by detecting "syn" after "both fin"). -- Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5ccab312-085c-b764-97c9-4c2bc846cd22>