Date: Sat, 18 Dec 2004 04:03:11 -0600 (CST) From: Mike Silbersack <silby@silby.com> To: net@freebsd.org Subject: Alternate port randomization approaches Message-ID: <20041218033226.L28788@odysseus.silby.com>
next in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1286915442-1103364191=:28788 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed There have been a few reports by users of front end web proxies and other systems under FreeBSD that port randomization causes them problems under load. This seems to be due to a combination of port randomization and rapid connections to the same host causing ports to be recycled before the ISN has advanced past the end of the previous connection, thereby causing the TIME_WAIT socket on the receiving end to ignore the new SYN. I haven't proven it, but I imagine that part of the problem is due to the interaction of established and finished connections. Say that ports 1500 -> 1600 have established connections, and that arc4random picks port 1504 as the next one to try. Rather than picking another random port, we simply look sequentially through the rest of the ports, and will pick 1601. I had considered continuing to do random port guesses until an unused one was found, but that has the nasty property of looping an undetermined amount of time, and it would most likely not help anything anyway. The solution I've come up with that should provide a decent tradeoff between the increased security of port randomization and the realities of the problem of quick port recycling is attached. What it does is only randomize the first net.inet.ip.portrange.randomized ports assigned each second. Beyond that, it uses sequentual allocation. The default I picked, 20, is arbitrary. Initial testing by a front-end web proxy admin shows that 20 is too high to avoid problems in his case, but 1 works decently. Although this isn't a perfect fix, I think that it should be acceptable for the vast majority of systems, and I'd like to get it in before 4.11-release ships. To be conservative, I'll probably choose a value like 5, which should be fine for most systems out there. Super specialized users will always be able to lower it to 0. Since this is an important issue, I was wondering if anyone had any suggestions on how to improve this method, or how to solve the problem in another way. Remember that 4.11 ships in the next few weeks, so any solution which gets added to it must be relatively simple, we can't be doing anything like changing the TCP state machine. (Yes, I've considered removing the ISN > window requirement from the TIME_WAIT state, but that's not something to be changing on a whim.) Also note that 5.x and 6.x handle TIME_WAIT recycling in the exact same way 4.x does; the sequence number check is just now in the function tcp_timewait rather than in tcp_input. So, the same problems plagues all versions. (Despite my other modifications to sequence number generation which were attempts to fix it.) Comments appreciated, Mike "Silby" Silbersack --0-1286915442-1103364191=:28788 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="in_pcb.c-lessrandomization.patch-4" Content-Transfer-Encoding: BASE64 Content-ID: <20041218040311.P28788@odysseus.silby.com> Content-Description: Content-Disposition: attachment; filename="in_pcb.c-lessrandomization.patch-4" LS0tIC91c3Ivc3JjL3N5cy5vbGQvbmV0aW5ldC9pbl9wY2IuYwlUaHUgRGVj IDE2IDAzOjI2OjExIDIwMDQNCisrKyBpbl9wY2IuYwlUaHUgRGVjIDE2IDAz OjI3OjI5IDIwMDQNCkBAIC05NSw4ICs5NSwxMCBAQA0KIGludAlpcHBvcnRf aGlmaXJzdGF1dG8gPSBJUFBPUlRfSElGSVJTVEFVVE87CS8qIDQ5MTUyICov DQogaW50CWlwcG9ydF9oaWxhc3RhdXRvICA9IElQUE9SVF9ISUxBU1RBVVRP OwkJLyogNjU1MzUgKi8NCiANCi0vKiBTaGFsbCB3ZSBhbGxvY2F0ZSBlcGhl bWVyYWwgcG9ydHMgaW4gcmFuZG9tIG9yZGVyPyAqLw0KLWludAlpcHBvcnRf cmFuZG9taXplZCA9IDA7DQorLyogSG93IG1hbnkgZXBoZW1lcmFsIHBvcnQg cmFuZG9taXphdGlvbnMgc2hvdWxkIG9jY3VyIGVhY2ggc2Vjb25kPyAqLw0K K2ludAkJaXBwb3J0X21heHJwcyA9IDIwOw0KK3N0cnVjdCB0aW1ldmFsCWlw cG9ydF9sYXN0cmFuZG9tOw0KK2ludAkJaXBwb3J0X2N1cnJwczsNCiANCiAj ZGVmaW5lIFJBTkdFQ0hLKHZhciwgbWluLCBtYXgpIFwNCiAJaWYgKCh2YXIp IDwgKG1pbikpIHsgKHZhcikgPSAobWluKTsgfSBcDQpAQCAtMTM1LDcgKzEz Nyw3IEBADQogU1lTQ1RMX1BST0MoX25ldF9pbmV0X2lwX3BvcnRyYW5nZSwg T0lEX0FVVE8sIGhpbGFzdCwgQ1RMVFlQRV9JTlR8Q1RMRkxBR19SVywNCiAJ ICAgJmlwcG9ydF9oaWxhc3RhdXRvLCAwLCAmc3lzY3RsX25ldF9pcHBvcnRf Y2hlY2ssICJJIiwgIiIpOw0KIFNZU0NUTF9JTlQoX25ldF9pbmV0X2lwX3Bv cnRyYW5nZSwgT0lEX0FVVE8sIHJhbmRvbWl6ZWQsIENUTEZMQUdfUlcsDQot CSAgICZpcHBvcnRfcmFuZG9taXplZCwgMCwgIiIpOw0KKwkgICAmaXBwb3J0 X21heHJwcywgMCwgIiIpOw0KIA0KIC8qDQogICogaW5fcGNiLmM6IG1hbmFn ZSB0aGUgUHJvdG9jb2wgQ29udHJvbCBCbG9ja3MuDQpAQCAtMzIzLDcgKzMy NSw4IEBADQogCQkJLyoNCiAJCQkgKiBjb3VudGluZyBkb3duDQogCQkJICov DQotCQkJaWYgKGlwcG9ydF9yYW5kb21pemVkKQ0KKwkJCWlmIChpcHBvcnRf bWF4cnBzICYmDQorCSBwcHNyYXRlY2hlY2soJmlwcG9ydF9sYXN0cmFuZG9t LCAmaXBwb3J0X2N1cnJwcywgaXBwb3J0X21heHJwcykpDQogCQkJCSpsYXN0 cG9ydCA9IGZpcnN0IC0NCiAJCQkJCSAgICAoYXJjNHJhbmRvbSgpICUgKGZp cnN0IC0gbGFzdCkpOw0KIAkJCWNvdW50ID0gZmlyc3QgLSBsYXN0Ow0KQEAg LTM0Myw3ICszNDYsOCBAQA0KIAkJCS8qDQogCQkJICogY291bnRpbmcgdXAN CiAJCQkgKi8NCi0JCQlpZiAoaXBwb3J0X3JhbmRvbWl6ZWQpDQorCQkJaWYg KGlwcG9ydF9tYXhycHMgJiYNCisJIHBwc3JhdGVjaGVjaygmaXBwb3J0X2xh c3RyYW5kb20sICZpcHBvcnRfY3VycnBzLCBpcHBvcnRfbWF4cnBzKSkNCiAJ CQkJKmxhc3Rwb3J0ID0gZmlyc3QgKw0KIAkJCQkJICAgIChhcmM0cmFuZG9t KCkgJSAobGFzdCAtIGZpcnN0KSk7DQogCQkJY291bnQgPSBsYXN0IC0gZmly c3Q7DQo= --0-1286915442-1103364191=:28788 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="in_pcb.c-lessrandomization.patch" Content-Transfer-Encoding: BASE64 Content-ID: <20041218040311.O28788@odysseus.silby.com> Content-Description: Content-Disposition: attachment; filename="in_pcb.c-lessrandomization.patch" LS0tIC91c3Ivc3JjL3N5cy5vbGQvbmV0aW5ldC9pbl9wY2IuYwlUaHUgTm92 IDI1IDAwOjQzOjIxIDIwMDQNCisrKyAvdXNyL3NyYy9zeXMvbmV0aW5ldC9p bl9wY2IuYwlUaHUgRGVjIDE2IDAyOjU1OjQ2IDIwMDQNCkBAIC05Nyw4ICs5 NywxMCBAQA0KIGludAlpcHBvcnRfcmVzZXJ2ZWRoaWdoID0gSVBQT1JUX1JF U0VSVkVEIC0gMTsJLyogMTAyMyAqLw0KIGludAlpcHBvcnRfcmVzZXJ2ZWRs b3cgPSAwOw0KIA0KLS8qIFNoYWxsIHdlIGFsbG9jYXRlIGVwaGVtZXJhbCBw b3J0cyBpbiByYW5kb20gb3JkZXI/ICovDQotaW50CWlwcG9ydF9yYW5kb21p emVkID0gMTsNCisvKiBIb3cgbWFueSBlcGhlbWVyYWwgcG9ydCByYW5kb21p emF0aW9ucyBzaG91bGQgb2NjdXIgZWFjaCBzZWNvbmQ/ICovDQoraW50CQlp cHBvcnRfbWF4cnBzID0gMjA7DQorc3RydWN0IHRpbWV2YWwJaXBwb3J0X2xh c3RyYW5kb207DQoraW50CQlpcHBvcnRfY3VycnBzOw0KIA0KICNkZWZpbmUg UkFOR0VDSEsodmFyLCBtaW4sIG1heCkgXA0KIAlpZiAoKHZhcikgPCAobWlu KSkgeyAodmFyKSA9IChtaW4pOyB9IFwNCkBAIC0xNDIsNyArMTQ0LDcgQEAN CiBTWVNDVExfSU5UKF9uZXRfaW5ldF9pcF9wb3J0cmFuZ2UsIE9JRF9BVVRP LCByZXNlcnZlZGxvdywNCiAJICAgQ1RMRkxBR19SV3xDVExGTEFHX1NFQ1VS RSwgJmlwcG9ydF9yZXNlcnZlZGxvdywgMCwgIiIpOw0KIFNZU0NUTF9JTlQo X25ldF9pbmV0X2lwX3BvcnRyYW5nZSwgT0lEX0FVVE8sIHJhbmRvbWl6ZWQs DQotCSAgIENUTEZMQUdfUlcsICZpcHBvcnRfcmFuZG9taXplZCwgMCwgIiIp Ow0KKwkgICBDVExGTEFHX1JXLCAmaXBwb3J0X21heHJwcywgMCwgIiIpOw0K IA0KIC8qDQogICogaW5fcGNiLmM6IG1hbmFnZSB0aGUgUHJvdG9jb2wgQ29u dHJvbCBCbG9ja3MuDQpAQCAtNDA0LDcgKzQwNiw4IEBADQogCQkJLyoNCiAJ CQkgKiBjb3VudGluZyBkb3duDQogCQkJICovDQotCQkJaWYgKGlwcG9ydF9y YW5kb21pemVkKQ0KKwkJCWlmIChpcHBvcnRfbWF4cnBzICYmDQorCSBwcHNy YXRlY2hlY2soJmlwcG9ydF9sYXN0cmFuZG9tLCAmaXBwb3J0X2N1cnJwcywg aXBwb3J0X21heHJwcykpDQogCQkJCSpsYXN0cG9ydCA9IGZpcnN0IC0NCiAJ CQkJCSAgICAoYXJjNHJhbmRvbSgpICUgKGZpcnN0IC0gbGFzdCkpOw0KIAkJ CWNvdW50ID0gZmlyc3QgLSBsYXN0Ow0KQEAgLTQyMiw3ICs0MjUsOCBAQA0K IAkJCS8qDQogCQkJICogY291bnRpbmcgdXANCiAJCQkgKi8NCi0JCQlpZiAo aXBwb3J0X3JhbmRvbWl6ZWQpDQorCQkJaWYgKGlwcG9ydF9tYXhycHMgJiYN CisJIHBwc3JhdGVjaGVjaygmaXBwb3J0X2xhc3RyYW5kb20sICZpcHBvcnRf Y3VycnBzLCBpcHBvcnRfbWF4cnBzKSkNCiAJCQkJKmxhc3Rwb3J0ID0gZmly c3QgKw0KIAkJCQkJICAgIChhcmM0cmFuZG9tKCkgJSAobGFzdCAtIGZpcnN0 KSk7DQogCQkJY291bnQgPSBsYXN0IC0gZmlyc3Q7DQo= --0-1286915442-1103364191=:28788--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20041218033226.L28788>