Date: Thu, 5 Jan 2012 20:18:39 -0500 From: J David <j.david.lists@gmail.com> To: freebsd-net@freebsd.org Subject: Re: openbgpds not talking each other since 8.2-STABLE upgrade Message-ID: <CABXB=RR7hwaQtQcOJks3ipt3iKM=_ViErxG8THSr2rsCoLgOTA@mail.gmail.com> In-Reply-To: <CABXB=RQFuAdkFiRgNH%2B9QWHMn8zMR31wmcSWumwWv54UwVyvvw@mail.gmail.com> References: <20120104.040611.1847309275485655567.hrs@allbsd.org> <4F036A7F.9030906@FreeBSD.org> <52D4B9DF-4BC3-4AF7-BCE0-A88E18F25650@gmail.com> <20120104.144214.74742226.sthaug@nethelp.no> <CABXB=RQFuAdkFiRgNH%2B9QWHMn8zMR31wmcSWumwWv54UwVyvvw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
To help understand what's going on and test some of this stuff, I hacked up a TCP-MD5-aware echo server and tried various things. The first thing I found was that setting net.inet.tcp.signature_verify_input to 0 does not stop the listener socket from setting TCP_MD5SIG. So, setting this is not a way to "mask" the availability of TCP signatures if the kernel supports them. Ultimately, the behavior of connections depends on four factors: - the server's net.inet.tcp.signature_verify_input setting - the client's net.inet.tcp.signature_verify_input setting - whether the server socket sets TCP_MD5SIG - whether the client socket sets TCP_MD5SIG (In all test cases the correct SA's are present.) Unfortunately, this creates sixteen test cases, so it took some time to try them all. 1. sverify = 0, cverify = 0, smd5 = n, cmd5 = n 5. sverify = 0, cverify = 1, smd5 = n, cmd5 = n 9. sverify = 1, cverify = 0, smd5 = n, cmd5 = n 13. sverify = 1, cverify = 1, smd5 = n, cmd5 = n These all work as expected. No MD5 signatures are requested, used or checked. 2. sverify = 0, cverify = 0, smd5 = n, cmd5 = y 10. sverify = 1, cverify = 0, smd5 = n, cmd5 = y These work. MD5 signatures generated client->server only. 6. sverify = 0, cverify = 1, smd5 = n, cmd5 = y 14. sverify = 1, cverify = 1, smd5 = n, cmd5 = y These work, but not properly. Connects and echoes, but client gets stuck in LAST_ACK, client seems to ignore server's ACK. MD5 signatures client->server only. 3. sverify = 0, cverify = 0, smd5 = y, cmd5 = n Works. MD5 signatures generated server->client only. 4. sverify = 0, cverify = 0, smd5 = y, cmd5 = y 12. sverify = 1, cverify = 0, smd5 = y, cmd5 = y Works. Signatures generated in both directions. 7. sverify = 0, cverify = 1, smd5 = y, cmd5 = n Times out. Client ignores SYN-ACK from server. Signatures generated server->client. 8. sverify = 0, cverify = 1, smd5 = y, cmd5 = y 16. sverify = 1, cverify = 1, smd5 = y, cmd5 = y Works, but not properly. Connects and echoes, but client gets stuck in LAST_ACK again, client seems to ignore server's final ACK. MD5 signatures in both directions. 11. sverify = 1, cverify = 0, smd5 = y, cmd5 = n Does not work. Server goes to SYN_RCVD, client goes to ESTABLISHED. Server appears to ignore client's ACK. MD5 signatures server->client only. 15. sverify = 1, cverify = 1, smd5 = y, cmd5 = n Times out. Server goes to SYN_RCVD, client sticks at SYN_SENT, appears to ignore SYN-ACK. Signatures server->client only. While this was a bit time-consuming, it seems worthwhile because it pointed up a couple of apparent problems that may have ramifications outside openbgpd. For openbgpd to work properly with both clients that do and don't use TCP-MD5, cases 11 and 16 both need to work. Currently neither do. That case 16 doesn't work suggests that openbgpd probably has hard-to-observe problems leaving sockets in LAST_ACK if it is the MD5-using client. Probably this has no effect in practice, because it only appears to happen as the client, which would use a different port to establish a new connection. But it does generate a lot of unnecessary retransmits so it would be nice to track this behavior down. Also, this is the actual as-designed/useful use of TCP-MD5 (generating and checking) so it seems the most important one to ensure is 100% correct. Case 11 certainly explains why OpenBGPd gets so confused; one side thinks the connection is established, the other doesn't. In order for this to work, there must be some method of "turning off" the expectation of receiving MD5-signed packets on a particular connection. In this context, the TCP_MD5SIG seems to be doing "double duty" in that it is telling the connection to both send and expect MD5 signatures, although the tcp(4) man page refers only to outgoing traffic. (Though again that may be a documentation error.) So there may be some bugs here on the FreeBSD side that may make it hard to get openbgpd working properly. There are a couple of other things that may be worth looking at, but if someone knows the way forward from here, any guidance would be appreciated. Thanks!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CABXB=RR7hwaQtQcOJks3ipt3iKM=_ViErxG8THSr2rsCoLgOTA>