Date: Tue, 26 Nov 2013 18:41:40 -0500 From: Antoine =?utf-8?Q?Beaupr=C3=A9?= <anarcat@koumbit.org> To: freebsd-net@freebsd.org Cc: bzeeb-lists@lists.zabbadoz.net, anarcat@koumbit.org Subject: OpenBGPd + TCP-MD5 sig fails after a few weeks Message-ID: <87zjoqu3wr.fsf@marcos.anarc.at>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
[please CC me I am not on the list. also cc'ing bzeeb since he was
working on a patch for this two years ago]
Hi,
I have configured an OpenBGPd daemon to connect to another provider with a
TCP-MD5 password.
It used to work when I set it up 30 days ago, but somehow the session is
down now.
# bgpctl show
Neighbor AS MsgRcvd MsgSent OutQ Up/Down State/PrfRcvd
Cogent 174 0 0 0 Never Active
I suspect this was working only because the remote server was
initializing the connexion and not us.
What is ackward is that OpenBGPd doesn't seem to properly initialise the
socket with an MD5 signature:
17:54:59.479900 IP (tos 0xc0, ttl 1, id 14983, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c0d9)!)
38.104.152.102.16295 > 38.104.152.101.179: Flags [S], cksum 0x096d (correct), seq 1556933933, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1324688 ecr 0], length 0
17:54:59.480593 IP (tos 0x0, ttl 255, id 30414, offset 0, flags [none],proto TCP (6), length 40)
38.104.152.101.179 > 38.104.152.102.16295: Flags [R.], cksum 0xa7df (correct), seq 0, ack 1556933934, win 0, length 0
Usually, this should mention "md5valid" in the options field if it is
enabled.
This actually works with netcat:
# nc -v -S 38.104.152.101 179
nc: connect to 38.104.152.101 port 179 (tcp) failed: Connection refused
# tcpdump -M [...] -i bge0 -n -vvv port 179
tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size
65535 bytes
18:15:43.534592 IP (tos 0x0, ttl 64, id 27666, offset 0, flags [DF], proto TCP (6), length 80, bad cksum 0 (->50fa)!)
38.104.152.102.26043 > 38.104.152.101.179: Flags [S], cksum 0xe73a (correct), seq 3803575904, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2568742 ecr 0,nop,nop,md5valid], length 0
18:15:43.536592 IP (tos 0x0, ttl 255, id 30526, offset 0, flags [none], proto TCP (6), length 60)
38.104.152.101.179 > 38.104.152.102.26043: Flags [R.], cksum 0x1566 (correct), seq 0, ack 3803575905, win 0, options [md5valid,eol], length 0
Notice, however, how the other side is still reseting our connexion, so
there's something wrong there - but it could be that they simply blocked
us because of too many failed attempts.
The SAD association is set properly:
# setkey -D
38.104.152.102 38.104.152.101
tcp mode=any spi=4096(0x00001000) reqid=0(0x00000000)
A: tcp-md5 [...]
seq=0x00000000 replay=0 flags=0x00000040 state=mature
created: Nov 26 17:37:59 2013 current: Nov 26 17:57:40 2013
diff: 1181(s) hard: 0(s) soft: 0(s)
last: hard: 0(s) soft: 0(s)
current: 0(bytes) hard: 0(bytes) soft: 0(bytes)
allocated: 0 hard: 0 soft: 0
sadb_seq=1 pid=31485 refcnt=1
38.104.152.101 38.104.152.102
tcp mode=any spi=4096(0x00001000) reqid=0(0x00000000)
A: tcp-md5 [...]
seq=0x00000000 replay=0 flags=0x00000040 state=mature
created: Nov 26 17:37:59 2013 current: Nov 26 17:57:40 2013
diff: 1181(s) hard: 0(s) soft: 0(s)
last: hard: 0(s) soft: 0(s)
current: 0(bytes) hard: 0(bytes) soft: 0(bytes)
allocated: 0 hard: 0 soft: 0
sadb_seq=0 pid=31485 refcnt=1
here's our ipsec.conf:
add -n 38.104.152.101 38.104.152.102 tcp 0x1000 -A tcp-md5 "[...]";
add -n 38.104.152.102 38.104.152.101 tcp 0x1000 -A tcp-md5 "[...]";
I have tried both openbgpd-5.2.20121209 and openbgpd-5.2.20121014. I
have also tried to disable TCP verification through sysctl, no luck:
net.inet.tcp.signature_verify_input: 0
We have the following kernel configuration:
include GENERIC
ident KOUMBIT1
device pf
device pflog
device pfsync
options ALTQ
options ALTQ_CBQ
options ALTQ_RED
options ALTQ_RIO
options ALTQ_HFSC
options ALTQ_CDNR
options ALTQ_PRIQ
options IPSEC #IP security
options TCP_SIGNATURE
device crypto
options DEVICE_POLLING
device carp
Our bgpd.conf doesn't specify a tcp password, otherwise we end up with
the following error:
root@rtr0:/usr/local/etc # bgpd -d
startup
rereading config
no kernel support for PF_KEY
route decision engine ready
session engine ready
RDE reconfigured
listening on 0.0.0.0
listening on ::
SE reconfigured
neighbor 38.104.152.101 (Cogent): state change None -> Idle, reason: None
neighbor 38.104.152.101 (Cogent): pfkey setup failed
Why is it that outgoing TCP connexions do not respect setkey settings?
I read a little bit of the source code, and it seems that openbgpd is
stuck in a catch-22: it can't setup the SAD associations (because they
are handled by setkey) so it doesn't set the MD5SIG option on the
socket... One horrible solution I found was to change session_connect()
as such:
- if (peer->conf.auth.method != AUTH_NONE && sysdep.no_pfkey) {
+ if (peer->conf.auth.method != AUTH_NONE && sysdep.no_pfkey || 1) {
log_peer_warnx(&peer->conf,
- "ipsec or md5sig configured but not available");
- bgp_fsm(peer, EVNT_CON_OPENFAIL);
- return (-1);
+ "ipsec or md5sig configured but not available, assuming set externally");
+ /* bgp_fsm(peer, EVNT_CON_OPENFAIL); */
+ /* return (-1); */
+ if (setsockopt(peer->fd, IPPROTO_TCP, TCP_MD5SIG,
+ &opt, sizeof(opt)) == -1) {
+ log_peer_warn(&peer->conf, "setsockopt md5sig");
+ bgp_fsm(peer, EVNT_CON_OPENFAIL);
+ return (-1);
+ }
+
}
It's pretty nasty, but with this at least the connexion gets initialized
going out with the right socket options.
A.
PS: this is a problem similar to what was reported here:
http://lists.freebsd.org/pipermail/freebsd-net/2012-January/030921.html
--
Il faut tout un village pour élever un enfant.
- Proverbe africain
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iQIcBAEBCAAGBQJSlTG1AAoJEHkhUlJ7dZIeElcP/30dP2sbM5wDZk+dMsXAyMyj
NrYlSc5VfYcgYL65LOK2AnS3fj21W7ov3V/LgOs4lKed7B3+hZaFTAL4jg6eq4l3
98y3W3Hv4cF7TVhUiqyJQ1G81fN2iD+GYQECDhmZE76hU0n+acoAn4pALlCq56Cn
aWwhdOqJBklCjmWRgYxXy/vY0sdkF0Lgq7Qj3wlqh5OOfwNFwYX0hgcSFdOn35Bq
EtiJFUlroUyMorNb7xiwCvuxS0wsQvF4RzTssxNB/921HMUMXWs0HiOuNEzdbbN7
g6BKn5fcbupWxPHC5KpWjqOQwz0b5lE6WoF3V/G58vpM9Bx4PAVVESHruZi75HCk
lCDfCGE9tLLEkqiobA4/h1cN1039FG6e7dqmfq8lYixQ7sGB7eX/TyJH4x/MR7/N
698IKtjWDeh9SesiTCkysRIyriM7MHPz52B/giz8NiGsgLTcgJiW9/Iv2Icb6V7P
acO8VuNbxs4VD2xqOJhrDZGNF2+R7MEIIW8WXskKwKmzdfmQ97XJPHyxR4LG7Xtr
XKVWMwed8jIucMLycqK4oOuAwY4ibu8sXfA43Jbx+dkqv8trYhTUX7Y5Tte95fH8
YySV0mXndPsBhCAXlNMcQYIMhNABDP3reHosMQxZ82/b8vcVOF+9Mx/xxWKaUpnB
SOWITR3dGp+ukcvBCUq+
=ZyGV
-----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?87zjoqu3wr.fsf>