Date: Mon, 20 Apr 2009 14:21:10 +0200 From: Sebastiaan van Erk <sebster@sebster.com> To: freebsd-cluster@freebsd.org Subject: pf and carp, BACKUP host dropping connection Message-ID: <49EC68B6.9090303@sebster.com>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Hi, I have 3 hosts set up with 1 virtual IP using carp. I don't yet have pfsync (which I'm planning to do next). However, there is a strange behavior that I cannot understand. The 3 machines are all gateways between two networks and have 2 VIP ips which are used for routing (actually they have 4 networks and 4 VIPs, but only 2 are relevant in this case). When I ssh from one network to the other however, connections are sometimes blocked by pf. However, they're dropped on the machine which is NOT currently master! That is, I have machines: 1) carp1: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500 inet 10.0.80.74 netmask 0xffffff00 carp: MASTER vhid 2 advbase 1 advskew 0 carp3: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500 inet 10.0.82.74 netmask 0xffffff00 carp: MASTER vhid 4 advbase 1 advskew 0 2) carp0: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500 inet 212.61.136.74 netmask 0xfffffff0 carp: BACKUP vhid 1 advbase 1 advskew 50 carp2: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500 inet 10.0.81.74 netmask 0xffffff00 carp: BACKUP vhid 3 advbase 1 advskew 50 3) carp1: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500 inet 10.0.80.74 netmask 0xffffff00 carp: BACKUP vhid 2 advbase 1 advskew 100 carp3: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500 inet 10.0.82.74 netmask 0xffffff00 carp: BACKUP vhid 4 advbase 1 advskew 100 Then from the 10.0.80 network I do a ssh to the 10.0.82 network. The router for the 10.0.82 network is 10.0.82.74 and the router for the 10.0.80 network is 10.0.80.74 (the VIPs): > ssh 10.0.82.5 sebster@10.0.82.5's password: > Read from remote host 10.0.82.5: Connection reset by peer Connection to 10.0.82.5 closed. And then I get on the backup gateways pf log: machine 2: # tcpdump -nttteli pflog0 not src or dst port 6155 and not src or dst host 224.0.0.18 and not src or dst port 68 tcpdump: WARNING: pflog0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 96 bytes 000000 rule 11/0(match): block in on em1: 10.0.80.3.58876 > 10.0.82.5.22: [|tcp] 001161 rule 11/0(match): block in on em1: 10.0.80.3.58876 > 10.0.82.5.22: [|tcp] 000018 rule 11/0(match): block in on em1: 10.0.80.3.58876 > 10.0.82.5.22: tcp 20 [bad hdr length 0 - too short, < 20] machine 3: # tcpdump -nttteli pflog0 not src or dst port 6155 and not src or dst host 224.0.0.18 and not src or dst port 68 tcpdump: WARNING: pflog0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 96 bytes 000000 rule 11/0(match): block in on em1: 10.0.80.3.58876 > 10.0.82.5.22: [|tcp] 001113 rule 11/0(match): block in on em1: 10.0.80.3.58876 > 10.0.82.5.22: [|tcp] 000019 rule 11/0(match): block in on em1: 10.0.80.3.58876 > 10.0.82.5.22: tcp 20 [bad hdr length 0 - too short, < 20] I'm wondering why these backup hosts are blocking these packets, even though the master is still up, and why they are causing the connection to fail. (The pf on all 3 hosts do a "block return log on devif all" where devif is the interface with the real 10.0.80.x ip; however, why is it returning a RST packet when it's backup?). I think once I have pfsync the problem will go away due to the synchronized state (the backups won't block anymore), but it still seems strange to me that all 3 machines will then be actively filtering the packets... Does anybody know what's going on? Regards, Sebastiaan [-- Attachment #2 --] 0 *H 01 0 + 0 *H Q00lS| 6$1-~j0 *H 0b1 0 UZA1%0#U Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA0 080630135157Z 090630135157Z0h10Uvan Erk10U* Sebastiaan10USebastiaan van Erk1"0 *H sebster@sebster.com0"0 *H 0 Va\bEnݚa<M8ʄ^tv>x73bohi2oqS_¶Bm^p* I x"9 pt!j ar#)n)^? 'z<).+Ѐ4ig R'UP*\Ւ ,?.;?f BܯTzM IDվCK*3Yŧ mcaztxʐsq/ 00.0 U 0sebster@sebster.com0 U 0 0 *H KT4W6ӽq] tS` %f1 G:H b zJj$EjE'JV~-VbVnJZ E/`@@04!+T: c پf`$Z=1#|oG[OB RG00lS| 6$1-~j0 *H 0b1 0 UZA1%0#U Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA0 080630135157Z 090630135157Z0h10Uvan Erk10U* Sebastiaan10USebastiaan van Erk1"0 *H sebster@sebster.com0"0 *H 0 Va\bEnݚa<M8ʄ^tv>x73bohi2oqS_¶Bm^p* I x"9 pt!j ar#)n)^? 'z<).+Ѐ4ig R'UP*\Ւ ,?.;?f BܯTzM IDվCK*3Yŧ mcaztxʐsq/ 00.0 U 0sebster@sebster.com0 U 0 0 *H KT4W6ӽq] tS` %f1 G:H b zJj$EjE'JV~-VbVnJZ E/`@@04!+T: c پf`$Z=1#|oG[OB RG0?0 0 *H 01 0 UZA10U Western Cape10U Cape Town10U Thawte Consulting1(0&U Certification Services Division1$0"UThawte Personal Freemail CA1+0) *H personal-freemail@thawte.com0 030717000000Z 130716235959Z0b1 0 UZA1%0#U Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CA00 *H 0 Ħ<UsUNʙZ hup [v :aQP 0cZ,p+Z?qV˯< 6$*+w=+>@dקe*TH<a@ dr` 00U 0 0CU <0:08642http://crl.thawte.com/ThawtePersonalFreemailCA.crl0 U 0)U "0 0 10UPrivateLabel2-1380 *H HP . fgCL!6 -6/ P p<ab:~ t%Pb'qW%ݩ9 Oe_N4[5MwV!x!5$ F]_eO1q0m0v0b1 0 UZA1%0#U Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS| 6$1-~j0 + 0 *H 1 *H 0 *H 1 090420122110Z0# *H 1$KRTC*DA5m0_ *H 1R0P0 `He0 *H 0*H 0 *H @0+0 *H (0 +71x0v0b1 0 UZA1%0#U Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS| 6$1-~j0 *H 1xv0b1 0 UZA1%0#U Thawte Consulting (Pty) Ltd.1,0*U#Thawte Personal Freemail Issuing CAS| 6$1-~j0 *H ' en{C} h9x {x4̇i*NJ; y▝qa=jO㰬<&pia;!4J"!17KޗůzToοKtIYg`K~"@"vwk}e?1=ArkIq'3"- ;t+^{_$lriiQfİ+# ]7t<ul îy.ѡ;TķI`&Q5K~:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?49EC68B6.9090303>