From owner-freebsd-hackers Wed Jan 17 0:13: 1 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.simplexi.com (unknown [203.231.63.2]) by hub.freebsd.org (Postfix) with ESMTP id 6AAD937B698 for ; Wed, 17 Jan 2001 00:12:41 -0800 (PST) Received: from chulwon (cwmin [203.231.63.178]) by mail.simplexi.com (8.9.3/8.9.3) with SMTP id RAA13942 for ; Wed, 17 Jan 2001 17:13:13 +0900 Message-ID: <002501c0805c$ea559e00$b23fe7cb@chulwon> From: "¹Îö¿ø" To: Subject: LVS with FreeBSD Date: Wed, 17 Jan 2001 17:10:12 +0900 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2615.200 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hello, I am setting up a LVS/DR cluster with 2 nodes(FreeBSD), but It doesn't work. Here is my network configuration; Internet (203.231.63.70 is Virtual IP) | | Router (203.231.63.0/24 network) | | <----- eth0 : 203.231.63.74 LVS(Linux) | <----- eth1 : 203.231.63.70 (VIP) | ------------------ | | SVR1 SVR2 <-- Real Servers are FreeBSD 4.2-RELEASE fxp0 : 203.231.63.70 203.231.63.70 (VIP) fxp1 : 203.231.63.71 203.231.63.72 (Real IP) ----] in Load Valancing Server(203.231.63.74); [root@ha1 log]# ifconfig -a eth0 Link encap:Ethernet HWaddr 00:10:5A:80:D7:FF inet addr:203.231.63.74 Bcast:203.231.63.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth1 Link encap:Ethernet HWaddr 00:10:5A:76:02:49 inet addr:203.231.63.70 Bcast:203.231.63.70 Mask:255.255.255.255 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:3924 Metric:1 [root@LVS /]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 203.231.63.70 0.0.0.0 255.255.255.255 UH 0 0 0 eth1 203.231.63.74 0.0.0.0 255.255.255.255 UH 0 0 0 eth0 203.231.63.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo 0.0.0.0 203.231.63.254 0.0.0.0 UG 0 0 0 eth0 [root@LVS /]# sysctl -p net.ipv4.ip_forward = 1 net.ipv4.conf.all.rp_filter = 1 net.ipv4.ip_always_defrag = 0 kernel.sysrq = 0 [root@LVS /]# vi /etc/ha.d/conf/ldirectord.cf timeout=3 checkinterval=5 autoreload=no fallback=127.0.0.1:80 virtual=203.231.63.70:80 real=203.231.63.71:80 gate 1 real=203.231.63.72:80 gate 1 service=http request="index.html" receive="Test Page" scheduler=rr protocol=tcp ----] in Real Server(231.63.71,72); SVR1# ifconfig -a fxp0: flags=8843 mtu 1500 inet 203.231.63.70 netmask 0xffffffff broadcast 203.231.63.70 fxp1: flags=8843 mtu 1500 inet 203.231.63.72 netmask 0xffffff00 broadcast 203.231.63.255 SVR2 in the same way.. ******* Problem: 1. LVS dosen't forward http request packet to the real server.. 2. I don't know how FreeBSD(real server) can avoid arp request.. About first problem: Client try to connect 203.231.63.70:80, but LVS doesn't reply to that SYN packet.. There are 2 things that seems odd.. First, When I start up ldirectord, output is like this.. [root@LVS /]# /etc/rc.d/init.d/ldirectord start Starting ldirectord [ OK ] [root@LVS /]# vi /var/log/ldirectord.log ...... [Tue Jan 16 13:47:48 2001..] Starting Linux Director Daemon [Tue Jan 16 13:47:48 2001..] Adding virtual server: 203.231.63.70:80 [Tue Jan 16 13:47:48 2001..] Starting fallback server for: 203.231.63.70:80 [Tue Jan 16 13:47:49 2001..] Adding real server: 203.231.63.71:80 (1*203.231.63.70:80) [Tue Jan 16 13:47:49 2001..] Turning off fallback server for: 203.231.63.70:80 [Tue Jan 16 13:47:49 2001..] system(/sbin/ipvsadm -a -t 203.231.63.70:80 -R 203.231.63.72:80 -g -w 1) failed [Tue Jan 16 13:47:49 2001..] Adding real server: 203.231.63.72:80 (2*203.231.63.70:80) ...... system(/sbin/ipvsadm -a -t 203.231.63.70:80 -R 203.231.63.72:80 -g -w 1) failed **Why this error occured?? What should I do to eliminate this error message?? Second, Here's my ipvsadm output: [root@LVS /]# ipvsadm -L -n IP Virtual Server version 0.9.7 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 203.231.63.70:www rr -> 255.255.255.255:52199 Masq 4194304 0 0 Last output line seems wrong,, I think It should look like this.. right? TCP 203.231.63.70:www rr -> 203.231.63.71:80 route 1 0 0 -> 203.231.63.72:80 route 1 0 0 **How can I fix this thing?? Second problem: As you know.. in LVS cluster, real servers should not reply to arp request that asks VIP's MAC address.. Only LVS should reply to that arp request.. I have an idea about it.. Let the real server reply to client's arp request(for VIP) with LVS's hardware address.. then all client's packet that towards VIP go to the LVS.. That's a good idea.. so I commanded like this.. arp -s 203.231.63.70 00:10:5A:76:02:49 pub 203.231.63.70 is VIP and 00:10:5A:76:02:49 is LVS's MAC address.. I got a meessage : set: proxy entry exists for non 802 device without 'pub' flag : set: can only proxy for 203.231.63.70 ***What is this message means?? ***ARP reply with another MAC address is impossible?? ***How can I solve this arp problem?? Thanks in advance.. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message