Date: Wed, 9 Mar 2005 19:52:40 +0800 From: "mc" <mc@netx.com.hk> To: "Nickolay Kritsky" <Nickolay.Kritsky@astra-sw.com>, <freebsd-net@freebsd.org> Subject: Re: multiple uplinks from ISP Message-ID: <007401c5249e$7ff80300$df63af0a@mcpm> References: <D86BF562467D944EB435513F725B236A07C1D5@exchange.stardevelopers4msi.com>
next in thread | previous in thread | raw e-mail | index | archive | help
In fact, the biggest problem with me is that I don't have any development machines for building a test network, in other words I cannot do experiments at anytime I want. I usually need to plan the experiment in details and do the experiment with some idle hot backup machine in the network, or I can also use things like VMware to setup a testing network, but getting VMware's network to work as expected is headache.... outgoing/incoming ratio: its the reverse actually :P. out/in is roughly equals to 10:1, usually 12Mbps incoming and 99Mbps outgoing. actually most the traffic is just generated by a single web server. > If it's like 1:10, maybe you won't need multipath routing. You will use only one interface for sending packets, and you will get them back via two interfaces. Think about it. In this case - everything that you need is two equal-cost routes to your network on the ISP side. Remember the KISS idea :-) imho this setup have several drawbacks...at least if the sending link fails, the packets would not automatically go to the other interface. also, firewalling could be made difficult if the packets are distributed like this. ----- Original Message ----- From: "Nickolay Kritsky" <Nickolay.Kritsky@astra-sw.com> To: "mc" <mc@netx.com.hk>; <freebsd-net@freebsd.org> Sent: Wednesday, March 09, 2005 19:24 Subject: RE: multiple uplinks from ISP 1. Yes I think that should be enough. 2. Um, that's a good question. I guess I don't know the answer. you can ask quagga maintainer about the details of quagga multipath routing. Maybe it just changes the gateway, say 10 times in a sec? Maybe it patches kernel binary code, who knows? The best way to know would be to build some test environment. What you need is two machines with 3 interfaces each. One would emulate the ISP side, one will be your side. and test. Plug them in between of some IP link and see what happens with tcpdump and other tools. And, as it suddenly came to my mind, there is another question: what is your outgoing/incoming traffic ratio? If it's like 1:10, maybe you won't need multipath routing. You will use only one interface for sending packets, and you will get them back via two interfaces. Think about it. In this case - everything that you need is two equal-cost routes to your network on the ISP side. Remember the KISS idea :-) Nick -----Original Message----- From: mc [mailto:mc@netx.com.hk] Sent: Wednesday, March 09, 2005 1:58 PM To: Nickolay Kritsky; freebsd-net@freebsd.org Subject: Re: multiple uplinks from ISP dst-ip is not supported on one side of the switch. src-mac does not work too, due to the fact that this would lead to a biased result, causing most of the traffic goes thru the first link. dst-mac would not work as the machine is sending traffic to a single router. > fxp0: 1.2.3.1/30 > fxp1: 1.2.3.5/30 > em0: 10.123.123.102/24 Does this imply I just need to ask my ISP for two /30 and two default gateways and that's it? No other 'special' configuration or registration procedures would be needed? One more question, did you mean if I am to use quagga as the bgp daemon, I don't need to apply some kernel patches for the eq cost multipath to work? 'coz if my memory serves, quagga or other routing daemons just insert/delete/update the route entries in the kernel, they do not take part in any packet routing decisions. ----- Original Message ----- From: "Nickolay Kritsky" <Nickolay.Kritsky@astra-sw.com> To: "mc" <mc@netx.com.hk>; <freebsd-net@freebsd.org> Sent: Wednesday, March 09, 2005 18:35 Subject: RE: multiple uplinks from ISP Why can't you use dst-ip hashing? You are using /24 network for your client machines, no? If FEC uses IP addresses for hashing that you are ok. If it uses MAC addresses for hashing, you need to test something else. Regarding your initial post here is my proposal: fxp0: 1.2.3.1/30 fxp1: 1.2.3.5/30 em0: 10.123.123.102/24 Your ISP gives you 2 more /30 nets for your uplinks You should have two default gateways on fxp0 and fxp1 (1.2.3.2 and 1.2.3.6 respectively) ISP AS should have two routes to your network with the same weight. Problem: FreeBSD natively does not support two different routes to the same destination. AFAIK this is by design. Solution: It can be solved using custom patch (I think I have seen such for 4.x systems) or using external routing daemon like quagga. Nick -----Original Message----- From: mc [mailto:mc@netx.com.hk] Sent: Wednesday, March 09, 2005 1:06 PM To: freebsd-net@freebsd.org Subject: Re: multiple uplinks from ISP Hi, I am using cisco 29xx and 3xxx switches. The problem with FEC is that I have no way to use dst-ip hashing as the load balancing option on these two switches, and that would cause biased utilization on a certain link only, i.e. impossible to utilize 2*100=200Mbps. and...if I were really to use FEC as the solution, I will need to get some much expensive switches from cisco, which is quite unaffordable and imho unnecessary in fact... ----- Original Message ----- From: "Nickolay Kritsky" <Nickolay.Kritsky@astra-sw.com> To: "mc" <mc@netx.com.hk>; <freebsd-net@freebsd.org> Sent: Wednesday, March 09, 2005 17:58 Subject: RE: multiple uplinks from ISP hello I do not think you should mess a lot with interdomain routing here. Such a scenario (multiple uplinks from the same ISP) IMHO is better be solved on the layer 2. What you need is some technology that utilizes two Ethernet ports at once. About a week or two ago on this list was discussed similar setup using Cisco technology. Search for subject "ng_fec and Cisco 2931". I f your ISP is using the switch/router that supports FEC, you could do this trick. Also most 3com intelligent switches support aggregating links via multiple 100Mbit channels. If you have put 3com equipment on both sides of your internet connection you'll can get what you want. Hope that helps. BTW the first and best thing to do is to ask such question to your ISP. Nick -----Original Message----- From: mc [mailto:mc@netx.com.hk] Sent: Wednesday, March 09, 2005 12:32 PM To: freebsd-net@freebsd.org Subject: Re: multiple uplinks from ISP Hi, The main problem is that I have no idea at all how should I setup everything..and what do I need from my ISP......I just know it was possible, but I can't recall the details inside, and a simple google did not return anything helpful to me. I agree with you that fbsd (or any other linux) is much better than cisco in terms of stability. The cisco routers at my site are crashing like cron jobs while the fbsd boxes usually have long uptimes. :) off topic: I used to be a network admin some time ago, but no longer true now....and unfortunately, in the past I had only very few chances to deal with interdomain routing, mainly in lab. I'm afraid I have forgotten everything by now :( ----- Original Message ----- From: "Goran Gajic" <ggajic@mail.sbb.co.yu> To: <freebsd-net@www.freebsd.org> Sent: Wednesday, March 09, 2005 6:01 Subject: Re: multiple uplinks from ISP > > Hi, > > I have used succesfuly FBSD 5.2.1 as BGP router and it is rock stable with > quagga (check out www.quagga.net) - more stable then 30k $ Cisco 7206 :)) > Problem is if you have AS and LIR and if you don't there are other > solutions. Of course much depends is your uplink ISP willing to cooperate. > > Regards, > gg. > > > >> Hi all, >> >> If I have the following on hand... >> - 2 FastEthernet uplinks from ISP >> - 1 GigabitEthernet port on my switch >> - a subset of a /24 allocated by ISP >> The gigabit ethernet link should be connecting to my internal network. > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?007401c5249e$7ff80300$df63af0a>