Date: Sat, 8 Apr 2000 18:52:07 -0500 (CDT) From: Joe Greco <jgreco@ns.sol.net> To: javier@nyi.net (Javier Frias) Cc: dev@inetu.net, isp@freebsd.org Subject: Re: flat network Message-ID: <200004082352.SAA86584@aurora.sol.net> In-Reply-To: <38ECCE61.511B5A98@nyi.net> from Javier Frias at "Apr 6, 2000 1:50:25 pm"
next in thread | previous in thread | raw e-mail | index | archive | help
From experience, vlan support in equipment has been known to be feeble, lame, and/or simply nonfunctional in a heterogeneous environment. vlans also do nothing to solve the problem of large numbers of ARP entries polluting your router and switch ARP tables, not to mention necessitating additional network traffic (and delays) to actually perform additional ARP requests. Additionally, the deployment of routing protocols allows you to do things such as building redundant networks at the IP level, which allows you to do all sorts of neat stuff. I routinely move servers between distant facilities _without_renumbering_ - since only the local ethernet address changes, not the actual advertised service address. > >From experience, the best solution is to implement vlans in your > network. > > Joe Greco wrote: > > > > > I know this may be a bit more of a network > > > problem, but in my experience, freebsd people have > > > the best skills here to :) > > > > > > We have a server farm of about 200 servers. > > > > > > We have a single router which connects to our bay > > > switches (about 10 switches, all uplink into 1 100 > > > mbps switch). > > > > > > The first 140+ servers were added with random ip > > > addresses assigned to random servers (a block of > > > 20 here, a block of 40 ip's there). > > > > > > Since then, we have started assigned logical > > > blocks (/28, /29, etc.) to servers and routing the > > > block directly to the server's main ip address (to > > > cut down on required arp entries in router). > > > > > > We have a problem where new servers, that don't > > > receive much traffic, tend to drop off the > > > network. After you ping them for about 30 seconds > > > plus they will return. > > > > > > If you constantly ping them, they will not fall > > > off the network (0% packet loss with over 64,000 > > > packets sent during the night). > > > > > > I was wondering if anyone had experienced similiar > > > problems. > > > > > > I think either our router or switch is expiring > > > the arp entry and taking time to re-learn it (due > > > to the large size of our flat network). But how > > > does one actually tell if this is the problem. > > > > > > Any assistance would be greatly apprecaited. > > > > You have 200 servers, or 200 virtual hosts on N (N << 200) servers? > > > > Adding additional alias interfaces is generally not the real cool > > way to do web service, in any event. It is the first obvious mistake > > that many ISP's make... the advertising of crap on large flat networks > > via ARP. I've seen an ISP that did its dial-in pool as a /18 and used > > ARP so that folks with static IP addresses worked. I've seen places > > with /16's with a 0xffff0000 netmask - which caused the obvious problems > > with all sorts of networking devices, since the network had ~8,000 nodes > > or so on it. > > > > Use routing protocols. Break down and learn OSPF. If you have ten > > switches being aggregated into a 100mbps switch, dump the 100mbps > > switch and replace it with a router with a bunch of 100mbps ports. > > Take each junior switch, put it on a 0xffffffe0 network off of the > > router, and populate that with ten or twenty machines that are > > running your servers. Then you allocate a bunch of address space > > for virtual services, and you use OSPF to advertise each. You bind > > additional aliases to lo0 and advertise them as stubs or something > > like that, I've explained methods here before. Then you can even do > > clever things like redundant ethernets for instant, automatic > > failover. This sort of design should allow you to go up to a few > > hundred physical servers supporting thousands of virtual web sites, > > with no problem. > > -- > > ... Joe > > > > ------------------------------------------------------------------------------- > > Joe Greco - Systems Administrator jgreco@ns.sol.net > > Solaria Public Access UNIX - Milwaukee, WI 414/342-4847 > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-isp" in the body of the message > > -- > MMM \|/ www __^__ > (o o) @ @ (O-O) /(o o)\ > -ooO-(_)-Ooo---oOO-(_)-OOo---oOO--(_)--OOo---oOO==(_)==OOo > > Javier A. Frias <javier@nyi.net> > Sr. System Administrator > > The New York Internet Company <http://www.nyi.net>; > 20 Exchange Place 21st Floor > New York, N.Y. 10005 > > > "Error #152 - Windows not found: (C)heer (P)arty (D)ance" > --------------------------------------------------------- > -- ... Joe ------------------------------------------------------------------------------- Joe Greco - Systems Administrator jgreco@ns.sol.net Solaria Public Access UNIX - Milwaukee, WI 414/342-4847 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200004082352.SAA86584>