Date: Thu, 11 May 2000 02:24:36 -0500 (CDT) From: Joe Greco <jgreco@ns.sol.net> To: freebsd@gndrsh.dnsmgr.net (Rodney W. Grimes) Cc: stable@freebsd.org, freebsd-net@freebsd.org Subject: Re: Any known problems with routing in 3.4R? Message-ID: <200005110724.CAA60699@aurora.sol.net> In-Reply-To: <200005060040.RAA15679@gndrsh.dnsmgr.net> from "Rodney W. Grimes" at "May 5, 2000 5:40:15 pm"
next in thread | previous in thread | raw e-mail | index | archive | help
> > I've set up a FreeBSD 3.4R box to do BGP. It takes full routes off an ATM
> ^^^^^^^^^^^^^^
>
> Upgrade to 3.4-Stable, the reference count of your network interface
> is wrapping past the 16 bit limit of a short and probably the cause
> of your panic.
Bleah. Fixed in 4.0R? I don't like running non-releases on production
equipment, too hard to rebuild in a crisis situation.
> > OC3 (hea0 set up as atm1) and routes packets between that and the 100mbps
> > Ethernet port.
>
> Also I hope you have tweaked up your KVM space, full routes take a fair
> bit of kernel virtual memory space to hold. I usually run with 48M of kvm,
> some times 64M on full feed BGP boxes, I can say it does work fine on
> 3.4-stable, and that I had nothing but regular panics on 3.4-Release and
> for a long time after that:
# netstat -rn|wc -l
79515
# vmstat -m|grep route
routetbl164144 22452K 22458K 40960K 257974 0 0 16,32,64,128,256
# netstat 1
input (Total) output
packets errs bytes packets errs bytes colls
3193 0 2969682 3226 0 2972594 0
1757 0 1565083 1742 0 1574289 0
2813 0 2545343 2851 0 2552411 0
2773 0 2559062 2740 0 2566358 0
3145 0 2842005 3215 0 2849997 0
3760 0 3326589 3742 0 3335934 0
2403 0 2242247 2382 0 2249117 0
4860 0 4499148 4897 0 4512972 0
I'm sorry but I wasn't going to run with something like 48M of kvm when
the default it was giving me was 40 and I was running out (since the limit
is half of kvm). I don't really need the case where some misconfigured
peer decides to hand me ten thousand internal routes or something stupid
like that, and the GRF handles up to 150,000 (in theory) - so a FreeBSD
router should probably be able to cope with that too.
So as you can see, I set 80M for kvm. This also leaves sufficient space
for other stuff.
> br1.CN85pm.abtltd.com# uname -a
> FreeBSD br1.CN85pm.abtltd.com 3.4-STABLE FreeBSD 3.4-STABLE #0: Mon Jan 3 02:40:43 PST 2000 root@br1.CN85pm.abtltd.com:/usr/src/sys/compile/BR1 i386
> br1.CN85pm.abtltd.com# netstat -rn | wc
> 77695 468055 5444199
>
> > I'm getting periodic (every few days) crashes. I recently started recording
> > the console output, and the last crash was due to a panic
>
> Yepp.. that sure sounds like the 3.4R if reference counter problem, usually
> right after a big flash update from one of your BGP peers.
Well, bearing in mind that on OC3c you get a full table in a matter of
seconds (it's fun because I can watch gated eat the CPU), what I was seeing
was more like it'd freak five to ten minutes after a route table reload. I
would kill and restart gated, with the chances apparently being 50/50ish as
to whether or not I'd lose it as described.
Now, the thing is, the system would generally be quiescent from a routing
update POV. While I do maintain a large number of internal OSPF routes,
they do not tend to flap, and inspection of the other side did not indicate
any change coming from BGP. I do not propagate external routes into OSPF
or anything fancy (I'd kill most of my boxes if I did) and just run a
default-with-some-preferred-routes type thing.
I also got one other crash that simply said "panic: rtfree" and it went,
so I'm really not sure exactly what is going on.
Overall, I'm pleased with the performance of FreeBSD in this application.
The machine it is now on is a K6/233, and seems to perform respectably. I
do not know if I'd want to push full OC3 data rates over it, but there's
always the K6-III/400 :-)
Which reminds me, Rod, I'm finally phasing out those PCI/I-SP3G boards you
sold me. Great investment, but what with me being able to pick up T2P4's
and AMD K6/233's for a song, it's time to move on. But I can't complain
about boards which ran very well for five years. Thanks.
--
... Joe
-------------------------------------------------------------------------------
Joe Greco - Systems Administrator jgreco@ns.sol.net
Solaria Public Access UNIX - Milwaukee, WI 414/342-4847
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200005110724.CAA60699>