From owner-freebsd-hackers Mon Apr 3 18:27:37 2000 Delivered-To: freebsd-hackers@freebsd.org Received: from lanturn.express.ru (lanturn.kmost.express.ru [212.24.37.109]) by hub.freebsd.org (Postfix) with ESMTP id 0D6A237B512 for ; Mon, 3 Apr 2000 18:27:33 -0700 (PDT) (envelope-from vova@express.ru) Received: from vova (helo=localhost) by lanturn.express.ru with local-esmtp (Exim 3.11 #1) id 12cI7Y-0004ch-00; Tue, 04 Apr 2000 05:27:08 +0400 Date: Tue, 4 Apr 2000 05:27:07 +0400 (MSD) From: vova@express.ru X-Sender: vova@lanturn.kmost.express.ru To: Jim Mercer Cc: hackers@freebsd.org Subject: Re: FBSD-3.4, full bgp routing, maxusers, NMBCLUSTERS In-Reply-To: <20000403090358.K10147@reptiles.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 3 Apr 2000, Jim Mercer wrote: > i've got a server that has the sole purpose of routing packters between > 4 100mbit interfaces. > > the server is running 3.4-stable. > > it has 128M RAM, and according to top, it isn't using much more than 80M. > > i'm using zebra to do full BGP routing with 2 peers. > > netstat -rn shows some 75,000 routes. > > i've got: > maxusers 32 > options NMBCLUSTERS=10000 > > vmstat -m shows: > routetbl 154337 21118K 21118K 21118K 237725 0 0 16,32,64,128,256 > Memory Totals: In Use Free Requests > 21842K 47K 249883 > > how do i increase the amount of RAM for the kernel? > i thought NMBCLUSTERS was the one, but i guess not. > > any recommendations? your in-kernel tables limited by 21118K each, you need increase this limit how ? I know two ways: first method - increase real memory of PC it will work with multiplier about 6 (with 192 real I have limit ~30M, with 64M real I have about 10M limit) second way: tune kernel paramets # cat /sys/i386/include/vmparam.h: ... /* virtual sizes (bytes) for various kernel submaps */ #ifndef VM_KMEM_SIZE #define VM_KMEM_SIZE (12 * 1024 * 1024) #endif /* * How many physical pages per KVA page allocated. * min(max(VM_KMEM_SIZE, Physical memory/VM_KMEM_SIZE_SCALE), VM_KMEM_SIZE_MAX) * is the total KVA space allocated for kmem_map. */ #ifndef VM_KMEM_SIZE_SCALE #define VM_KMEM_SIZE_SCALE (3) #endif ... so you can higher VM_KMEM_SIZE or lower VM_KMEM_SIZE_SCALE in your kernel config file I decrease VM_KMEM_SIZE_SCALE to (1) and have got ~ 96M kernel limits (about half of PC's RAM) # grep VM_ /sys/i386/conf/LANTURN options VM_KMEM_SIZE_SCALE="(1)" # vmstat -m | grep routetbl routetbl 212269 39797K 39797K 95256K 351036 0 0 16,32,64,128,256 and then I have installed about 100K routes with script (much more than in BGP full-view) for testing: # netstat -rn | wc -l 106111 # it takes 39797K > -- > [ Jim Mercer jim@reptiles.org +1 416 506-0654 ] > [ Reptilian Research -- Longer Life through Colder Blood ] > [ Don't be fooled by cheap Finnish imitations; BSD is the One True Code. ] -- TSB Russian Express, Moscow Vladimir B. Grebenschikov, vova@express.ru To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message