Date: Sat, 11 Feb 2006 00:39:42 +0300 From: dima <_pppp@mail.ru> To: Marcos Bedinelli <bedinelli@madhaus.cns.utoronto.ca> Cc: freebsd-net@freebsd.org Subject: Re: Network performance in a dual CPU system Message-ID: <E1F7fzS-000GUW-00._pppp-mail-ru@f16.mail.ru> In-Reply-To: <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
> Hello all, > > thanks for the replies. Most of you have suggested that I turn on > polling and give it a try. The machine is in production, hence I need > to schedule downtime for that. > > The system is mainly being used as a dedicated router. It runs OSPF, > BGP and IPFW (around 150 rules). OSPF and BGP are managed by Quagga. > The box has 2 gigabit interfaces that handle on average 200Mbp/s - 50K > packets/s (inbound and outbound combined), each one of them. The second CPU wouldn't help you for sure. There's only one [swi1: net] kernel thread which deals with all the kernel traffic. The option of per-CPU [swi: net] threads was discussed on freebsd-arch@ several months ago, but it wouldn't be implemented soon. So, the only hardware option is installing the fastest CPU possible. There are several software (FreeBSD specific) options though: 1. You should surely try polling(4). 50kpps mean 50000 interrupts and the same amount of context switches, which are quite expensive. 2. FastForwarding. It's the most suitable for you. As I know, Quagga inserts its dynamic routes to the system routing table. And FastForwarding is aware of routing table and firewall rules. And the most exciting: you can switch it on/off without reboot: # sysctl net.inet.ip.fastforwarding=1 The only limitation is it applies to IPv4 unicast traffic only. There's no documentation on this feature as i know (am I wrong, or should I report this as a documentation bug?) but you can look at the comments in the beginning of /sys/netinet/ip_fastfwd.c The authors reported up to 1Mpps (see page 10 at http://people.freebsd.org/~andre/FreeBSD-5.3-Networking.pdf) > > > Some of you have asked for the following information: > > > - As I indicated before, polling is currently disabled. > > > - Hyperthreading (HTT) is disabled. > > > mull [~]$vmstat -i > interrupt total rate > irq1: atkbd0 3466 0 > irq6: fdc0 10 0 > irq13: npx0 1 0 > irq14: ata0 47 0 > irq21: fxp1 20462527 8 > irq28: bge0 3511765157 1444 > irq29: bge1 3633124373 1494 > irq30: aac0 1842472 0 > cpu0: timer 566751007 233 > Total 7733949060 3181 > > > mull [~]$netstat -m > 644/646/1290 mbufs in use (current/cache/total) > 643/407/1050/17088 mbuf clusters in use (current/cache/total/max) > 0/5/4528 sfbufs in use (current/peak/max) > 1447K/975K/2422K bytes allocated to network (current/cache/total) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > > > Thank you, > > -- > Marcos
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1F7fzS-000GUW-00._pppp-mail-ru>