Date: Tue, 14 Feb 2006 10:46:06 +0300 From: Gleb Smirnoff <glebius@FreeBSD.org> To: Marcos Bedinelli <bedinelli@madhaus.cns.utoronto.ca> Cc: freebsd-net@FreeBSD.org Subject: Re: Network performance in a dual CPU system Message-ID: <20060214074606.GH86448@FreeBSD.org> In-Reply-To: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Feb 10, 2006 at 08:46:00AM -0500, Marcos Bedinelli wrote: M> We have a 2.4GHz Intel Xeon machine running FreeBSD 6.0-RELEASE-p2. Due M> to heavy network traffic, CPU utilization on that machine is 100%: M> M> === M> M> mull [~]$top -S M> last pid: 94989; load averages: 3.69, 4.02, 4.36 up M> 25+07:21:34 14:51:43 M> 105 processes: 2 running, 46 sleeping, 57 waiting M> CPU states: 0.0% user, 0.0% nice, 0.3% system, 99.4% interrupt, M> 0.3% idle M> Mem: 20M Active, 153M Inact, 84M Wired, 4K Cache, 60M Buf, 237M Free M> Swap: 999M Total, 999M Free M> M> PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND M> 60 root 1 -44 -163 0K 8K WAIT 355.6H 72.17% swi1: M> net M> 39 root 1 -68 -187 0K 8K WAIT 52.3H 5.22% irq28: M> bge0 M> 40 root 1 -68 -187 0K 8K WAIT 28.3H 2.25% irq29: M> bge1 M> 11 root 1 171 52 0K 8K RUN 166.6H 0.00% idle M> 63 root 1 -16 0 0K 8K - 121:55 0.00% yarrow M> 61 root 1 -32 -151 0K 8K WAIT 46:21 0.00% swi4: M> clock sio M> [...] You didn't described what the box is actually. According to top(1) output, looks like your system is routing between bge0 and bge1. If this is true, then you should try: sysctl net.inet.ip.fastforwarding=1 This will split the work into the ithreads and also this will use a faster code path. The performance will improve, and obtaining a second CPU will give you a benefit, since you will have two heavy loaded threads. You should also try upgrade to 6.1, since bge(4) driver has undergone some improvements. If my guess was incorrect, and you aren't routing traffic, then you should follow Robert's advice and try: sysctl net.isr.direct=1 And see whether load is splitted betweeen processes. If it is, then purchase second CPU. M> Does anyone know whether a dual CPU system can help us improve the M> situation? I was wondering if the software interrupt threads would be M> divided between the two processors. -- Totus tuus, Glebius. GLEBIUS-RIPN GLEB-RIPE
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060214074606.GH86448>