Date: Thu, 27 Apr 2006 16:38:14 +0200 From: Jeremie Le Hen <jeremie@le-hen.org> To: Robert Watson <rwatson@FreeBSD.org> Cc: Marcos Bedinelli <bedinelli@madhaus.cns.utoronto.ca>, freebsd-net@freebsd.org Subject: Re: [fbsd] Re: [fbsd] Network performance in a dual CPU system Message-ID: <20060427143814.GD84148@obiwan.tataz.chchile.org> In-Reply-To: <20060427145252.I75848@fledge.watson.org> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <20060427093916.GC84148@obiwan.tataz.chchile.org> <20060427145252.I75848@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, Robert, On Thu, Apr 27, 2006 at 02:54:21PM +0100, Robert Watson wrote: > > On Thu, 27 Apr 2006, Jeremie Le Hen wrote: > > >> PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND > >> 60 root 1 -44 -163 0K 8K WAIT 355.6H 72.17% swi1: > >>net > >> 39 root 1 -68 -187 0K 8K WAIT 52.3H 5.22% irq28: > >>bge0 > >> 40 root 1 -68 -187 0K 8K WAIT 28.3H 2.25% irq29: > >>bge1 > >> 11 root 1 171 52 0K 8K RUN 166.6H 0.00% idle > >> 63 root 1 -16 0 0K 8K - 121:55 0.00% yarrow > >> 61 root 1 -32 -151 0K 8K WAIT 46:21 0.00% swi4: > >>clock sio > >>[...] > >> > >>Does anyone know whether a dual CPU system can help us improve the > >>situation? I was wondering if the software interrupt threads would be > >>divided between the two processors. > > > >I am a few weeks late, I just saw this very interesting thread. What > >solution did you finally employ to circumvent your high interrupt load ? > > I missed the original thread, but in answer to the question: if you set > net.isr.direct=1, then FreeBSD 6.x will run the netisr code in the ithread > of the network device driver. This will allow the IP forwarding and > related paths in two threads instead of one, potentially allowing greater > parallelism. Of course, you also potentially contend more locks, you may > increase the time it takes for the ithread to respond to new interrupts, > etc, so it's not quite cut and dry, but with a workload like the one shown > above, it might make quite a difference. Actually you already replied in the original thread, explaining mostly the same thing. The whole thread [1] brought up multiple valuable network performance tuning knobs, such as polling, fastforwarding, net.isr.direct but there is no happy end to the thread. Given this is a real world situation, I wanted to know how Marcos revolved his problem. BTW, what I understand is that net.isr.direct=1 prevents from multiplexing all packets on the netisr thread and instead makes the ithread do the job. In this case, what happens to the netisr thread ? Does it still have some work to do or is it removed ? Thank you. Regards, [1] http://lists.freebsd.org/pipermail/freebsd-net/2006-February/thread.html#9725 -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060427143814.GD84148>