Date: Tue, 2 Feb 1999 19:25:23 +0100 From: Andre Albsmeier <andre.albsmeier@mchp.siemens.de> To: waterman@acm.org Cc: stable@FreeBSD.ORG Subject: Re: network slowdown with rc5des Message-ID: <19990202192523.A6023@internal> In-Reply-To: <199902020546.VAA06347@home>; from TS Waterman on Mon, Feb 01, 1999 at 09:46:52PM -0800 References: <19990201212954.A17716@internal> <19990201111258.A89636@rain.futuresouth.com> <Pine.BSF.4.05.9902011317000.8524-100000@destiny.erols.com> <19990201212954.A17716@internal> <199902020546.VAA06347@home>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 01-Feb-1999 at 21:46:52 -0800, TS Waterman wrote: Thanks for your explanation. I came to a similar point while thinking about it. However, I don't know which sideeffects I am going to have if I change HZ and since these are important machines, I will stick with my crontab controlled 'kill -STOP PID' :-) Thanks again, -Andre > Andre Albsmeier writes: > >On Mon, 01-Feb-1999 at 13:27:45 -0500, John Dowdal wrote: > >> On Mon, 1 Feb 1999, Stormy Henderson wrote: > >> > >> > A happy camper (John Dowdal, jdowdal@destiny.erols.com) once wrote... > >> > > There is a dramatic network slowdown on 3.0-stable when usign 100BT > >> > > cards and rc5des at the same time. The rc5des process causes the > [...] > >> This should imply that it doesn't matter whether I use 1 or 31 for > >> priority, because I do not have any other processes with idle priority. > >> > >> Based on the behavior (and not te sources), it appears that the > >> ethernet-card-is-ready interrupt is placed into a queue to be scheduled > >> later, instead of immediately preempting the idprio processes. IT also > >> appears that the interrupt will preempt the kernel idle loop causing > >> better performance if I kill -STOP the rc5des process. The big question > >> is why the system behaves differently when its executing a idprio job and > >> its internal idle loop. > > [some actual empirical TESTS of the problem! wee need more posts like this!] > > >So we get: > > > >no process : 10170 KB/s > >sender running : 185 KB/s > >sender running with idprio 5 : 689 KB/s > >receiver running : 949 KB/s > >receiver running with idprio 5: 991 KB/s > > > > > >What bugs me is that the rates are so bad in the both idprio'ed > >cases. Is there an explanation for that behaviour? Is it possible > >to catch the case when the system is _really_ idle so my process > >runs only then? > > I arrived at the following explanation a while ago, after having > similar behavior (not networking related, though), and poking at > the kernel sources some: > > FreeBSD is not a real-time operationg system, and we shouldn't expect > it to be. The interrupt from the card may be dealt with immediately, > but the process that's dealing with the data isn't. > > The reading/sending process really _is_ placed in the queue behind > whatever is running at the moment, but at a high priority. > Processes are time-sliced, according to the 'hz' kernel variable. > (see the -curent archives for some discussion of upping this -- > 'hz and scheduling' will probably get you on track) > > If your idprio proc is running, and there aren't any interrupts, it > will take 1/hz for it be shut down in favor of whatever is on > the higher priority queues (3 separate queues are kept: "real-time", > normal, and idprio). If you're in the idle_loop, then the highest > thing on the queues is woken up and run. > The idprio proc (remember rc5des is a hog and never needs to wait) > will wake up whenever nothing else is on the queue, and will take 1/hz > to finish if there aren't any interrupts; and will take long enough > to do a context switch out if there are. > > If we're in a gap where the network proc has done it's thing with > the card, especially the sending proc, which might fill the card's > send buffer or whatever, it goes to sleep. The idprio stuff wakes > up, and then it's 1/hz until we get any more action. > > > In a similar vein, maybe even more responsible for this behavior, > the kernel networking code has some time-slice variables set as > multiples of 'hz'. But I haven't dug around in that stuff ever -- > someone who really knows should answer that part of things. > The given values were probably arrived at long before 100Base-T was. > > Try rebuilding a kernel with HZ set 10x higher or so, and run your test > numbers again. > > --ts > <waterman@acm.org> > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-stable" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19990202192523.A6023>