Date: Sat, 13 Feb 1999 17:00:29 -0500 (EST) From: John Dowdal <jdowdal@destiny.erols.com> To: waterman@acm.org Cc: Andre Albsmeier <andre.albsmeier@mchp.siemens.de>, Stormy Henderson <stormy@futuresouth.com>, stable@FreeBSD.ORG Subject: Re: network slowdown with rc5des Message-ID: <Pine.BSF.4.05.9902131659020.739-100000@destiny.erols.com> In-Reply-To: <199902020546.VAA06347@home>
next in thread | previous in thread | raw e-mail | index | archive | help
I got my chance to repeat these experiments with the following parameters
3.0-stable cvsupped on 02/13/99 (only kernel recompiled)
HZ=1000
Apparnatly HZ=1000 makes little or no difference. Inserting a 'usleep(0)'
into the code makes a difference.
John
rc5des running, idprio 1
bigfile(bsd)->bigfile(windoze) : 1200
bigfile(windoze)->bigfile(bsd) : 3800
rc5des running, nice 20
bigfile(bsd)->bigfile(windoze) : 1300
bigfile(windoze)->bigfile(bsd) : 4200
rc5des running, idprio 31
bigfile(bsd)->bigfile(windoze) : 1200
bigfile(windoze)->bigfile(bsd) : 4400
no rc5des running
bigfile(bsd)->bigfile(windoze) : 6900
bigfile(windoze)->bigfile(bsd) : 5100
instead of rc5des, run the following program
main() {
while(1) usleep(0);
}
Run above program at normal priority
bigfile(bsd)->bigfile(windoze) : 5800
bigfile(windoze)->bigfile(bsd) : 2800
Run above program at idprio 1
bigfile(bsd)->bigfile(windoze) : 5800
bigfile(windoze)->bigfile(bsd) : 5008
On Mon, 1 Feb 1999, TS Waterman wrote:
>
> Andre Albsmeier writes:
> >On Mon, 01-Feb-1999 at 13:27:45 -0500, John Dowdal wrote:
> >> On Mon, 1 Feb 1999, Stormy Henderson wrote:
> >>
> >> > A happy camper (John Dowdal, jdowdal@destiny.erols.com) once wrote...
> >> > > There is a dramatic network slowdown on 3.0-stable when usign 100BT
> >> > > cards and rc5des at the same time. The rc5des process causes the
> [...]
> >> This should imply that it doesn't matter whether I use 1 or 31 for
> >> priority, because I do not have any other processes with idle priority.
> >>
> >> Based on the behavior (and not te sources), it appears that the
> >> ethernet-card-is-ready interrupt is placed into a queue to be scheduled
> >> later, instead of immediately preempting the idprio processes. IT also
> >> appears that the interrupt will preempt the kernel idle loop causing
> >> better performance if I kill -STOP the rc5des process. The big question
> >> is why the system behaves differently when its executing a idprio job and
> >> its internal idle loop.
>
> [some actual empirical TESTS of the problem! wee need more posts like this!]
> > >So we get:
> >
> >no process : 10170 KB/s
> >sender running : 185 KB/s
> >sender running with idprio 5 : 689 KB/s
> >receiver running : 949 KB/s
> >receiver running with idprio 5: 991 KB/s
> >
> >
> >What bugs me is that the rates are so bad in the both idprio'ed
> >cases. Is there an explanation for that behaviour? Is it possible
> >to catch the case when the system is _really_ idle so my process
> >runs only then?
>
> I arrived at the following explanation a while ago, after having
> similar behavior (not networking related, though), and poking at
> the kernel sources some:
>
> FreeBSD is not a real-time operationg system, and we shouldn't expect
> it to be. The interrupt from the card may be dealt with immediately,
> but the process that's dealing with the data isn't.
>
> The reading/sending process really _is_ placed in the queue behind
> whatever is running at the moment, but at a high priority.
> Processes are time-sliced, according to the 'hz' kernel variable.
> (see the -curent archives for some discussion of upping this --
> 'hz and scheduling' will probably get you on track)
>
> If your idprio proc is running, and there aren't any interrupts, it
> will take 1/hz for it be shut down in favor of whatever is on
> the higher priority queues (3 separate queues are kept: "real-time",
> normal, and idprio). If you're in the idle_loop, then the highest
> thing on the queues is woken up and run.
> The idprio proc (remember rc5des is a hog and never needs to wait)
> will wake up whenever nothing else is on the queue, and will take 1/hz
> to finish if there aren't any interrupts; and will take long enough
> to do a context switch out if there are.
>
> If we're in a gap where the network proc has done it's thing with
> the card, especially the sending proc, which might fill the card's
> send buffer or whatever, it goes to sleep. The idprio stuff wakes
> up, and then it's 1/hz until we get any more action.
>
>
> In a similar vein, maybe even more responsible for this behavior,
> the kernel networking code has some time-slice variables set as
> multiples of 'hz'. But I haven't dug around in that stuff ever --
> someone who really knows should answer that part of things.
> The given values were probably arrived at long before 100Base-T was.
>
> Try rebuilding a kernel with HZ set 10x higher or so, and run your test
> numbers again.
>
> --ts
> <waterman@acm.org>
>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.05.9902131659020.739-100000>