Date: Sat, 13 Feb 1999 17:00:29 -0500 (EST) From: John Dowdal <jdowdal@destiny.erols.com> To: waterman@acm.org Cc: Andre Albsmeier <andre.albsmeier@mchp.siemens.de>, Stormy Henderson <stormy@futuresouth.com>, stable@FreeBSD.ORG Subject: Re: network slowdown with rc5des Message-ID: <Pine.BSF.4.05.9902131659020.739-100000@destiny.erols.com> In-Reply-To: <199902020546.VAA06347@home>
next in thread | previous in thread | raw e-mail | index | archive | help
I got my chance to repeat these experiments with the following parameters 3.0-stable cvsupped on 02/13/99 (only kernel recompiled) HZ=1000 Apparnatly HZ=1000 makes little or no difference. Inserting a 'usleep(0)' into the code makes a difference. John rc5des running, idprio 1 bigfile(bsd)->bigfile(windoze) : 1200 bigfile(windoze)->bigfile(bsd) : 3800 rc5des running, nice 20 bigfile(bsd)->bigfile(windoze) : 1300 bigfile(windoze)->bigfile(bsd) : 4200 rc5des running, idprio 31 bigfile(bsd)->bigfile(windoze) : 1200 bigfile(windoze)->bigfile(bsd) : 4400 no rc5des running bigfile(bsd)->bigfile(windoze) : 6900 bigfile(windoze)->bigfile(bsd) : 5100 instead of rc5des, run the following program main() { while(1) usleep(0); } Run above program at normal priority bigfile(bsd)->bigfile(windoze) : 5800 bigfile(windoze)->bigfile(bsd) : 2800 Run above program at idprio 1 bigfile(bsd)->bigfile(windoze) : 5800 bigfile(windoze)->bigfile(bsd) : 5008 On Mon, 1 Feb 1999, TS Waterman wrote: > > Andre Albsmeier writes: > >On Mon, 01-Feb-1999 at 13:27:45 -0500, John Dowdal wrote: > >> On Mon, 1 Feb 1999, Stormy Henderson wrote: > >> > >> > A happy camper (John Dowdal, jdowdal@destiny.erols.com) once wrote... > >> > > There is a dramatic network slowdown on 3.0-stable when usign 100BT > >> > > cards and rc5des at the same time. The rc5des process causes the > [...] > >> This should imply that it doesn't matter whether I use 1 or 31 for > >> priority, because I do not have any other processes with idle priority. > >> > >> Based on the behavior (and not te sources), it appears that the > >> ethernet-card-is-ready interrupt is placed into a queue to be scheduled > >> later, instead of immediately preempting the idprio processes. IT also > >> appears that the interrupt will preempt the kernel idle loop causing > >> better performance if I kill -STOP the rc5des process. The big question > >> is why the system behaves differently when its executing a idprio job and > >> its internal idle loop. > > [some actual empirical TESTS of the problem! wee need more posts like this!] > > >So we get: > > > >no process : 10170 KB/s > >sender running : 185 KB/s > >sender running with idprio 5 : 689 KB/s > >receiver running : 949 KB/s > >receiver running with idprio 5: 991 KB/s > > > > > >What bugs me is that the rates are so bad in the both idprio'ed > >cases. Is there an explanation for that behaviour? Is it possible > >to catch the case when the system is _really_ idle so my process > >runs only then? > > I arrived at the following explanation a while ago, after having > similar behavior (not networking related, though), and poking at > the kernel sources some: > > FreeBSD is not a real-time operationg system, and we shouldn't expect > it to be. The interrupt from the card may be dealt with immediately, > but the process that's dealing with the data isn't. > > The reading/sending process really _is_ placed in the queue behind > whatever is running at the moment, but at a high priority. > Processes are time-sliced, according to the 'hz' kernel variable. > (see the -curent archives for some discussion of upping this -- > 'hz and scheduling' will probably get you on track) > > If your idprio proc is running, and there aren't any interrupts, it > will take 1/hz for it be shut down in favor of whatever is on > the higher priority queues (3 separate queues are kept: "real-time", > normal, and idprio). If you're in the idle_loop, then the highest > thing on the queues is woken up and run. > The idprio proc (remember rc5des is a hog and never needs to wait) > will wake up whenever nothing else is on the queue, and will take 1/hz > to finish if there aren't any interrupts; and will take long enough > to do a context switch out if there are. > > If we're in a gap where the network proc has done it's thing with > the card, especially the sending proc, which might fill the card's > send buffer or whatever, it goes to sleep. The idprio stuff wakes > up, and then it's 1/hz until we get any more action. > > > In a similar vein, maybe even more responsible for this behavior, > the kernel networking code has some time-slice variables set as > multiples of 'hz'. But I haven't dug around in that stuff ever -- > someone who really knows should answer that part of things. > The given values were probably arrived at long before 100Base-T was. > > Try rebuilding a kernel with HZ set 10x higher or so, and run your test > numbers again. > > --ts > <waterman@acm.org> > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.05.9902131659020.739-100000>