Date: Thu, 28 Jan 1999 11:11:18 -0500 (EST) From: "John S. Dyson" <dyson@iquest.net> To: wes@softweyr.com (Wes Peters) Cc: dyson@iquest.net, dillon@apollo.backplane.com, toasty@home.dragondata.com, hackers@FreeBSD.ORG Subject: Re: High Load cron patches - comments? Message-ID: <199901281611.LAA21412@y.dyson.net> In-Reply-To: <36B06FC5.1EA65667@softweyr.com> from Wes Peters at "Jan 28, 99 07:10:13 am"
next in thread | previous in thread | raw e-mail | index | archive | help
Wes Peters said:
> >
> > I can develop some rather complete and lossless calculations that
> > provide accurate and useful load indicators. In fact, I think that
> > a generic load mgmt scheme might be useful.
>
> Especially as we start diving more into SMP and threaded applications;
> which will need some effective means of throttling themselves. The
> problem with Matt's comment above is he doesn't offer any useful
> alternative, and couting child processes just isn't an effective means
> of throttling the overall load on a machine.
>
It is *sometimes* appropriate to criticize, even when alternatives aren't
provided. The kind of technique that I have successfully experimented with
is a scheme that has two phases: A costing mechanism and a stats mechanism.
The costing mechanism is a direct call from when the resource is attempted
to be allocated. It checks immediately if the cost (and recent incurred
costs, since the last stats run) is greater than a threshold. If that
threshold is attained, then the process blocks until the stats mechanism
sees that the recent peak + less recently decayed peak is less than a certain
threshold.
The scheme is as follows (note the lack of arrays -- this is a single resource
example). See that this looks like a nearly peak responding filter with a
decay. I am also using floats to throw out scale factor issues:
float rate_resource_history;
float rate_resource_recent;
float rate_resource_threshold; (In items / sec )
#define TC .1 seconds;
#define TCATTACK .50 /* Attack time is approx TC/TCATTACK */
#define TCDECAY .02 /* Decay time is approx TC/TCDECAY */
rate_resource_charge()
{
rate_resource_recent += 1;
while ((rate_resource_recent / TC) + rate_resource_history > rate_resource_threshold) {
block_on_rate_resource_history;
}
}
rate_resource_scan_TCorso()
{
float new_rate_resource_history, new_rate_resource_recent, new_rate_resource_add;
new_rate_resource_add = rate_resource_recent * TCATTACK;
rate_resource_recent -= new_rate_resource_add;
if (new_rate_resource_add > new_rate_resource_history)
new_rate_resource_history = new_rate_resource_add;
else
new_rate_resource_history = TCDECAY * new_rate_resource_add +
(1 - TCDECAY) * rate_resource_history;
rate_resource_history = new_rate_resource_history;
if (wakeup_needed &&
((rate_resource_history + rate_resource_recent) < rate_resource_threshold))
wakeup_rate_resource_history;
}
}
There might be some errors in the above (because this was interpreted from my
VM test code), but you can see the peak sensing behavior between TC sample rate
samples. There is also an average decay behavior, and there is no way that the
resource can be over allocated (in a rate sense), even in a peak rate sense (esp if
the TCATTACK is set to 1.0.) The attack time allows for bursty behavior, and
still limits the rate.
This concept can also be extended in a hierarchical fashion. Problems with this
include the issue of integer truncation (which can be dealt with by scaling.) No
information is lost by using a slightly slower attack time also -- which does
alot to allow for some burstiness without forgetting about it.
This code also doesn't loose information, so hierarchical designs might make
sense.
--
John | Never try to teach a pig to sing,
dyson@iquest.net | it makes one look stupid
jdyson@nc.com | and it irritates the pig.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199901281611.LAA21412>