Date: Sun, 18 Oct 2015 10:06:35 -0500 From: Joe Maloney <jmaloney@pcbsd.org> To: Cyril Vechera <cv@jet9.net> Cc: Mark Felder <feld@FreeBSD.org>, freebsd-hackers@freebsd.org, freebsd-rc@freebsd.org, marck@rinet.ru Subject: Re: rc(8) parallel tasks Message-ID: <6D32B114-453C-4D25-8FF4-C8777D78C50C@pcbsd.org> In-Reply-To: <562143E7.3030104@jet9.net> References: <560EAC05.6050308@jet9.net> <1445011561.1233840.412174489.688C5822@webmail.messagingengine.com> <562143E7.3030104@jet9.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks for sharing this. It worked great for me on FreeBSD as well with = minimal services, and does indeed make a difference. It blew up on = PCBSD which has many more services running out of box. I also noticed = /tmp/init has to be removed manually in some cases when boot hangs. = Very impressive otherwise. Joe Maloney > On Oct 16, 2015, at 1:37 PM, Cyril Vechera <cv@jet9.net> wrote: >=20 > On 10/16/2015 07:06 PM, Mark Felder wrote: >> On Fri, Oct 2, 2015, at 11:08, Cyril Vechera wrote: >>> Hi there. >>>=20 >>> We've got a small launcher script (~250 loc) for parallel services >>> start/stop etc. It is used on our embedded systems and our users >>> containers. And I've done a proof of concept for implanting it to = the >>> FreeBSD's standard /etc/rc for execution starting scripts in = parallel. >>> It gave me a boot time reduction of rc part from 27 to 7 seconds, = mostly >>> on eliminating jams for network or other long-latency resources = waiting. >>>=20 >>> The launcher is written in pure POSIX shell and uses FIFOs (named = pipes) >>> as a mutexes for synchronization. So it is embedded into /etc/rc and >>> /etc/rc.d preserving rc.subr preloading. As a primary requirement, = it >>> guarantees topological order (strict partial order) defined by >>> dependencies. It requires only POSIX shell, FreeBSD or Linux kernel, >>> mkfifo and a writeable file system. Due to last requirement, it can = be >>> run on the late stage or should be supplied by some kinf of = writtable >>> fs, ie tmpfs. The FreeBSD-integrated version uses standard rcorder >>> annotations (REQUIRE, BEFORE and PROVIDE) and there's no need to = change >>> rc.d scripts >>>=20 >>> It's not a full init replacement or a kind of services supervision = tool. >>> It only starts or invokes a group of scripts in parallel with = resolving >>> and assuring execution in dependencies order. >>>=20 >>> Please take a look at the script and patch set for FreeBSD: >>>=20 >>> = https://github.com/cvss/jet9-multitask-init/blob/master/jet9-multitask-ini= t >>> = https://github.com/cvss/jet9-multitask-init/tree/master/examples/freebsd >>>=20 >>>=20 >> Your first link is a 404, but this looks really nice! >=20 > In the last commit (v1.3.0) I've renamed 'jet9-multitask-init' to = 'jet9-multitask-flow' to avoid confusion with naming, because it's not = as it seems, from name, an init replacement, but just a parallel task = launcher. And now the actual repository URL is = https://github.com/cvss/jet9-multitask-flow = <https://github.com/cvss/jet9-multitask-flow>; >=20 > In this commit I've complete the FreeBSD compatibility. Now a script = or dependency name can include minuses `-` and dots `.` (the first stone = I stumbled over was ftp-proxy). And I've cleaned up the main script = code https://github.com/cvss/jet9-multitask-flow/jet9-multitask-flow = <https://github.com/cvss/jet9-multitask-flow/jet9-multitask-flow>; and = have split it to more functions that can be redefined. So it's now = easier to rewrite dependency extraction for FreeBSD rc-scripts - current = implementation is too rough and takes three 'awk' runs for each = rc-script. The last is not only time loss, but as DMarck mentioned = before, using awk restricts parallel rc to be run only after FILESYSTEMS = stage is done. Maybe it would be better to add to the rcorder(8) some = new option to dump the gathered dependencies in tsort-compatible listing = and insert them directly to flow execution plan. >=20 > I've also added rc.conf variable `rc_parallel` to turn on and off = parallel execution. There's a risk to discover an incomplete dependency = annotations in some rc-scripts that earlier were masked by serial = execution. I've done some checks by enabling as much rc.conf variables = as possible to start more rc-scripts, and didn't found any error. But it = looks too good to be true and I'm afraid that it's just to poor testing. = I think if some ordering conflict will be found, it could be = worked-around with introducing a white-list for script names that must = be run only in sequentially. >=20 > So it remained first to check if it really works in different = conditions. >=20 >=20 >=20 >=20 > --=20 > Cyril Vechera >=20 > _______________________________________________ > freebsd-hackers@freebsd.org <mailto:freebsd-hackers@freebsd.org> = mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers = <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>; > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org = <mailto:freebsd-hackers-unsubscribe@freebsd.org>"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6D32B114-453C-4D25-8FF4-C8777D78C50C>