Date: Sat, 17 Oct 2015 00:14:01 +0200 From: Mark Martinec <Mark.Martinec+freebsd@ijs.si> To: freebsd-hackers@freebsd.org, freebsd-rc@freebsd.org Cc: Cyril Vechera <cv@jet9.net> Subject: Re: rc(8) parallel tasks Message-ID: <2c307d32ddd8ed582ac496d56a75c3e4@mailbox.ijs.si> In-Reply-To: <562143E7.3030104@jet9.net> References: <560EAC05.6050308@jet9.net> <1445011561.1233840.412174489.688C5822@webmail.messagingengine.com> <562143E7.3030104@jet9.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Tried a today's git checkout on one of our small virtual 10.2 boxes with not many services. The installation is almost trivial and a reboot went without any problems. Time for a reboot was shrunk by about 10 seconds (which is understandable with a small number of services). Very good, almost too good to be true for such fairly small and non-intrusive piece of shell/awk scripting - thanks! Mark 2015-10-16 20:37, Cyril Vechera wrote: > On 10/16/2015 07:06 PM, Mark Felder wrote: >> On Fri, Oct 2, 2015, at 11:08, Cyril Vechera wrote: >>> Hi there. >>> >>> We've got a small launcher script (~250 loc) for parallel services >>> start/stop etc. It is used on our embedded systems and our users >>> containers. And I've done a proof of concept for implanting it to the >>> FreeBSD's standard /etc/rc for execution starting scripts in >>> parallel. >>> It gave me a boot time reduction of rc part from 27 to 7 seconds, >>> mostly >>> on eliminating jams for network or other long-latency resources >>> waiting. >>> >>> The launcher is written in pure POSIX shell and uses FIFOs (named >>> pipes) >>> as a mutexes for synchronization. So it is embedded into /etc/rc and >>> /etc/rc.d preserving rc.subr preloading. As a primary requirement, it >>> guarantees topological order (strict partial order) defined by >>> dependencies. It requires only POSIX shell, FreeBSD or Linux kernel, >>> mkfifo and a writeable file system. Due to last requirement, it can >>> be >>> run on the late stage or should be supplied by some kinf of writtable >>> fs, ie tmpfs. The FreeBSD-integrated version uses standard rcorder >>> annotations (REQUIRE, BEFORE and PROVIDE) and there's no need to >>> change >>> rc.d scripts >>> >>> It's not a full init replacement or a kind of services supervision >>> tool. >>> It only starts or invokes a group of scripts in parallel with >>> resolving >>> and assuring execution in dependencies order. >>> >>> Please take a look at the script and patch set for FreeBSD: >>> >>> https://github.com/cvss/jet9-multitask-init/blob/master/jet9-multitask-init >>> https://github.com/cvss/jet9-multitask-init/tree/master/examples/freebsd >>> >>> >> Your first link is a 404, but this looks really nice! > > In the last commit (v1.3.0) I've renamed 'jet9-multitask-init' to > 'jet9-multitask-flow' to avoid confusion with naming, because it's not > as it seems, from name, an init replacement, but just a parallel task > launcher. And now the actual repository URL is > https://github.com/cvss/jet9-multitask-flow > > In this commit I've complete the FreeBSD compatibility. Now a script > or dependency name can include minuses `-` and dots `.` (the first > stone I stumbled over was ftp-proxy). And I've cleaned up the main > script code > https://github.com/cvss/jet9-multitask-flow/jet9-multitask-flow and > have split it to more functions that can be redefined. So it's now > easier to rewrite dependency extraction for FreeBSD rc-scripts - > current implementation is too rough and takes three 'awk' runs for > each rc-script. The last is not only time loss, but as DMarck > mentioned before, using awk restricts parallel rc to be run only after > FILESYSTEMS stage is done. Maybe it would be better to add to the > rcorder(8) some new option to dump the gathered dependencies in > tsort-compatible listing and insert them directly to flow execution > plan. > > I've also added rc.conf variable `rc_parallel` to turn on and off > parallel execution. There's a risk to discover an incomplete > dependency annotations in some rc-scripts that earlier were masked by > serial execution. I've done some checks by enabling as much rc.conf > variables as possible to start more rc-scripts, and didn't found any > error. But it looks too good to be true and I'm afraid that it's just > to poor testing. I think if some ordering conflict will be found, it > could be worked-around with introducing a white-list for script names > that must be run only in sequentially. > > So it remained first to check if it really works in different > conditions.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2c307d32ddd8ed582ac496d56a75c3e4>