From owner-freebsd-hackers@freebsd.org Thu Aug 27 21:38:24 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D455A9C310A for ; Thu, 27 Aug 2015 21:38:24 +0000 (UTC) (envelope-from devin@shxd.cx) Received: from shxd.cx (mail.shxd.cx [64.201.244.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BEFA67F7; Thu, 27 Aug 2015 21:38:24 +0000 (UTC) (envelope-from devin@shxd.cx) Received: from 50-196-156-133-static.hfc.comcastbusiness.net ([50.196.156.133]:64962 helo=tinkerbell.pixel8networks.com) by shxd.cx with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1ZUxpM-000Lls-8f; Thu, 27 Aug 2015 07:06:52 -0700 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) Subject: Re: How to control and setup service? From: Devin Teske In-Reply-To: <988F49BB-5F42-4563-98AF-B8947FAC1EDC@FreeBSD.org> Date: Thu, 27 Aug 2015 14:38:25 -0700 Cc: freebsd-hackers@freebsd.org, Devin Teske Message-Id: <7D057F49-F1AC-4224-9D48-07E958F610F9@FreeBSD.org> References: <55DF261C.80009@freebsd.org> <988F49BB-5F42-4563-98AF-B8947FAC1EDC@FreeBSD.org> To: Allan Jude X-Mailer: Apple Mail (2.2102) Sender: devin@shxd.cx Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Aug 2015 21:38:25 -0000 My last e-mail needs some distilling perhaps =85 > On Aug 27, 2015, at 2:21 PM, Devin Teske wrote: >=20 >> On Aug 27, 2015, at 8:00 AM, Allan Jude > wrote: >>=20 >> sysrc automatically detects values from rc.conf.d directories, and = can >> edit them if you ask it to. It just defaults to writing your changes = to >> /etc/rc.conf >>=20 >> My puppet scripts use: >> sysrc -f /etc/rc.conf.d/varnish varnishd_identity=3D"PDX1-01" >>=20 >>=20 >=20 [snip] > You can better simulate this effect without the `-s service_name=92 = enhancement, > using the currently available `-f file=92 argument (more pedantic than = Allan Jude=92s > approach from above): >=20 > sysrc -f "/etc/rc.conf.d/$_name $( sysrc -n local_startup%/rc.d = )/rc.conf.d/$_name" vimage_enable >=20 > Where $_name is the service_name. > A `-s service_name=92 argument to sysrc would make this easier. >=20 Don=92t do that. That assumes that $local_startup is a single directory when in reality, it should be treated as a white-space separated list of one-or-more directories where startup scripts can live. NB: It also ignores the fact that some rc.d scripts call = load_rc_config() more than once with different arguments. E.g, dhclient > ASIDE: While I was diving into this, I discovered a code typo in = /etc/rc.subr > NB: The typo has gone unnoticed because it has no effect on outcome >=20 > =3D=3D=3D BEGIN DIFF =3D=3D=3D > --- rc.subr.orig 2015-08-27 12:56:24.445475772 -0700 > +++ rc.subr 2015-08-27 12:56:33.980474637 -0700 > @@ -1333,7 +1333,7 @@ load_rc_config() > _rc_conf_loaded=3Dtrue > fi > =20 > - for _d in /etc ${local_startup%*/rc.d}; do > + for _d in /etc ${local_startup%/rc.d}; do > if [ -f ${_d}/rc.conf.d/"$_name" ]; then > debug "Sourcing ${_d}/rc.conf.d/$_name" > . ${_d}/rc.conf.d/"$_name=94 > =3D=3D=3D END DIFF =3D=3D=3D >=20 Oh, looks like someone already fixed that=85 (thx Jilles) https://svnweb.freebsd.org/base?view=3Drevision&revision=3D286163 > FURTHER ASIDE: Why is the code in /etc/rc.subr treating > $local_startup as though it=92s a single item? There are many > locations in code that consider $local_startup to be a white- > space separated list of directories where rc.d scripts can live? > I think the above code should perhaps be changed to: >=20 > =3D=3D=3D BEGIN DIFF =3D=3D=3D > --- rc.subr.orig 2015-08-27 12:56:24.445475772 -0700 > +++ rc.subr 2015-08-27 13:53:30.976240109 -0700 > @@ -1333,7 +1333,8 @@ load_rc_config() > _rc_conf_loaded=3Dtrue > fi > =20 > - for _d in /etc ${local_startup%*/rc.d}; do > + for _d in /etc/rc.d $local_startup; do > + _d=3D"${_d%/rc.d}" > if [ -f ${_d}/rc.conf.d/"$_name" ]; then > debug "Sourcing ${_d}/rc.conf.d/$_name" > . ${_d}/rc.conf.d/"$_name" > =3D=3D=3D END DIFF =3D=3D=3D Lol, jilles arrived at the same exact change (albeit superfluous use of = curlies). https://svnweb.freebsd.org/base?view=3Drevision&revision=3D286163 >=20 >=20 >>>=20 >>> As a user I would love to have a cool tool to control and configure >>> services in way like OpenBSD's rcctl(8) does >>> = http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/rcctl.8?query=3D= rcctl = . >>> # cooltool set mysql-server status on >>> That's all. It would take care of those things like getting right >>> $rcvar name, choose right rc config file to edit (remember, we have = a >>> lot of places to check, see above) and enable service, etc. >>> This $cooltool can be an extended version of service(8) tool. >>> More difficult example: >>> # cooltool set flow_capture status on flags "-e 2200 -n 23 -N 0 -V = 5" >>> port "8787" datadir "/storage/flows/all_routers" >>> would enable flow_capture service and set other stuff using right = $rcvar. >>> # cooltool get flow_capture >>> would print all of the configured stuff for $service >>> # cooltool get flow_capture status >>> would print only a status YES/NO (don't forget about exit code ;) ) >>> and etc. >>=20 >> sysrc can do that, it will search all of the directories and find the >> final answer. >>=20 >> The only thing it doesn't do is the translation between the 'service >> name' (mysql-server) and the 'rcvar prefix' (mysql_), >=20 > The rcvar =93prefix" is not actually what you=92re after. > In load_rc_config() of /etc/rc.subr we can see that it uses $_name > to append to /etc/rc.conf.d/ and /usr/local/etc/rc.conf.d/ >=20 > What exactly is =93$_name=94 you ask? Good question=85 >=20 > Almost 100% of the time, it=92s whatever =93name=3D=93 in the rc.d = script. > This is not guaranteed to be ${0##*/} (the rc.d script name). > However, this just happens to be because almost 100% of the > time, rc.d scripts do the following: >=20 > load_rc_config "$name" >=20 > So for sysrc to know the proper name of the thing that /etc/rc.subr > will source from rc.conf.d collection of directories, we need to know > what name=3D in the rc.d script. >=20 > This seems to be the simplest approach: >=20 > =3D=3D=3D BEGIN SCRIPT: sconfigs.sh =3D=3D=3D > #!/bin/sh > sname=3D"$1" sconfigs=3D > _name=3D$( service "$sname" rcvar 2> /dev/null | > awk 'NR=3D=3D1&&sub(/^# /,""){print;exit}?' 2> /dev/null ) > if [ "$_name" ]; then > sconfigs=3D"/etc/rc.conf.d/$_name" > local_etc=3D$( sysrc -qn local_startup%/rc.d 2> /dev/null ) > [ "$local_etc" ] && > sconfigs=3D"$sconfigs $local_etc/rc.d/rc.conf.d/$_name" > sconfigs=3D"${sconfigs# }" > fi > echo "sconfigs=3D[$sconfigs]" > =3D=3D=3D END SCRIPT: sconfigs.sh =3D=3D=3D >=20 > Examples: >=20 > $ ./sconfigs.sh > sconfigs=3D[] > $ ./sconfigs.sh zfs > sconfigs=3D[/etc/rc.conf.d/zfs /usr/local/etc/rc.d/rc.conf.d/zfs] > $ ./sconfigs.sh jail > sconfigs=3D[/etc/rc.conf.d/jail /usr/local/etc/rc.d/rc.conf.d/jail] > $ ./sconfigs.sh nosuchservice > sconfigs=3D[] >=20 > So naturally, if the logic in sconfigs.sh yields a non-NULL > value for $sconfigs, this would become the value of `-f files=92 > via service name. >=20 The =93simplest=94 approach has edge-cases described below and should be avoided. The more holistic approach is below. >=20 >> which might be a >> useful addition, but might make more sense in the service command, >> because when I pass something to sysrc, I expect it to be interpreted >> literally. I guess it could be a flag for sysrc to specify the = service >> instead of the rcvar. >>=20 >> sysrc -s mysql datadir=3D/var/db/mysql >=20 > Given above talking-points, that would cause: >=20 > a. sysrc makes sure that =93mysql=94 is an actual service > b. sysrc uses =93service mysql rcvar" (piped into awk) to get `name=3D=91= value > NB: This could potentially result in a disconnect IFF the rc.d script = passes something other than =93$name=94 to load_rc_config() (if we=92re = worried about this, I have another recipe =97 further below). > c. Sets `-f files=92 value to "/etc/rc.conf.d/mysql = /usr/local/etc/rc.conf.d/mysql=94 >=20 > ASIDE: The only edge-case would be, for example... >=20 > =3D=3D=3D BEGIN FILE: /etc/rc.d/fooserv =3D=3D=3D > #!/bin/sh > . /etc/rc.subr > name=3DfoooooooooooooOOOooo > load_rc_config foooooooooo > run_rc_command "$@=93 > =3D=3D=3D END FILE: /etc/rc.d/fooserv =3D=3D=3D >=20 > Wherein we get the following from =93service =85 rcvar=94: >=20 > $ service fooserv rcvar | awk NR=3D=3D1 > # foooooooooooooOOOooo >=20 > The above command returns the =93name=3D=93 value but the script = doesn=92t > actually pass $name to load_rc_config() and thus what /etc/rc.subr > would check for in this case is /etc/rc.conf.d/foooooooooo and > /usr/local/etc/rc.conf.d/foooooooooo (versus the camel-case $name). >=20 > This edge-case exists for a number of services in the base: >=20 > $ awk '$1 ~ /^name=3D/ { name =3D $0; fn =3D FILENAME } $1 =3D=3D = "load_rc_config" && $2 !~ /\$(name|{name})/{ print "--"; if (fn =3D=3D = FILENAME) print fn ":" name; print FILENAME ":" $0 }' /etc/rc.d/* > -- > /etc/rc.d/dhclient:name=3D"dhclient" > /etc/rc.d/dhclient:load_rc_config network > -- > /etc/rc.d/fooserv:name=3DfoooooooooooooOOOooo > /etc/rc.d/fooserv:load_rc_config foooooooooo > -- > /etc/rc.d/initrandom:name=3D"initrandom" > /etc/rc.d/initrandom:load_rc_config random > -- > /etc/rc.d/othermta:load_rc_config 'XXX' > -- > /etc/rc.d/postrandom:name=3D"postrandom" > /etc/rc.d/postrandom:load_rc_config random > -- > /etc/rc.d/swaplate:name=3D"swaplate" > /etc/rc.d/swaplate:load_rc_config swap >=20 >=20 > NB: Ignore the /etc/rc.d/fooserv script which I created specifically = to illustrate the disconnect. >=20 > For the above edge-case services in base, load_rc_config() of = /etc/rc.subr will check the following: >=20 > dhclient: > /etc/rc.conf.d/network > /usr/local/etc/rc.conf.d/network > initrandom: > /etc/rc.conf.d/random > /usr/local/etc/rc.conf.d/random > othermta: > /etc/rc.conf.d/XXX > /usr/local/etc/rc.conf.d/XXX > postrandom: > /etc/rc.conf.d/random > /usr/local/etc/rc.conf.d/random > swaplate: > /etc/rc.conf.d/swap > /usr/local/etc/rc.conf.d/swap >=20 > Let=92s ignore for a second that it seems like an obvious error to be = sourcing > /usr/local/etc/rc.conf.d/ANYTHING for a base service in /etc/rc.d/ > NB: I can see someone utilizing that as a form of value-add anyways >=20 > What stands out at first glance is: >=20 > + If /etc/rc.conf.d/XXX or /usr/local/etc/rc.conf.d/XXX exists, > it will be sourced when working on the =93othermta=94 service >=20 > NB: My testing indicates that othermta can obtain the desired results > (of not sourcing any additional rc.conf.d files) by instead passing a > single =93.=94 to load_c_config() =97 which does not allow a NULL = arg1. >=20 > + The =93service rcvar | awk =85=94 approach to get =93name=3D=93= > value fails for dhclient, initrandom, postrandom, and swaplate. >=20 > Oh, but it gets worse when you look deeper at the above-mentioned = scripts: >=20 > $ grep -c load_rc_config /etc/rc.d/* | grep -v ':[01]$' | sed -e = 's/:[[:digit:]]\{1,\}$//' | xargs grep -n load_rc_config=20 > /etc/rc.d/dhclient:54:load_rc_config $name > /etc/rc.d/dhclient:55:load_rc_config network > /etc/rc.d/local_unbound:32:load_rc_config $name > /etc/rc.d/local_unbound:90:load_rc_config $name >=20 > It turns out that the dhclient script will actually source all of (if = any exist; in order): > 1. /etc/rc.conf.d/dhclient > 2. /usr/local/etc/rc.conf.d/dhclient > 3. /etc/rc.conf.d/network > 4. /usr/local/etc/rc.conf.d/network >=20 > I have a recipe that overcomes this edge-case that I=92d like to offer > as a potential solution: >=20 > =3D=3D=3D BEGIN FILE: sconfigs2.sh =3D=3D=3D > #!/bin/sh > sname=3D"$1" sconfigs=3D > local_startup=3D$( sysrc -qn local_startup ) > for dir in /etc/rc.d $local_startup; do > spath=3D"$dir/$sname" > [ -f "$spath" -a -x "$spath" ] || spath=3D continue > break > done > if [ ! "$spath" ]; then > echo "$sname does not exist in /etc/rc.d or the local startup" > echo "directories ($local_startup)" > exit 1 > fi > _names=3D > case "$( file -b "$spath" 2> /dev/null )" in *"shell script"*) > _names=3D$( exec 9<&1 1>&- 2>&- > last_name=3D > print_name() { > local name=3D"$1" > [ "$name" =3D "$last_name" ] && return > echo "$name" >&9 > last_name=3D"$name" > } > eval "$( awk '{ > gsub(/load_rc_config /, "print_name ") > gsub(/run_rc_command /, ": ") > print > }' "$spath" )" > ) ;; > esac > for _name in $_names; do > for dir in /etc/rc.d $local_startup; do > sconfigs=3D"$sconfigs ${dir%/rc.d}/rc.conf.d/$_name" > done > done > sconfigs=3D"${sconfigs# }" > echo "sconfigs=3D[$sconfigs]=94 > =3D=3D=3D END FILE: sconfigs2.sh =3D=3D=3D >=20 > Given the above code, we can handle the afore-mentioned edge-casen: >=20 > $ ./sconfigs2.sh dhclient > sconfigs=3D[/etc/rc.conf.d/dhclient /usr/local/etc/rc.conf.d/dhclient = /etc/rc.conf.d/network /usr/local/etc/rc.conf.d/network] > $ ./sconfigs2.sh initrandom > sconfigs=3D[/etc/rc.conf.d/random /usr/local/etc/rc.conf.d/random] > $ ./sconfigs2.sh postrandom > sconfigs=3D[/etc/rc.conf.d/random /usr/local/etc/rc.conf.d/random] > $ ./sconfigs2.sh swaplate > sconfigs=3D[/etc/rc.conf.d/swap /usr/local/etc/rc.conf.d/swap] > $ ./sconfigs2.sh local_unbound > sconfigs=3D[/etc/rc.conf.d/local_unbound = /usr/local/etc/rc.conf.d/local_unbound] > $ ./sconfigs2.sh=20 > does not exist in /etc/rc.d or the local startup > directories (/usr/local/etc/rc.d) > $ ./sconfigs2.sh nosuchservice > nosuchservice does not exist in /etc/rc.d or the local startup > directories (/usr/local/etc/rc.d) >=20 >=20 > This is looking like a much better approach. >=20 [snip] =97=20 Cheers, Devin=