Date: Wed, 19 Dec 2012 10:48:39 +0000 (UTC) From: Dag-Erling Smørgrav <des@FreeBSD.org> To: src-committers@freebsd.org, svn-src-user@freebsd.org Subject: svn commit: r244419 - user/des/tinderbox Message-ID: <201212191048.qBJAmds6074438@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: des Date: Wed Dec 19 10:48:39 2012 New Revision: 244419 URL: http://svnweb.freebsd.org/changeset/base/244419 Log: Greatly improve the handling, and particularly expansion, of multiple-value configuration variables. Switch RECIPIENT from single-value to multiple-value. Add code that strips @freebsd.org addresses from the list unless the user has jumped through the required hoops, since the stock configuration mails failure reports to the mailing lists. Revise and update the documentation. Bump $VERSION from 2.6 to 2.9 to match tinderbox(1). Modified: user/des/tinderbox/tbmaster.1 user/des/tinderbox/tbmaster.pl Modified: user/des/tinderbox/tbmaster.1 ============================================================================== --- user/des/tinderbox/tbmaster.1 Wed Dec 19 09:55:13 2012 (r244418) +++ user/des/tinderbox/tbmaster.1 Wed Dec 19 10:48:39 2012 (r244419) @@ -1,5 +1,5 @@ .\"- -.\" Copyright (c) 2003-2011 Dag-Erling Coïdan Smørgrav +.\" Copyright (c) 2003-2012 Dag-Erling Coïdan Smørgrav .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd November 12, 2008 +.Dd December 18, 2012 .Dt TBMASTER 1 .Os .Sh NAME @@ -59,12 +59,12 @@ respectively. The name of the configuration to use. If specified multiple times, all listed configurations will be run in sequence. -The default is the hostname minus the domain part. +The default value is the hostname minus the domain part. .It Fl d , Fl -dump Dumps the configuration and exits without running the tinderbox. .It Fl e Ar DIR , Fl -etcdir Ns = Ns Ar DIR The directory where configuration files are located. -The default is +The default value is .Pa $HOME/etc . .It Fl l Ar FILE The name of a file to lock upon startup. @@ -73,8 +73,8 @@ If the lock is already held by another p will terminate immediately rather than block. .It Fl n Ar ncpu The number of concurrent jobs to run. -The default is the number of CPUs in the machine, or 1 if that number -could not be determined. +The default value is the number of CPUs in the machine, or 1 if that +number could not be determined. .El .Ss Configuration The @@ -197,6 +197,14 @@ The default value is .Pq Vt single The machine currently being built for. Read-only. +.It OBJDIR +.Pq Vt single +The object directory. +There is no default value; see the +.Xr tinderbox 1 +script's +.Fl -objdir +option for details. .It OPTIONS .Pq Vt multiple A list of additional options to pass to the @@ -222,9 +230,16 @@ the architecture name. The default value is .Dq i386 . .It RECIPIENT -.Pq Vt single -The address to which failure reports should be mailed. -No default value. +.Pq Vt multiple +The addresses to which failure reports should be mailed. +The default value is +.Dq %%SENDER%% . +.Pp +To avoid unintentional spamming, +.Nm +will strip recipients in the +.Li freebsd.org +domain from this list unless the correct magic sauce is used. .It REPOSITORY .Pq Vt single The location of the @@ -244,6 +259,12 @@ No default value. .It SRCDIR .Pq Vt single The source directory. +There is no default value; see the +.Xr tinderbox 1 +script's +.Fl -srcdir +option for details. +.Pp Normally, a separate directory within the sandbox will be used for each build. Using a shared source directory for all builds reduces disk space @@ -261,45 +282,47 @@ The subject to use on failure reports. The default value is .Dq Tinderbox failure on %%arch%%/%%machine%% . .It SVNBASE +.Pq Vt single The URL to the base of the Subversion repository. .It TARGETS .Pq Vt multiple -A list of targets (commands) to specify to the +A list of targets (commands) to pass to the .Xr tinderbox 1 script. -The default is +The default value is .Dq world . .It TIMEOUT +.Pq Vt single The number of seconds after which each tinderbox invocation will time out. No default value. .It TINDERBOX +.Pq Vt single The location of the .Xr tinderbox 1 script. The default value is .Dq %%HOME%%/bin/tinderbox . .It URLBASE +.Pq Vt single If defined, a URL constructed by appending the file name of the full -log file to the expanded value of this variable will be appended to +log file to the expanded value of this variable will be included in failure reports. -No default value. +There is no default value. .El -.Pp -Fields which are left undefined are filled in from the -.Dq global -setup. .Ss Variable Substitution -All single-valued configuration variables are subject to variable -substitution immediately before use. -If a variable's value contains strings of the form +All configuration variables are subject to variable substitution +immediately before use: +.Bl -bullet +.It +If a single-value variable contains substrings of the form .Va %%VAR%% or .Va %%var%% , -those strings are replaced with the values of the corresponding -variables. +those substrings are replaced with the values of the corresponding +variables, after recursive substitution. The difference between the first and the second form is that the -latter expands to lower-case. +latter is converted to lower-case before use. For instance, .Dq %%BRANCH%% might expand to @@ -308,6 +331,18 @@ while .Dq %%branch%% would expand to .Dq releng_4 . +.It +If an element of a multiple-value variable is of the form +.Va %%VAR%% +or +.Va %%var%% +and the corresponding variable is a multiple-value variable, recursive +substitution is first performed on that variable, and the resulting +values are included individually in the result. +.It +Otherwise, elements of multiple-value variables are expanded +individually according to the same rules as single-value variables. +.El .Sh SEE ALSO .Xr perl 1 , .Xr tinderbox 1 Modified: user/des/tinderbox/tbmaster.pl ============================================================================== --- user/des/tinderbox/tbmaster.pl Wed Dec 19 09:55:13 2012 (r244418) +++ user/des/tinderbox/tbmaster.pl Wed Dec 19 10:48:39 2012 (r244419) @@ -33,8 +33,8 @@ use Fcntl qw(:DEFAULT :flock); use POSIX; use Getopt::Long; -my $VERSION = "2.6"; -my $COPYRIGHT = "Copyright (c) 2003-2011 Dag-Erling Coïdan Smørgrav. " . +my $VERSION = "2.9"; +my $COPYRIGHT = "Copyright (c) 2003-2012 Dag-Erling Coïdan Smørgrav. " . "All rights reserved."; my $abbreviate; # Abbreviate path names in log file @@ -52,15 +52,15 @@ my %INITIAL_CONFIG = ( 'COPTFLAGS' => '', 'COMMENT' => '', 'CVSUP' => '', - 'ENV' => [], + 'ENV' => [ ], 'HOSTNAME' => '', 'JOBS' => '', 'LOGDIR' => '%%SANDBOX%%/logs', 'OBJDIR' => '', - 'OPTIONS' => [], + 'OPTIONS' => [ ], 'PATCH' => '', 'PLATFORMS' => [ 'i386' ], - 'RECIPIENT' => '', + 'RECIPIENT' => [ '%%SENDER%%' ], 'REPOSITORY'=> '', 'SANDBOX' => '/tmp/tinderbox', 'SENDER' => '', @@ -111,13 +111,33 @@ sub expand($) { return "??$key??" unless exists($CONFIG{uc($key)}); - return $CONFIG{uc($key)} - if (ref($CONFIG{uc($key)})); - my $str = $CONFIG{uc($key)}; - while ($str =~ s/\%\%(\w+)\%\%/expand($1)/eg) { - # nothing + my $value = $CONFIG{uc($key)}; + my @elements = ref($value) ? @{$value} : $value; + my @expanded; + while (@elements) { + my $elem = shift(@elements); + if (ref($elem)) { + # prepend to queue for further processing + unshift(@elements, @{$elem}); + } elsif ($elem =~ m/^\%\%(\w+)\%\%$/) { + # prepend to queue for further processing + # note - can expand to a list + unshift(@elements, expand($1)); + } else { + $elem =~ s/\%\%(\w+)\%\%/expand($1)/eg; + push(@expanded, $elem); + } + } + if ($key !~ m/[A-Z]/) { + @expanded = map { lc($_) } @expanded; + } + if (ref($value)) { + return @expanded; + } elsif (@expanded != 1) { + die("expand($key): expected one value, got ", int(@expanded), "\n"); + } else { + return $expanded[0]; } - return ($key =~ m/[A-Z]/) ? $str : lc($str); } # @@ -289,7 +309,7 @@ sub tinderbox($$$) { } # Fork and start the tinderbox - my @args = @{$CONFIG{'OPTIONS'}}; + my @args = expand('OPTIONS'); push(@args, "--hostname=" . expand('HOSTNAME')); push(@args, "--sandbox=" . realpath(expand('SANDBOX'))); push(@args, "--srcdir=" . realpath(expand('SRCDIR'))) @@ -311,8 +331,8 @@ sub tinderbox($$$) { if ($CONFIG{'SVNBASE'}); push(@args, "--timeout=" . expand('TIMEOUT')) if ($CONFIG{'TIMEOUT'}); - push(@args, @{$CONFIG{'TARGETS'}}); - push(@args, @{$CONFIG{'ENV'}}); + push(@args, expand('TARGETS')); + push(@args, expand('ENV')); push(@args, "CFLAGS=" . expand('CFLAGS')) if ($CONFIG{'CFLAGS'}); push(@args, "COPTFLAGS=" . expand('COPTFLAGS')) @@ -397,10 +417,17 @@ sub tinderbox($$$) { # Record result in history file history($start, $end, !$error); + # Filter recipients + my @recipients = expand('RECIPIENT'); + if (!$ENV{'MAGIC_SAUCE'} || + $ENV{'MAGIC_SAUCE'} ne 'FREEBSD_TINDERBOX') { + @recipients = grep { ! m/\@freebsd\.org/i } @recipients; + } + # Mail out error reports - if ($error && $CONFIG{'RECIPIENT'}) { + if ($error && @recipients) { my $sender = expand('SENDER'); - my $recipient = expand('RECIPIENT'); + my $recipient = join(', ', @recipients); my $subject = expand('SUBJECT'); if ($CONFIG{'URLBASE'}) { $summary .= "\n\n" . expand('URLBASE') . "$logfile.full"; @@ -512,8 +539,8 @@ sub tbmaster($) { my $stopfile = expand('SANDBOX') . "/stop"; my @jobs; - foreach my $branch (@{$CONFIG{'BRANCHES'}}) { - foreach my $platform (@{$CONFIG{'PLATFORMS'}}) { + foreach my $branch (expand('BRANCHES')) { + foreach my $platform (expand('PLATFORMS')) { next if (%platforms && !$platforms{$platform}); my ($arch, $machine) = split('/', $platform, 2); $machine = $arch
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201212191048.qBJAmds6074438>