From owner-svn-src-user@FreeBSD.ORG Mon Mar 24 22:34:23 2014 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A39677F; Mon, 24 Mar 2014 22:34:23 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2AFA630F; Mon, 24 Mar 2014 22:34:23 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s2OMYNwu033604; Mon, 24 Mar 2014 22:34:23 GMT (envelope-from des@svn.freebsd.org) Received: (from des@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s2OMYMG5033600; Mon, 24 Mar 2014 22:34:22 GMT (envelope-from des@svn.freebsd.org) Message-Id: <201403242234.s2OMYMG5033600@svn.freebsd.org> From: Dag-Erling Smørgrav Date: Mon, 24 Mar 2014 22:34:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org Subject: svn commit: r263695 - user/des/tinderbox X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Mar 2014 22:34:23 -0000 Author: des Date: Mon Mar 24 22:34:22 2014 New Revision: 263695 URL: http://svnweb.freebsd.org/changeset/base/263695 Log: tinderbox: use the build-tool version of config(8) if available. tbmaster: implement subtraction from multiple-value variables. both: fix some taint issues. Modified: user/des/tinderbox/tbmaster.1 user/des/tinderbox/tbmaster.pl user/des/tinderbox/tinderbox.1 user/des/tinderbox/tinderbox.pl Modified: user/des/tinderbox/tbmaster.1 ============================================================================== --- user/des/tinderbox/tbmaster.1 Mon Mar 24 20:30:39 2014 (r263694) +++ user/des/tinderbox/tbmaster.1 Mon Mar 24 22:34:22 2014 (r263695) @@ -1,5 +1,5 @@ .\"- -.\" Copyright (c) 2003-2013 Dag-Erling Smørgrav +.\" Copyright (c) 2003-2014 Dag-Erling Smørgrav .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd October 2, 2013 +.Dd March 24, 2014 .Dt TBMASTER 1 .Os .Sh NAME @@ -115,13 +115,33 @@ configuration files. Each configuration file consists of a list of single- or multiple-value variable assignments: .Bl -tag -.It Va single_variable = Ar value -.It Va multi_variable = Ar value1 Op , Ar value2 ... -.It Va multi_variable += Ar value3 Op , Ar value4 ... +.It Va variable No = Ar value +Assigns +.Ar value +to the single-value variable +.Va variable . +.It Va variable No = Ar value1 Op No , Ar value2 ... +Assigns +.Ar value1 , +.Ar value2 +etc. to the multi-value variable +.Va variable . +.It Va variable No += Ar value3 Op No , Ar value4 ... +Appends +.Ar value3 , +.Ar value4 +etc. to the multi-value variable +.Va variable . +.It Va variable No -= Ar value5 Op No , Ar value6 ... +Removes +.Ar value5 , +.Ar value6 +etc. from the multi-value variable +.Va variable . .El .Pp -Whitespace around the equal sign and around the commas separating -multiple values is optional. +Whitespace around the assigment operator and around the commas +separating multiple values is optional. .Pp Blank lines are ignored, as is anything following a hash sign .Pq Sq # . Modified: user/des/tinderbox/tbmaster.pl ============================================================================== --- user/des/tinderbox/tbmaster.pl Mon Mar 24 20:30:39 2014 (r263694) +++ user/des/tinderbox/tbmaster.pl Mon Mar 24 22:34:22 2014 (r263695) @@ -1,6 +1,6 @@ #!/usr/bin/perl -Tw #- -# Copyright (c) 2003-2013 Dag-Erling Smørgrav +# Copyright (c) 2003-2014 Dag-Erling Smørgrav # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -34,8 +34,8 @@ use POSIX; use Getopt::Long; use Storable qw(dclone); -my $VERSION = "2.20"; -my $COPYRIGHT = "Copyright (c) 2003-2013 Dag-Erling Smørgrav. " . +my $VERSION = "2.21"; +my $COPYRIGHT = "Copyright (c) 2003-2014 Dag-Erling Smørgrav. " . "All rights reserved."; my $BACKLOG = 8; @@ -204,7 +204,7 @@ sub readconf($) { if ($line =~ m/^include\s+([\w-]+)$/) { readconf("$1.rc") or die("$fn: include $1: $!\n"); - } elsif ($line =~ m/^(\w+)\s*([+]?=)\s*(.*)$/) { + } elsif ($line =~ m/^(\w+)\s*([+-]?=)\s*(.*)$/) { my ($key, $op, $val) = (uc($1), $2, $3); $val = '' unless defined($val); @@ -219,6 +219,10 @@ sub readconf($) { $CONFIG{$key} = \@a; } elsif ($op eq '+=') { push(@{$CONFIG{$key}}, @a); + } elsif ($op eq '-=') { + my %a = map { $_ => $_ } @a; + @{$CONFIG{$key}} = + grep { !exists($a{$_}) } @{$CONFIG{$key}}; } else { die("can't happen\n"); } @@ -226,7 +230,7 @@ sub readconf($) { $val =~ s/^\'([^\']*)\'$/$1/; if ($op eq '=') { $CONFIG{$key} = $val; - } elsif ($op eq '+=') { + } elsif ($op eq '+=' || $op eq '-=') { die("$fn: $key is not an array on line $n\n"); } else { die("can't happen\n"); @@ -404,12 +408,12 @@ sub tinderbox($$$) { my $error = 0; my $summary = ""; my $root = realpath(expand('SANDBOX') . "/$branch/$arch/$machine"); - my $srcdir = realpath(expand('SRCDIR')) || "$root/src"; - my $objdir = realpath(expand('OBJDIR')) || "$root/obj"; + my $srcdir = realpath(expand('SRCDIR') || "$root/src"); + my $objdir = realpath(expand('OBJDIR') || "$root/obj"); while (<$rpipe>) { if ($abbreviate) { - s/\Q$srcdir\E/\/src/g; - s/\Q$objdir\E/\/obj/g; + s/\Q$srcdir\E/\/src/go; + s/\Q$objdir\E/\/obj/go; } print($full $_); if (/^TB ---/ || /^>>> /) { Modified: user/des/tinderbox/tinderbox.1 ============================================================================== --- user/des/tinderbox/tinderbox.1 Mon Mar 24 20:30:39 2014 (r263694) +++ user/des/tinderbox/tinderbox.1 Mon Mar 24 22:34:22 2014 (r263695) @@ -1,5 +1,5 @@ .\"- -.\" Copyright (c) 2003-2012 Dag-Erling Smørgrav +.\" Copyright (c) 2003-2014 Dag-Erling Smørgrav .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd October 2, 2013 +.Dd March 24, 2014 .Dt TINDERBOX 1 .Os .Sh NAME Modified: user/des/tinderbox/tinderbox.pl ============================================================================== --- user/des/tinderbox/tinderbox.pl Mon Mar 24 20:30:39 2014 (r263694) +++ user/des/tinderbox/tinderbox.pl Mon Mar 24 22:34:22 2014 (r263695) @@ -1,6 +1,6 @@ #!/usr/bin/perl -Tw #- -# Copyright (c) 2003-2013 Dag-Erling Smørgrav +# Copyright (c) 2003-2014 Dag-Erling Smørgrav # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -34,8 +34,8 @@ use POSIX; use Getopt::Long; use Scalar::Util qw(tainted); -my $VERSION = "2.20"; -my $COPYRIGHT = "Copyright (c) 2003-2013 Dag-Erling Smørgrav. " . +my $VERSION = "2.21"; +my $COPYRIGHT = "Copyright (c) 2003-2014 Dag-Erling Smørgrav. " . "All rights reserved."; my $arch; # Target architecture @@ -46,6 +46,8 @@ my $hostname; # Name of the host runni my $logfile; # Path to log file my $machine; # Target machine my $objdir; # Location of object tree +my $objpath; # Full path to object tree +my $obj32path; # Full path to 32-bit object tree my $patch; # Patch to apply before building my $sandbox; # Location of sandbox my $srcdir; # Location of source tree @@ -171,6 +173,34 @@ sub logenv() { } # +# Expand a path +# +sub realpath($;$); +sub realpath($;$) { + my $path = shift; + my $base = shift || ""; + + my $realpath = ($path =~ m|^/|) ? "" : $base; + foreach my $part (split('/', $path)) { + if ($part eq '' || $part eq '.') { + # nothing + } elsif ($part eq '..') { + $realpath =~ s|/[^/]+$|| + or die("'$path' is not a valid path relative to '$base'\n"); + } elsif (-l "$realpath/$part") { + my $target = readlink("$realpath/$part") + or die("unable to resolve symlink '$realpath/$part': $!\n"); + $realpath = realpath($target, $realpath); + } else { + $part =~ m/^([\w.-]+)$/ + or die("unsafe path '$realpath/$part'\n"); + $realpath .= "/$1"; + } + } + return $realpath; +} + +# # Open and lock a file reliably # sub open_locked($;$$) { @@ -448,12 +478,14 @@ MAIN:{ "verbose+" => \$verbose, ) or usage(); - if ($jobs < 0) { + if ($jobs !~ m/^(\d+)$/) { error("invalid number of jobs"); } - if ($timeout < 0) { + $jobs = $1; + if ($timeout !~ m/^(\d+)$/) { error("invalid timeout"); } + $timeout = $1; if ($branch !~ m|^(\w+)$|) { error("invalid source branch"); } @@ -553,6 +585,7 @@ MAIN:{ truncate($lockfile, 0); print($lockfile "$$\n"); + # Validate source directory if (defined($srcdir)) { if ($srcdir !~ m|^(/[\w./-]+)$|) { error("invalid source directory"); @@ -561,7 +594,9 @@ MAIN:{ } else { $srcdir = "$sandbox/src"; } + $srcdir = realpath($srcdir); + # Validate object directory if (defined($objdir)) { if ($objdir !~ m|^(/[\w./-]+)$|) { error("invalid object directory"); @@ -570,6 +605,11 @@ MAIN:{ } else { $objdir = "$sandbox/obj"; } + $objdir = realpath($objdir); + + # Construct full path to object directory + $objpath = "$objdir/$machine.$arch$srcdir"; + $obj32path = "$objdir/$machine.$arch/lib32$srcdir"; # Clean up remains from old runs do_clean(); # no prefix for backward compatibility @@ -754,7 +794,10 @@ MAIN:{ # Check that the config is appropriate for this target. cd("$srcdir/sys/$machine/conf"); local *PIPE; - my @cmdline = ("/usr/sbin/config", "-m", "$kernel"); + # ugh, we really shouldn't need to know that. + my $cmd = "$objpath/tmp/legacy/usr/sbin/config"; + $cmd = "/usr/sbin/config" unless -x $cmd; + my @cmdline = ($cmd, "-m", $kernel); message(@cmdline); if (open(PIPE, "-|", @cmdline)) { local $/;