Date: Tue, 4 Mar 2003 17:20:11 -0600 (CST) From: Mark Linimon <linimon@lonesome.com> To: FreeBSD-gnats-submit@FreeBSD.org Subject: ports/48935: [patch] eliminate false indications in bento errorlog reports Message-ID: <200303042320.h24NKBTH053263@lonesome.lonesome.com>
next in thread | raw e-mail | index | archive | help
>Number: 48935
>Category: ports
>Synopsis: [patch] eliminate false indications in bento errorlog reports
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: freebsd-ports-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Mar 04 15:20:20 PST 2003
>Closed-Date:
>Last-Modified:
>Originator: Mark Linimon
>Release: FreeBSD-4.7
>Organization:
FreeBSD
>Environment:
System: FreeBSD lonesome.lonesome.com 4.7-STABLE FreeBSD 4.7-STABLE #0: Fri Nov 8 23:46:29 CST 2002 root@lonesome.lonesome.com:/usr/src/sys/compile/MULTIMEDIA i386
>Description:
The /usr/ports/Tools/portbuild/scripts/processlogs script becomes
confused when handling very short port build error logs. These
generally are created when there is some kind of error on the bento
cluster itself. The symptom you'll see is something bogus like
"Exec" or "fetch" in the Maintainer field.
More detail: the script uses an intermediate file to cache results,
one line per port. The current handling of this file assumes that all
fields are non-empty. However, for sufficiently truncated errorlogs,
at least some fields are indeed empty.
>How-To-Repeat:
View the current http://bento.freebsd.org/errorlogs/ia64-5-full/ report.
>Fix:
The following catches these cases and classifies them as error
type "cluster" with an empty Maintainer and Date field. (I am
not married to the "cluster" name but some kind of metatoken
needs to be used).
Also, while I had the patient on the table, I added a couple
of more cases of "config error" and "perl" that seem to have
shown up in recent errorlog runs.
--- processlogs.dist Fri Feb 14 03:28:42 2003
+++ processlogs Tue Mar 4 15:53:44 2003
@@ -29,19 +29,19 @@
portname=$(basename $filename .log)
affected=$(($(grep -cF $portname < INDEX) -1))
logsize=$(/bin/ls -sk $filename | awk '{print $1}')
- dir=$(sed -n -e '3p' $filename | awk '{print $4}' | \
+ dir=$(sed -n -e '3p' $filename | grep '^with arguments' | awk '{print $4}' | \
sed -e 's,^/[^/]*/[^/]*/,,')
- maintainer=$(sed -n -e '4p' $filename | awk '{print $3}')
- datetime=$(grep 'build started at' $filename | \
+ maintainer=$(sed -n -e '4p' $filename | grep '^maintained by' | awk '{print $3}')
+ datetime=$(sed -n -e '5p' $filename | grep '^build started at' | \
sed -e 's/build started at ...//' | tr ' ' '_' )
- # throw out totally bogus files (e.g., blank file, error 404, etc.)
- if [ -z "$dir" -a -z "$maintainer" -a -z "$datetime" ]; then
- shift
- continue
- fi
-
- if grep -q 'list of extra files and directories' $1; then
+ # now try to classify the type of error found in the file.
+ # the first case handles failures to even try to build any
+ # port (i.e. HTML file no longer there, bento being unable
+ # to fetch any file, bento being able to build any port, etc.)
+ if [ -z "$dir" -o -z "$datetime" ]; then
+ reason="cluster"; tag="cluster"
+ elif grep -q 'list of extra files and directories' $1; then
reason="mtree"; tag="mtree"
elif grep -q "See <URL:http://www.gnu.org/software/gcc/bugs.html>; for instructions." $1; then
reason="gcc_bug"; tag="gcc-bug"
@@ -227,6 +227,8 @@
reason="chown"; tag="chown"
elif grep -q "Cannot stat: " $1; then
reason="configure_error"; tag="configure"
+ elif grep -q "make: cannot open Makefile" $1; then
+ reason="configure_error"; tag="configure"
elif grep -qE "Script.*configure.*failed unexpectedly" $1; then
reason="configure_error"; tag="configure"
elif grep -q "Cannot open /dev/tty for read" $1; then
@@ -267,7 +269,9 @@
reason="mtree"; tag="mtree"
elif grep -qE "cp:.*site_perl: No such file or directory" $1; then
reason="perl"; tag="perl"
- elif grep -q "Perl .* required--this is only version" $1; then
+ elif grep -qE "perl. is needed, and you do not seem to have it" $1; then
+ reason="perl"; tag="perl"
+ elif grep -qE "Perl .* required--this is only version" $1; then
reason="perl"; tag="perl"
elif grep -q "pod2man: not found" $1; then
reason="pod2man"; tag="pod2man"
@@ -299,11 +303,26 @@
reason="threads"; tag="threads"
elif grep -q "/usr/bin/ld: cannot find -lXThrStub" $1; then
reason="threads"; tag="threads"
+ elif grep -q "port wants the OpenSSL library from the FreeBSD base system" $1; then
+ reason="use_openssl_base"; tag="use_openssl_base"
else
reason="???"; tag="unknown"
fi
+ # clean up some error cases -- the way .logs works, it expects that
+ # every field in it MUST be nonblank, so we insert a metatoken here.
+ # See below.
+ if [ -z "$dir" ]; then
+ dir="NONE"
+ fi
+ if [ -z "$maintainer" ]; then
+ maintainer="NONE"
+ fi
+ if [ -z "$datetime" ]; then
+ datetime="NONE"
+ fi
+
broken="no"
if grep -q "Trying build of .* even though it is marked BROKEN" $1; then
broken="broken"
@@ -352,6 +371,11 @@
echo "</html>" >>$of
}
+# Now reread the .logs file and create the reports. If the .logs file
+# has blanks in any field, this code won't work. Therefore, the
+# code above has to guarantee that that won't happen, and the code
+# below has to catch the metatoken.
+
#
# Create "default" output, sorted on portname
#
@@ -359,14 +383,22 @@
for i in `cat .logs | sort`; do
set $(echo $i | tr \| " ")
+ if [ "$6" = "NONE" ] ; then
+ mailto=" "
+ else
+ mailto="<a href=\"mailto:$6\">$6</a>"
+ fi
+ set $(echo $i | tr \| " " | sed -e "s@NONE@\\ @g")
+
echo "<tr>" >> $of
+
echo "<td><a href=\"$1\">$2</a></td>" >> $of
affby=$3
test $affby = "0" -o $affby = "-1" && affby=" "
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
- echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
+ echo "<td>$mailto</td>" >> $of
echo "<td>" >> $of
test "$9" = "broken" && echo "<font color=\"red\">[B]</font>" >> $of
@@ -389,14 +421,22 @@
for i in `cat .logs | sort -t \\| +4`; do
set $(echo $i | tr \| " ")
+ if [ "$6" = "NONE" ] ; then
+ mailto=" "
+ else
+ mailto="<a href=\"mailto:$6\">$6</a>"
+ fi
+ set $(echo $i | tr \| " " | sed -e "s@NONE@\\ @g")
+
echo "<tr>" >> $of
+
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
affby=$3
test $affby = "0" -o $affby = "-1" && affby=" "
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"$1\">$2</a></td>" >> $of
- echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
+ echo "<td>$mailto</td>" >> $of
echo "<td>" >> $of
test "$9" = "broken" && echo "<font color=\"red\">[B]</font>" >> $of
@@ -419,8 +459,16 @@
for i in `cat .logs | sort -t \\| +5`; do
set $(echo $i | tr \| " ")
+ if [ "$6" = "NONE" ] ; then
+ mailto=" "
+ else
+ mailto="<a href=\"mailto:$6\">$6</a>"
+ fi
+ set $(echo $i | tr \| " " | sed -e "s@NONE@\\ @g")
+
echo "<tr>" >> $of
- echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
+
+ echo "<td>$mailto</td>" >> $of
echo "<td><a href=\"$1\">$2</a></td>" >> $of
affby=$3
@@ -449,6 +497,13 @@
for i in `cat .logs | sort -t \\| +7`; do
set $(echo $i | tr \| " ")
+ if [ "$6" = "NONE" ] ; then
+ mailto=" "
+ else
+ mailto="<a href=\"mailto:$6\">$6</a>"
+ fi
+ set $(echo $i | tr \| " " | sed -e "s@NONE@\\ @g")
+
echo "<tr>" >> $of
echo "<td>" >> $of
@@ -463,7 +518,7 @@
test $affby = "0" -o $affby = "-1" && affby=" "
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
- echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
+ echo "<td>$mailto</td>" >> $of
date=$(echo ${10} | tr '_' ' ')
echo "<td>$date</td>" >> $of
@@ -480,6 +535,13 @@
for i in `cat .logs | sort -t \\| +9`; do
set $(echo $i | tr \| " ")
+ if [ "$6" = "NONE" ] ; then
+ mailto=" "
+ else
+ mailto="<a href=\"mailto:$6\">$6</a>"
+ fi
+ set $(echo $i | tr \| " " | sed -e "s@NONE@\\ @g")
+
echo "<tr>" >> $of
date=$(echo ${10} | tr '_' ' ')
@@ -491,7 +553,7 @@
test $affby = "0" -o $affby = "-1" && affby=" "
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
- echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
+ echo "<td>$mailto</td>" >> $of
echo "<td>" >> $of
test "$9" = "broken" && echo "<font color=\"red\">[B]</font>" >> $of
@@ -509,6 +571,13 @@
#
for i in `cat .logs | sort -t \\| +9`; do
set $(echo $i | tr \| " ")
+ if [ "$6" = "NONE" ] ; then
+ mailto=" "
+ else
+ mailto="<a href=\"mailto:$6\">$6</a>"
+ fi
+ set $(echo $i | tr \| " " | sed -e "s@NONE@\\ @g")
+
maints="$maints $6"
done
>Release-Note:
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-ports-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200303042320.h24NKBTH053263>