Date: Fri, 11 Oct 2002 22:45:07 +1000 (EST) From: Edwin Groothuis <edwin@mavetju.org> To: FreeBSD-gnats-submit@FreeBSD.org Subject: ports/43927: [patch] sort bentos logs on subjects for easier tracking Message-ID: <20021011124507.9568F6A712B@k7.mavetju>
next in thread | raw e-mail | index | archive | help
>Number: 43927
>Category: ports
>Synopsis: [patch] sort bentos logs on subjects for easier tracking
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: freebsd-ports
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Fri Oct 11 05:50:00 PDT 2002
>Closed-Date:
>Last-Modified:
>Originator: Edwin Groothuis
>Release: FreeBSD 4.5-RELEASE i386
>Organization:
-
>Environment:
System: FreeBSD k7.mavetju 4.5-RELEASE FreeBSD 4.5-RELEASE #4: Sat Sep 14 10:41:50 EST 2002 edwin@k7.mavetju:/usr/src/sys/compile/k7 i386
>Description:
Right now, the overview of the error-logs on Bento is sorted on
filename. Sometimes it would be nice to have an overview on maintainer,
or on category, or on error etc. I don't know about other people,
but I would like to see them in that form sometimes :-)
>How-To-Repeat:
http://bento.freebsd.org/errorlogs/index.html
|
+--- http://bento.freebsd.org/errorlogs/4-full/
+--- http://bento.freebsd.org/errorlogs/5-full/
etc
>Fix:
Attached is a extended version of ports/Tools/portbuild/scripts/processlogs.
Besides a nicer seperation of input- and output-procedures it
generates five reports: one by portname, one by maintaner, one by
categorie, one by error and one by builddate.
To choose one of them from the bento-index-page
(http://bento.freebsd.org/errorlogs/index.html) that one should
also a be little bit modified into: ([] is a href)
Error logs ([by maintainer])
* Last full run on 4-stable, by [portname] [categorie] [maintainer] [errors] [builddate]
* Latest run on 4-stable, by [portname] [categorie] [maintainer] [errors] [builddate]
* Last full run on 5-current, by [portname] [categorie] [maintainer] [errors] [builddate]
* Latest run on 5-current, by [portname] [categorie] [maintainer] [errors] [builddate]
* Last full run on 4-stable with experimental port patches, by [portname] [categorie] [maintainer] [errors] [builddate]
* Latest run on 4-stable with experimental port patches, by [portname] [categorie] [maintainer] [errors] [builddate]
The result would be an easier to track build-log-overview.
----- 8< ---- ports/Tools/portbuild/scripts/processlogs ----- 8< ----
#!/bin/sh
of=.index.html
# delete .new file if it is more than a day old
find $of -mtime 2 -delete 2>/dev/null
if [ -f $of ]; then exit; fi
if [ -e .stamp -a $(echo $(find . -maxdepth 1 -newer .stamp -type f -name '*.log' 2>/dev/null | wc -l)) = "0" ]; then exit; fi
touch .stamp
set *.log
> .logs
#
# Read the log-files and write it to .logs in the format
# $filename|$portname|$affected|$logsize|$dir|$maintainer|\
# $reason|$tag|$broken|$datetime
#
#
if [ $# != 1 -o "x$1" != "x*.log" ]; then
while [ ! -z "$1" ]; do
filename=$1
portname=$(basename $filename .log)
affected=$(($(grep -cF $portname < INDEX) -1))
logsize=$(/bin/ls -sk $filename | awk '{print $1}')
dir=$(sed -n -e '3p' $filename | awk '{print $4}' | \
sed -e 's,^/[^/]*/[^/]*/,,')
maintainer=$(sed -n -e '4p' $filename | awk '{print $3}')
datetime=$(grep 'build started at' $filename | \
sed -e 's/build started at ...//' | tr ' ' '_' )
if grep -q 'list of extra files and directories' $1; then
reason="mtree"; tag="mtree"
elif grep -q "See <URL:http://www.gnu.org/software/gcc/bugs.html>; for instructions." $1; then
reason="gcc_bug"; tag="gcc bug"
elif grep -q 'Checksum mismatch' $1; then
reason="checksum"; tag="checksum"
elif grep -qE '(No checksum recorded for|(Maybe|Either) .* is out of date, or)' $1; then
reason="distinfo_update"; tag="distinfo"
elif grep -qE '(configure: error:|script.*failed: here are the contents of)' $1; then
reason="configure_error"; tag="configure"
elif grep -qE '(bison:.*(No such file|not found)|multiple definition of `yy)' $1; then
reason="bison"; tag="bison"
elif grep -q "Couldn't fetch it - please try" $1; then
reason="fetch"; tag="fetch"
elif grep -q "out of .* hunks .*--saving rejects to" $1; then
reason="patch"; tag="patch"
elif grep -q 'Error: category .* not in list of valid categories' $1; then
reason="CATEGORIES"; tag="categories"
elif grep -q 'make: don.t know how to make .*\.man. Stop' $1; then
reason="X_manpage"; tag="xfree4man"
elif grep -q 'Xm/Xm\.h: No such file' $1; then
reason="MOTIF"; tag="motif"
elif grep -q 'undefined reference to `Xp' $1; then
reason="MOTIFLIB"; tag="motiflib"
elif grep -qi 'read-only file system' $1; then
reason="WRKDIR"; tag="wrkdir"
elif grep -q 'makeinfo: .* use --force' $1; then
reason="texinfo"; tag="texinfo"
elif grep -q 'means that you did not run the h2ph script' $1; then
reason="perl5"; tag="perl5"
elif grep -q "Error: shared library \".*\" does not exist" $1; then
reason="LIB_DEPENDS"; tag="libdepends"
elif grep -qE "(crt0|c\+\+rt0)\.o: No such file" $1; then
reason="ELF"; tag="elf"
elif grep -q "machine/soundcard.h: No such file or directory" $1; then
reason="soundcard.h"; tag="soundcard.h"
elif grep -q "values.h: No such file or directory" $1; then
reason="values.h"; tag="values.h"
elif grep -q ".*\.h: No such file" $1; then
if grep -qE "(X11/.*|Xosdefs)\.h: No such file" $1; then
if grep -q "XFree86-.*\.tgz" $1; then
reason="missing_header"; tag="header"
else
reason="USE_XLIB"; tag="usexlib"
fi
else
reason="missing_header"; tag="header"
fi
elif grep -q "pnohang: killing make checksum" $1; then
reason="fetch_timeout"; tag="fetch-timeout"
elif grep -q "pnohang: killing make package" $1; then
reason="runaway_process"; tag="runaway"
elif grep -q "cd: can't cd to" $1; then
reason="NFS"; tag="nfs"
elif grep -qE "pkg_add: (can't find enough temporary space|projected size of .* exceeds available free space)" $1; then
reason="disk_full"; tag="df"
elif grep -qE '(parse error|too (many|few) arguments to|argument.*doesn.*prototype|incompatible type for argument|conflicting types for|undeclared \(first use (in |)this function\)|incorrect number of parameters|has incomplete type and cannot be initialized)' $1; then
reason="compiler_error"; tag="cc"
elif grep -qE '(ANSI C.. forbids|is a contravariance violation|changed for new ANSI .for. scoping|[0-9]: passing .* changes signedness|discards qualifiers|lacks a cast|redeclared as different kind of symbol|invalid type .* for default argument to|wrong type argument to unary exclamation mark|duplicate explicit instantiation of|incompatible types in assignment|assuming . on overloaded member function|call of overloaded .* is ambiguous|declaration of C function .* conflicts with|initialization of non-const reference type|using typedef-name .* after|[0-9]: implicit declaration of function|[0-9]: size of array .* is too large|fixed or forbidden register .* for class)' $1; then
reason="new_compiler_error"; tag="newgcc"
elif grep -qE '(syntax error before|ISO C++ forbids|friend declaration|no matching function for call to|.main. must return .int.|invalid conversion from|cannot be used as a macro name as it is an operator in C\+\+|is not a member of type|after previous specification in|no class template named|because worst conversion for the former|better than worst conversion|no match for.*operator|no match for call to|undeclared in namespace|is used as a type, but is not)' $1; then
reason="bad_C++_code"; tag="badc++"
elif grep -qE '(/usr/libexec/elf/ld: cannot find|undefined reference to|cannot open -l.*: No such file)' $1; then
reason="linker_error"; tag="ld"
elif grep -qE 'chown:.*invalid argument' $1; then
reason="chown"; tag="chown"
elif grep -q 'install: .*: No such file' $1; then
reason="install_error"; tag="install"
elif grep -q "/usr/.*/man/.*: No such file or directory" $1; then
reason="manpage"; tag="manpage"
elif grep -q "pkg_create: make_dist: tar command failed with code" $1; then
reason="PLIST"; tag="plist"
elif grep -q "Can't open display" $1; then
reason="DISPLAY"; tag="display"
elif grep -q " is already installed - perhaps an older version" $1; then
reason="depend_object"; tag="dependobj"
elif grep -q "error in dependency .*, exiting" $1; then
reason="depend_package"; tag="dependpkg"
elif grep -q "#error \"<malloc.h> has been replaced by <stdlib.h>\"" $1; then
reason="malloc.h"; tag="malloc.h"
elif grep -q "core dumped" $1; then
reason="coredump"; tag="coredump"
elif grep -q "Segmentation fault" $1; then
reason="segfault"; tag="segfault"
elif grep -q "structure has no member named" $1; then
reason="struct_changes"; tag="struct"
else
reason="???"; tag="unknown"
fi
broken="no"
if grep -q "Trying build of .* even though it is marked BROKEN" $1; then
broken="broken"
fi
shift
echo "$filename|$portname|$affected|$logsize|$dir|$maintainer|$reason|$tag|$broken|$datetime" >> .logs
done
fi
num=$(wc -l < .logs)
header() {
echo "<html><head><title>Package building errors</title>" >$of
echo "</head><body><h1>Package building errors</h1>" >>$of
echo "<p>View by " >>$of
echo "[ <a href=\"index-port.html\">port</a> " >>$of
echo "| <a href=\"index-maintainer.html\">maintainer</a> " >>$of
echo "| <a href=\"index-categorie.html\">categorie</a> " >>$of
echo "| <a href=\"index-reason.html\">error</a> " >>$of
echo "| <a href=\"index-builddate.html\">builddate</a> " >>$of
echo "]</p>" >>$of
if [ $num = "0" ]; then
echo "No errors (yet)" >>$of
else
if [ -s cvsdone ]; then
echo "CVS update finished at: $(cat cvsdone)<br>" >> $of
fi
echo "Timestamp of newest log: $(ls -rtTl *.log | tail -1 | awk '{printf("%s %s %s %s\n",$6,$7,$8,$9)}')<br><br>" >> $of
echo "\"Aff.\" is number of ports that depend on this one<br>" >> $of
echo "\"<font color=\"red\">[B]</font>\" indicates port is marked BROKEN<br><br>" >> $of
echo "<p>$num errors</p>" >> $of
echo "<table border=1>" >>$of
echo "<tr>$1</tr>" >>$of
fi
}
#
# Create "default" output, sorted on portname
#
header "<th>Port</th><th>Aff.</th><th>Size</th><th>CVS</th><th>Maintainer</th><th>Reason</th><th>Date build</th>"
for i in `cat .logs | sort`; do
set $(echo $i | tr \| " ")
echo "<tr>" >> $of
echo "<td><a href=\"$1\">$2</a></td>" >> $of
affby=$3
test $affby = "0" && affby=""
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
echo "<td>" >> $of
test "$9" = "broken" && echo "<font color=\"red\">[B]</font>" >> $of
reason=$(echo $7 | tr '_' ' ')
echo "<a href=\"http://bento.freebsd.org/#$8\">$reason</a>" >> $of
echo "</td>" >> $of
date=$(echo ${10} | tr '_' ' ')
echo "<td>$date</td>" >> $of
echo "</tr>" >> $of
done
mv -f $of index-port.html
#
# Create output by categorie
#
header "<th>CVS</th><th>Aff.</th><th>Size</th><th>Port</th><th>Maintainer</th><th>Reason</th><th>Date build</th>"
for i in `cat .logs | sort -t \\| +4`; do
set $(echo $i | tr \| " ")
echo "<tr>" >> $of
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
affby=$3
test $affby = "0" && affby=""
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"$1\">$2</a></td>" >> $of
echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
echo "<td>" >> $of
test "$9" = "broken" && echo "<font color=\"red\">[B]</font>" >> $of
reason=$(echo $7 | tr '_' ' ')
echo "<a href=\"http://bento.freebsd.org/#$8\">$reason</a>" >> $of
echo "</td>" >> $of
date=$(echo ${10} | tr '_' ' ')
echo "<td>$date</td>" >> $of
echo "</tr>" >> $of
done
mv -f $of index-categorie.html
#
# Create output by maintainer
#
header "<th>Maintainer</th><th>Port</th><th>Aff.</th><th>Size</th><th>CVS</th><th>Reason</th><th>Date build</th>"
for i in `cat .logs | sort -t \\| +5`; do
set $(echo $i | tr \| " ")
echo "<tr>" >> $of
echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
echo "<td><a href=\"$1\">$2</a></td>" >> $of
affby=$3
test $affby = "0" && affby=""
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
echo "<td>" >> $of
test "$9" = "broken" && echo "<font color=\"red\">[B]</font>" >> $of
reason=$(echo $7 | tr '_' ' ')
echo "<a href=\"http://bento.freebsd.org/#$8\">$reason</a>" >> $of
echo "</td>" >> $of
date=$(echo ${10} | tr '_' ' ')
echo "<td>$date</td>" >> $of
echo "</tr>" >> $of
done
mv -f $of index-maintainer.html
#
# Create output by error
#
header "<th>Reason</th><th>Port</th><th>Aff.</th><th>Size</th><th>CVS</th><th>Maintainer</th><th>Date build</th>"
for i in `cat .logs | sort -t \\| +7`; do
set $(echo $i | tr \| " ")
echo "<tr>" >> $of
echo "<td>" >> $of
test "$9" = "broken" && echo "<font color=\"red\">[B]</font>" >> $of
reason=$(echo $7 | tr '_' ' ')
echo "<a href=\"http://bento.freebsd.org/#$8\">$reason</a>" >> $of
echo "</td>" >> $of
echo "<td><a href=\"$1\">$2</a></td>" >> $of
affby=$3
test $affby = "0" && affby=""
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
date=$(echo ${10} | tr '_' ' ')
echo "<td>$date</td>" >> $of
echo "</tr>" >> $of
done
mv -f $of index-reason.html
#
# Create output by builddate
#
header "<th>Date build</th><th>Port</th><th>Aff.</th><th>Size</th><th>CVS</th><th>Maintainer</th><th>Reason</th>"
for i in `cat .logs | sort -t \\| +9`; do
set $(echo $i | tr \| " ")
echo "<tr>" >> $of
date=$(echo ${10} | tr '_' ' ')
echo "<td>$date</td>" >> $of
echo "<td><a href=\"$1\">$2</a></td>" >> $of
affby=$3
test $affby = "0" && affby=""
echo "<td align=\"right\">$affby</td><td align=\"right\">$4 Kb</td>" >> $of
echo "<td><a href=\"http://www.FreeBSD.org/cgi/cvsweb.cgi/ports/$5\">$5</a></td>" >> $of
echo "<td><a href=\"mailto:$6\">$6</a></td>" >> $of
echo "<td>" >> $of
test "$9" = "broken" && echo "<font color=\"red\">[B]</font>" >> $of
reason=$(echo $7 | tr '_' ' ')
echo "<a href=\"http://bento.freebsd.org/#$8\">$reason</a>" >> $of
echo "</td>" >> $of
echo "</tr>" >> $of
done
mv -f $of index-builddate.html
#
# Get list of maintainers.
#
for i in `cat .logs | sort -t \\| +9`; do
set $(echo $i | tr \| " ")
maints="$maints $6"
done
echo $maints | sed -e 's/ /\
/g' | sort -fu > maintainers
----- 8< ---- ports/Tools/portbuild/scripts/processlogs ----- 8< ----
>Release-Note:
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20021011124507.9568F6A712B>