Date: Fri, 27 Nov 2009 17:00:47 +0300 (MSK) From: Eygene Ryabinkin <rea-fbsd@codelabs.ru> To: FreeBSD-gnats-submit@FreeBSD.org Cc: nectar@FreeBSD.org, miwi@FreeBSD.org Subject: ports/140939: [patch] security/vuxml: fix and extend files/newentry.sh Message-ID: <20091127140047.6A690DA81A@void.codelabs.ru> Resent-Message-ID: <200911271410.nAREA2Hf035717@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 140939
>Category: ports
>Synopsis: [patch] security/vuxml: fix and extend files/newentry.sh
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: freebsd-ports-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Fri Nov 27 14:10:01 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator: Eygene Ryabinkin
>Release: FreeBSD 8.0-RC1 amd64
>Organization:
Code Labs
>Environment:
System: FreeBSD 8.0-RC1 amd64
>Description:
There are two problems in the current implementation of newentry.sh:
1. cleanup handler is installed after the temporary file is created,
so there is a small window during which the script can receive the
signal and temporary file won't be deleted;
2. cleanup handler isn't called when external editor is executed, so
temporary file is left in its place.
Moreover, since patches to the vuln.xml tend to expire quickly: once the
new entry is added to the vuln.xml, the patches that are created for the
previous versions won't apply anymore. Since vuln.xml changes rather
often, it is desirable to find a way to apply these patches
automatically and newentry.sh can be extended to handle it.
miwi@ sometimes asks me to redo my diffs for the VuXML entries since
they went out-of-date, so this functionality should help in such cases.
>How-To-Repeat:
Point 1: look at the code:
-----
tmp="`mktemp ${TMPDIR:-/tmp}/vuxml.XXXXXXXXXX`" || exit 1
doclean="yes"
cleanup() {
if [ "${doclean}" = "yes" ]; then
rm -f "${tmp}"
fi
}
trap cleanup EXIT 1 2 13 15
-----
Compare with
-----
doclean="yes"
cleanup() {
if [ "${doclean}" = "yes" ]; then
[ -n "${tmp}" ] && rm -f "${tmp}"
fi
}
trap cleanup EXIT 1 2 13 15
tmp="`mktemp ${TMPDIR:-/tmp}/vuxml.XXXXXXXXXX`" || exit 1
-----
Point 2: try to invoke 'make newentry' and watch for file(s)
vuxml.<anything> in the /tmp or $TMPDIR.
>Fix:
The following patch fixes the point 1:
--- 0001-install-cleanup-handler-earlier.diff begins here ---
>From 4375203e1602acacfc0ef230fd2cabd42d142463 Mon Sep 17 00:00:00 2001
From: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
Date: Fri, 27 Nov 2009 12:35:17 +0300
There is a gap between 'mktemp' and installation of the trap handler.
We can initialize 'doclean' before trapping signals and additionally
check if ${tmp} is empty inside the cleanup handler. This will
practically eliminate the gap where temporary file won't be removed
if the script will receive some signal.
Signed-off-by: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
---
security/vuxml/files/newentry.sh | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/security/vuxml/files/newentry.sh b/security/vuxml/files/newentry.sh
index b6337da..d08bbc8 100644
--- a/security/vuxml/files/newentry.sh
+++ b/security/vuxml/files/newentry.sh
@@ -6,14 +6,14 @@ if [ -z "${vuxml_file}" ]; then
exit 1
fi
-tmp="`mktemp ${TMPDIR:-/tmp}/vuxml.XXXXXXXXXX`" || exit 1
-doclean="yes"
cleanup() {
if [ "${doclean}" = "yes" ]; then
- rm -f "${tmp}"
+ [ -n "${tmp}" ] && rm -f "${tmp}"
fi
}
+doclean="yes"
trap cleanup EXIT 1 2 13 15
+tmp="`mktemp ${TMPDIR:-/tmp}/vuxml.XXXXXXXXXX`" || exit 1
vid="`uuidgen | tr '[:upper:]' '[:lower:]'`"
[ -z "$vid" ] && exit 1
--
1.6.5.3
--- 0001-install-cleanup-handler-earlier.diff ends here ---
The following patch fixes the point 2:
--- 0002-cleanup-before-spawning-editor.diff begins here ---
>From b5ebab7ccd3ac8d484bbe6bc037b94601dc0c7be Mon Sep 17 00:00:00 2001
From: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
Date: Fri, 27 Nov 2009 12:52:07 +0300
Subject: [PATCH 2/4] security/vuxml: newentry.sh: cleanup before executing editor
When we do 'exec EDITOR', cleanup handler isn't called, so the temporary
file is left in place (/tmp or $TMPDIR). We fix it by explicitely
calling cleanup() before exec'ing editor.
Signed-off-by: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
---
security/vuxml/files/newentry.sh | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/security/vuxml/files/newentry.sh b/security/vuxml/files/newentry.sh
index d08bbc8..f84d0ea 100644
--- a/security/vuxml/files/newentry.sh
+++ b/security/vuxml/files/newentry.sh
@@ -51,6 +51,7 @@ awk '/^[[:space:]]+<vuln /,/^NONE$/ { print }' \
"${vuxml_file}" >> "${tmp}" || exit 1
if cp "${tmp}" "${vuxml_file}"; then
+ cleanup
exec ${EDITOR:-vi} "${vuxml_file}"
else
doclean="no"
--
1.6.5.3
--- 0002-cleanup-before-spawning-editor.diff ends here ---
Two next patches implement the functionality to add existing VuXML
entries or single-entry patches for vuln.xml automatically.
First patch refactors newentry.sh to place currently spreaded code for
creation of a new entry to a single function.
Second patch implements new functionality.
--- 0003-refactor-newentry.sh.diff begins here ---
>From 277a3c98c2449fc9c965ba3d8a46a6e78d4db5f0 Mon Sep 17 00:00:00 2001
From: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
Date: Fri, 27 Nov 2009 13:11:58 +0300
Subject: [PATCH 3/4] security/vuxml: refactor newentry.sh
Make new procedure, newentry(), that creates new VuXML entry,
thus incapsulating the needed functionality into the single
function. Main code just calls it when needed.
Also make usage() to be stand-alone function.
Signed-off-by: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
---
security/vuxml/files/newentry.sh | 54 ++++++++++++++++++++++++--------------
1 files changed, 34 insertions(+), 20 deletions(-)
diff --git a/security/vuxml/files/newentry.sh b/security/vuxml/files/newentry.sh
index f84d0ea..f2b9e61 100644
--- a/security/vuxml/files/newentry.sh
+++ b/security/vuxml/files/newentry.sh
@@ -1,27 +1,17 @@
#! /bin/sh
-vuxml_file="$1"
-if [ -z "${vuxml_file}" ]; then
- exec >&2
- echo "Usage: newentry.sh /path/to/vuxml/document"
- exit 1
-fi
-cleanup() {
- if [ "${doclean}" = "yes" ]; then
- [ -n "${tmp}" ] && rm -f "${tmp}"
- fi
-}
-doclean="yes"
-trap cleanup EXIT 1 2 13 15
-tmp="`mktemp ${TMPDIR:-/tmp}/vuxml.XXXXXXXXXX`" || exit 1
+new_entry () {
+ local vid discovery entry
-vid="`uuidgen | tr '[:upper:]' '[:lower:]'`"
-[ -z "$vid" ] && exit 1
-discovery="`date -u '+%Y-%m'`-FIXME" || exit 1
-entry="`date -u '+%Y-%m-%d'`" || exit 1
+ vid="`uuidgen | tr '[:upper:]' '[:lower:]'`"
+ discovery="`date -u '+%Y-%m'`-FIXME" || exit 1
+ entry="`date -u '+%Y-%m-%d'`" || exit 1
+ if [ -z "$vid" -o -z "$discovery" -o -z "$entry" ]; then
+ false
+ return
+ fi
-awk '/^<\?/,/^<vuxml/ { print }' "${vuxml_file}" >> "${tmp}" || exit 1
-cat << EOF >> "${tmp}" || exit 1
+ cat << EOF
<vuln vid="${vid}">
<topic> -- </topic>
<affects>
@@ -47,6 +37,30 @@ cat << EOF >> "${tmp}" || exit 1
</vuln>
EOF
+}
+
+usage () {
+ echo "Usage: newentry.sh /path/to/vuxml/document" >&2
+}
+
+cleanup() {
+ if [ "${doclean}" = "yes" ]; then
+ [ -n "${tmp}" ] && rm -f "${tmp}"
+ fi
+}
+
+vuxml_file="$1"
+if [ -z "${vuxml_file}" ]; then
+ usage
+ exit 1
+fi
+
+doclean="yes"
+trap cleanup EXIT 1 2 13 15
+tmp="`mktemp ${TMPDIR:-/tmp}/vuxml.XXXXXXXXXX`" || exit 1
+
+awk '/^<\?/,/^<vuxml/ { print }' "${vuxml_file}" >> "${tmp}" || exit 1
+new_entry >> "${tmp}" || exit 1
awk '/^[[:space:]]+<vuln /,/^NONE$/ { print }' \
"${vuxml_file}" >> "${tmp}" || exit 1
--
1.6.5.3
--- 0003-refactor-newentry.sh.diff ends here ---
--- 0004-teach-to-accept-pre-made-entries.diff begins here ---
>From 94dd35b3f635ef665426c4b339fe69982a8fb392 Mon Sep 17 00:00:00 2001
From: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
Date: Fri, 27 Nov 2009 15:06:09 +0300
Subject: [PATCH 4/4] security/vuxml: teach newentry.sh to accept existing entries
Now newentry.sh can add not only generic template for the new entry,
but existing entries as well. Both plain entries and patches are
accepted.
This mode was created mainly due to the fact that vuln.xml changes
frequently and if one sends patch for the obsolete version of vuln.xml,
the patch won't be applied cleanly and manual work is needed. Script
automates such insertions.
Moreover, plain VuXML entries that are not patches, but are just pieces
of vuln.xml can be sent via PRs -- they are inserted automatically too.
'make addentry E=/path/to/entry/file' will do the work -- Makefile was
extended to handle the new target.
Signed-off-by: Eygene Ryabinkin <rea-fbsd@codelabs.ru>
---
security/vuxml/Makefile | 6 ++
security/vuxml/files/newentry.sh | 115 +++++++++++++++++++++++++++++++++++++-
2 files changed, 119 insertions(+), 2 deletions(-)
diff --git a/security/vuxml/Makefile b/security/vuxml/Makefile
index d347c39..8aa177e 100644
--- a/security/vuxml/Makefile
+++ b/security/vuxml/Makefile
@@ -83,4 +83,10 @@ tidy:
newentry:
@${SH} ${FILESDIR}/newentry.sh "${VUXML_FILE}"
+addentry:
+.if empty(E)
+.error "Set make variable E to the location of the VuXML entry to be added."
+.endif
+ @${SH} ${FILESDIR}/newentry.sh -i "${E}" "${VUXML_FILE}"
+
.include <bsd.port.mk>
diff --git a/security/vuxml/files/newentry.sh b/security/vuxml/files/newentry.sh
index f2b9e61..d546a79 100644
--- a/security/vuxml/files/newentry.sh
+++ b/security/vuxml/files/newentry.sh
@@ -1,5 +1,87 @@
#! /bin/sh
+# Patch detection strategy: we are looking for the three successive
+# lines starting with "--- ", "+++ " and "@@ [+-]". The next line
+# should start with space. This catches unified diffs.
+#
+# The number of the line where the patch contents start (the line with
+# first '@@ [+-]') is printed to the standard output if the patch was
+# really found.
+is_patch () {
+ awk '
+BEGIN { seen_minus = 0; seen_plus = 0; seen_at = 0; diff = 0 }
+/^--- / { seen_minus = 1; seen_plus = 0; seen_at = 0; diff = 0; next; }
+seen_minus == 1 && /^\+\+\+ / { seen_plus = 1; next; }
+seen_plus == 1 && /^@@ [+-]/ { seen_at = 1; next; }
+seen_at == 1 && /^ / { print FNR; diff = 1; exit(0); }
+{ seen_minus = 0; seen_plus = 0; seen_at = 0; }
+END { exit (diff != 1); }
+' "$1"
+}
+
+# Checks if the patch contains one continuous hunk that contains
+# only additions.
+# Arguments:
+# $1 -- patch filename;
+# $2 -- line where patch contents start.
+is_continuous_patch () {
+
+ if [ -z "$1" -o -z "$2" ]; then
+ false
+ return
+ fi
+ awk -v start="$2" '
+BEGIN { blks = 0; in_blk = 0; }
+FNR < start { next; }
+/^-/ { exit(1); }
+in_blk == 0 && /^+/ { in_blk = 1; blks++; if (blks > 1) { exit(1); } next; }
+in_blk == 1 && !/^+/ { in_blk = 0; next; }
+END { exit(blks != 1); };
+' "$1"
+}
+
+# $1 is the file with the existing VuXML entry; it is guaranteed
+# to be a real file.
+#
+# We should check if the supplied file contains a patch that has
+# one continuous entry that is meant to be added -- only such files
+# are treated as patches that add VuXML entries. If patch has more
+# than one chunk or it deletes anything, we won't accept it.
+#
+# For the non-patch files, we will add them "as is".
+
+existing_entry () {
+ local patch_start
+
+ if [ -z "$1" ]; then
+ false
+ return
+ fi
+
+ # Plain file? Just cat it!
+ patch_start=`is_patch "$1"`
+ if [ "$?" != 0 ]; then
+ cat "$1"
+ echo
+ return
+ fi
+
+ if is_continuous_patch "$1" "$patch_start"; then
+ awk -v start="$patch_start" '
+FNR < start { next; }
+{ print; }
+' "$1" | grep '^\+' | sed -e's/^\+//'
+ else
+ cat >&2 << EOF
+Input VuXML entry looks like a patch, but it doesn't contain a single
+hunk that I can use as the entry body. Human insight is needed, may
+be it is better to merge the entry by hand.
+EOF
+ false
+ return
+ fi
+}
+
new_entry () {
local vid discovery entry
@@ -40,7 +122,7 @@ EOF
}
usage () {
- echo "Usage: newentry.sh /path/to/vuxml/document" >&2
+ echo "Usage: newentry.sh [-i /existing/entry] /path/to/vuxml/document" >&2
}
cleanup() {
@@ -49,6 +131,31 @@ cleanup() {
fi
}
+while getopts "i:h" opt; do
+ case "$opt" in
+ h)
+ usage
+ exit 0
+ ;;
+ i)
+ if [ ! -e "$OPTARG" ]; then
+ echo "File '$OPTARG' does not exist." >&2
+ exit 1
+ fi
+ entry="$OPTARG"
+ ;;
+ ?)
+ usage
+ exit 1
+ ;;
+ -)
+ break
+ ;;
+ esac
+done
+
+shift $(($OPTIND - 1))
+
vuxml_file="$1"
if [ -z "${vuxml_file}" ]; then
usage
@@ -60,7 +167,11 @@ trap cleanup EXIT 1 2 13 15
tmp="`mktemp ${TMPDIR:-/tmp}/vuxml.XXXXXXXXXX`" || exit 1
awk '/^<\?/,/^<vuxml/ { print }' "${vuxml_file}" >> "${tmp}" || exit 1
-new_entry >> "${tmp}" || exit 1
+if [ -n "$entry" ]; then
+ existing_entry "$entry" >> "${tmp}" || exit 1
+else
+ new_entry >> "${tmp}" || exit 1
+fi
awk '/^[[:space:]]+<vuln /,/^NONE$/ { print }' \
"${vuxml_file}" >> "${tmp}" || exit 1
--
1.6.5.3
--- 0004-teach-to-accept-pre-made-entries.diff ends here ---
To ease the things, patches are mirrored at the following locations:
http://codelabs.ru/fbsd/patches/vuxml/newentry.sh/0001-install-cleanup-handler-earlier.diff
http://codelabs.ru/fbsd/patches/vuxml/newentry.sh/0002-cleanup-before-spawning-editor.diff
http://codelabs.ru/fbsd/patches/vuxml/newentry.sh/0003-refactor-newentry.sh.diff
http://codelabs.ru/fbsd/patches/vuxml/newentry.sh/0004-teach-to-accept-pre-made-entries.diff
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091127140047.6A690DA81A>