From owner-svn-ports-head@FreeBSD.ORG Thu Aug 23 14:07:49 2012 Return-Path: Delivered-To: svn-ports-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF177106564A; Thu, 23 Aug 2012 14:07:48 +0000 (UTC) (envelope-from sbz@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id C842E8FC12; Thu, 23 Aug 2012 14:07:48 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q7NE7mSP036842; Thu, 23 Aug 2012 14:07:48 GMT (envelope-from sbz@svn.freebsd.org) Received: (from sbz@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q7NE7mej036838; Thu, 23 Aug 2012 14:07:48 GMT (envelope-from sbz@svn.freebsd.org) Message-Id: <201208231407.q7NE7mej036838@svn.freebsd.org> From: Sofian Brabez Date: Thu, 23 Aug 2012 14:07:48 +0000 (UTC) To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org X-SVN-Group: ports-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r303011 - head/Tools/scripts X-BeenThere: svn-ports-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the ports tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 14:07:49 -0000 Author: sbz Date: Thu Aug 23 14:07:48 2012 New Revision: 303011 URL: http://svn.freebsd.org/changeset/ports/303011 Log: - Add getpatch script to download patch attachments on bug tracking systems Approved by: portmgr (bapt) Added: head/Tools/scripts/README.getpatch (contents, props changed) head/Tools/scripts/getpatch (contents, props changed) Modified: head/Tools/scripts/README (contents, props changed) Modified: head/Tools/scripts/README ============================================================================== --- head/Tools/scripts/README Thu Aug 23 13:26:52 2012 (r303010) +++ head/Tools/scripts/README Thu Aug 23 14:07:48 2012 (r303011) @@ -24,6 +24,7 @@ distclean - compare md5 sums of distfile unmatched entries explicit_lib_depends.sh - shows the current explicit dependency list of libs for a given installed port +getpatch - downloads patch attachments on Bug Tracking Systems getpr - downloads a problem report from GNATS and attempts to extract the patch, shar, uuencoded file from it. this probably needs to be checked for potential security problems. Added: head/Tools/scripts/README.getpatch ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/Tools/scripts/README.getpatch Thu Aug 23 14:07:48 2012 (r303011) @@ -0,0 +1,83 @@ +GETPATCH(1) FreeBSD General Commands Manual GETPATCH(1) + +NAME + getpatch - An utility to download patch attachments on Bug Tracking Systems + +SYNOPSIS + getpatch [-h] [--mode gnats|bz] [--last] [--stdout] + +DESCRIPTION + getpatch is an utility to download patch attachments on Bug Tracking + Systems such Gnats and Bugzilla. It currently supports retrieving multiple + attachments from the command line. + + Its written in python witout any extra dependencies. Compare to getpr + utility it does web scrapping on BTS web interface in order to retrieve + patchs attached. + + The following command line options are supported: + + -h Prints a multi-line help message and exits + + --mode Specify BTS mode. Supported are "gnats" or "bz". + + --last Only retrieve last iteration patch attached. + + --stdout Output patch to stdout file descriptor. + + Options can be used after or before the on the command line. + +FILES + ${PORTSDIR}/Tools/scripts/getpatch + +EXAMPLES + Retrieve all patchs attached on pr ports/166692 on a Gnats BTS: + + getpatch --mode gnats ports/166692 + + or + + getpatch 166692 + + Gnats is the default mode and category isn't mandatory. + + Retrieve all patchs on a Bugzilla BTS: + + getpatch --mode bz ports/166692 + + Retrieve only last iteration of the patch: + + getpatch --last ports/166692 + + Retrieve patch on standard output + + getpatch --stdout ports/166692 + + On fly patching can be done in a port directory this way: + + For a diff + + cd ${PORTSDIR}/category/port + patch -p0 < <(getpatch 166692 --stdout) + + For a shar + + cd ${PORTSDIR}/category/port + sh <(getpatch 166692 --stdout) + + Redirection <() depends on the shell you're using, validated with zsh and bash. + +DIAGNOSTICS + getpatch exits 0 on success or 1 if a help message was output. + +SEE ALSO + getpr, prpatch + +AUTHORS + Sofian Brabez + +BUGS + If you're using getpatch and you encounter a bug or want an improvement + don't hesitate to mail me. + +FreeBSD 14 August 2012 FreeBSD Added: head/Tools/scripts/getpatch ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/Tools/scripts/getpatch Thu Aug 23 14:07:48 2012 (r303011) @@ -0,0 +1,191 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# +# Copyright (c) 2012 Sofian Brabez +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# +# $FreeBSD$ +# +# MAINTAINER= sbz@FreeBSD.org + +import argparse +import re +import sys +import urllib2 + +""" +FreeBSD getpatch handles Gnats and Bugzilla patch attachments +""" + +class GetPatch(object): + + def __init__(self, pr, category='ports'): + self.pr = pr + self.category = category + self.patchs = list() + self.url = str() + self.patch = str() + self.output_stdout = False + + def fetch(self, *largs, **kwargs): + raise NotImplementedError() + + def write(self, filename, data): + if filename.endswith(('.patch', '.txt')): + filename = filename[:filename.rindex('.')]+'.diff' + f=open(filename, 'w') + f.write(data) + f.close() + self.out("[+] %s created" % filename) + + def get(self,only_last=False, output_stdout=False): + self.output_stdout = output_stdout + self.fetch(self.pr, category=self.category) + + if len(self.patchs) == 0: + self.out("[-] No patch found") + sys.exit(1) + + if only_last: + self.patchs = [self.patchs.pop()] + + for patch in self.patchs: + url = patch['url'] + p = patch['name'] + + data = urllib2.urlopen(url).read() + + if self.output_stdout: + sys.stdout.write(data) + else: + self.write(p, data) + + def out(self, s): + if not self.output_stdout: + print(s) + +class GnatsGetPatch(GetPatch): + + URL_BASE = 'http://www.freebsd.org/cgi' + URL = '%s/query-pr.cgi?pr=' % URL_BASE + REGEX = r'Download ([^<]*)' + + def __init__(self, pr, category): + GetPatch.__init__(self, pr, category) + + def fetch(self, *largs, **kwargs): + category = kwargs['category'] + target = ("%s/%s" % (category, self.pr), "%s" % self.pr)[category==''] + self.out("[+] Fetching patch for pr %s" % target) + pattern = re.compile(self.REGEX) + u = urllib2.urlopen(self.URL+'%s' % target) + data = u.read() + if data == None: + self.out("[-] No patch found") + sys.exit(1) + + for patchs in re.findall(pattern, data): + self.patchs.append({'url': patchs[0], 'name': patchs[1]}) + +class BzGetPatch(GetPatch): + + URL_BASE='https://bugzilla.freebsd.org' + URL_SHOW = '%s/show_bug.cgi?id=' % URL_BASE + REGEX_URL = r'Details' + REGEX = r'
([^ ]+) \(text/plain\)' + + def __init__(self, pr, category): + GetPatch.__init__(self, pr, category) + + def _extract_patchs_url(self, data): + pattern = re.compile(self.REGEX_URL) + return re.findall(pattern, data) + + def _extract_patchs_name(self, urls): + names = [] + pattern = re.compile(self.REGEX) + for url in urls: + u = urllib2.urlopen('%s/%s' % (self.URL_BASE, url)) + data = u.read() + names.append(re.findall(pattern, data)[0]) + + return names + + def fetch(self, *largs, **kwargs): + category = kwargs['category'] + self.out("[+] Fetching patch for pr %s/%s" % (category, self.pr)) + u = urllib2.urlopen(self.URL_SHOW+'%s' % self.pr) + data = u.read() + + if data == None: + self.out("[-] No patch found") + sys.exit(1) + + urls = self._extract_patchs_url(data) + nb_urls = len(urls) + names = self._extract_patchs_name(urls) + nb_names = len(names) + + urls = ['%s/%s' % (self.URL_BASE, u[:u.find('&')]) for u in urls] + + if nb_names == 0 or nb_urls == 0 or nb_names != nb_urls: + self.out("[-] No patch found") + sys.exit(1) + + for i in xrange(nb_urls): + self.patchs.append({'url': urls[i], 'name': names[i]}) + +def main(): + + parser = argparse.ArgumentParser( + description='Gets patch attachments from Bug Tracking System' + ) + parser.add_argument('pr', metavar='pr', type=str, nargs=1, + help='Pr id number') + parser.add_argument('--mode', type=str, choices=['gnats','bz'], + default='gnats', help='available modes to retrieve patch') + parser.add_argument('--last', action='store_true', + help='only retrieve last iteration of the patch') + parser.add_argument('--stdout', action='store_true', + help='output patch on stdout') + + if len(sys.argv) == 1: + parser.print_help() + sys.exit(1) + + args = parser.parse_args() + + category = str() + pr = str(args.pr[0]) + + if '/' in pr and pr is not None: + category = pr.split('/')[0] + pr = pr.split('/')[1] + + Clazz = globals()['%sGetPatch' % args.mode.capitalize()] + gp = Clazz(pr, category) + gp.get(only_last=args.last, output_stdout=args.stdout) + +if __name__ == '__main__': + + main()