From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 02:44:54 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BAC79185 for ; Tue, 15 Oct 2013 02:44:54 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-la0-f45.google.com (mail-la0-f45.google.com [209.85.215.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4724A290A for ; Tue, 15 Oct 2013 02:44:53 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id eh20so6343287lab.18 for ; Mon, 14 Oct 2013 19:44:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-type; bh=kSVimKYCohdsgBkr3QnV/9nGzL5y53onkpO7e6GEUI0=; b=PucwUP8Rr/pl1yEtFGE0qnjbpMpkGbRMb1XjhgI3N3/rVGuMAtUPAi3OIrNuO3Nmqb LgnI1cq2i6GJ4i4W62aIedUkgw2yjrtcJubXjKTEmjxArPM5x8iSi66stP8b6X4L6YLk HjB0dyzI/geFqwsjMBZeDFGRvOjBuJVOK1zQhPiXMV68QGRH7CKupeEcrpNkE1+Qq0IF OIcUWdSyJu8BaRWRYxupKL5OyD71cnk+m+hgMYHoLHJv2nXqBG4+LHrT2FQZsCUX3Cdc 4jjus90z8dUPbO8fHUIaSp5hTMC+XQFQ3LvBC8qgQX/bPSbusK528fzAZZx/epvzId5Q GsSQ== X-Gm-Message-State: ALoCoQlicn8FjBH78XJT+c5fpi1sCaz/cL3x2rfJICcIzrZLYRFqSGnxbSDhA/+ZNOic9IL2QI25 X-Received: by 10.112.146.200 with SMTP id te8mr4678174lbb.32.1381805085682; Mon, 14 Oct 2013 19:44:45 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.132.135 with HTTP; Mon, 14 Oct 2013 19:44:25 -0700 (PDT) X-Originating-IP: [108.176.158.82] From: Julio Merino Date: Mon, 14 Oct 2013 22:44:25 -0400 Message-ID: Subject: Refactoring of *.test.mk To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Rui Paulo , "yaneurabeya ." , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 02:44:54 -0000 Hello all, [ CCing a few people directly that are probably interested. ] After the recent removal of the deprecated ATF tools from the tree, I have been working on a refactoring of the FreeBSD *.test.mk Makefiles to add various features that spilled from that change. The highlights of this work so far: - Clearly documented the interface of each *.test.mk file by putting the list of public variables at the top and adding comments where due. - Split the logic to build framework-less test programs into a new plain.test.mk file. This is in accordance with atf.test.mk, but also helps clarifying the code, defining a proper interfac in bsd.test.mk and makes some of the other changes simpler. - Added support to generate both Atffile and Kyuafile files. - Added support to run the tests using both atf-run and kyua (from ports). You can find the relevant patches in: http://portal.meroh.net/~jmmv/freebsd-testing/ . The patches have been generated using quilt so take a look at the contents of the 'series' file to see in which order they would need to be applied. Each patch has a description at the top which can later be used as the commit log. Note that, at the moment, this can only be exercised in lib/libcrypt/tests/ . I actually wanted to write some tests for *.test.mk themselves (!) but I wouldn't want to spend time on that before agreeing that this structure is sensible. If it is, I might put the effort ;-) See the ones here http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/share/mk/?only_with_tag=MAIN to get an idea of what they could look like. Anybody up for reviewing these patches and sharing their comments? Thanks! -- Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 03:25:08 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 97E6C814 for ; Tue, 15 Oct 2013 03:25:08 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-lb0-f177.google.com (mail-lb0-f177.google.com [209.85.217.177]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 224202B71 for ; Tue, 15 Oct 2013 03:25:07 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id w7so6161847lbi.8 for ; Mon, 14 Oct 2013 20:24:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=Lii09pDL2WZbDPwi30+SjwRVdUm+poo+opQ9vO4AXhc=; b=Co09aKt6JoG/3D2+DrJL7aZhDnhvX3tcNIuYB2NQO/8ewC23duw3i/BxDiC9QKuNJ2 Y8+d1i1RZBbZzjStlT+dILnhCmsr5Fwoe1u/CHrNNf/3LdZaRKqRiLKLS7s9AWqF067r 6+Adc/dvEfbCupAQDxgHtXJCpARTRFwZEwMPmhyBw9vS2XFx958S41yoB/joSkKhAw8P BYIMsdw4zvItbL6KzTfAPECoE9yXKPo9gRKblHGhD/V8T7nbSwuUzmlWj2U90GrH5pFV Y3ctCc9dsmHr028BteMMKsMMtXeECWc8Ctjn04jf8boApiZU1MucVDOAYFOtqVo4y0WP KfKw== X-Gm-Message-State: ALoCoQnr0PRrjEtoDN0X5bRtipOnzhhpcvGSferXqJKNsXkEKwGvRvWcn1gXlikrKtfRS0r97PLQ X-Received: by 10.112.42.68 with SMTP id m4mr33569266lbl.4.1381807499694; Mon, 14 Oct 2013 20:24:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.132.135 with HTTP; Mon, 14 Oct 2013 20:24:39 -0700 (PDT) X-Originating-IP: [108.176.158.82] In-Reply-To: References: From: Julio Merino Date: Mon, 14 Oct 2013 23:24:39 -0400 Message-ID: Subject: Re: Refactoring of *.test.mk To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Rui Paulo , Garrett Cooper , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 03:25:08 -0000 [Fixing Garrett's address...] On Mon, Oct 14, 2013 at 10:44 PM, Julio Merino wrote: > Hello all, > > [ CCing a few people directly that are probably interested. ] > > After the recent removal of the deprecated ATF tools from the tree, I > have been working on a refactoring of the FreeBSD *.test.mk Makefiles > to add various features that spilled from that change. > > The highlights of this work so far: > > - Clearly documented the interface of each *.test.mk file by putting > the list of public variables at the top and adding comments where due. > > - Split the logic to build framework-less test programs into a new > plain.test.mk file. This is in accordance with atf.test.mk, but also > helps clarifying the code, defining a proper interfac in bsd.test.mk > and makes some of the other changes simpler. > > - Added support to generate both Atffile and Kyuafile files. > > - Added support to run the tests using both atf-run and kyua (from ports). > > You can find the relevant patches in: > http://portal.meroh.net/~jmmv/freebsd-testing/ . > > The patches have been generated using quilt so take a look at the > contents of the 'series' file to see in which order they would need to > be applied. > > Each patch has a description at the top which can later be used as the > commit log. > > Note that, at the moment, this can only be exercised in > lib/libcrypt/tests/ . I actually wanted to write some tests for > *.test.mk themselves (!) but I wouldn't want to spend time on that > before agreeing that this structure is sensible. If it is, I might put > the effort ;-) See the ones here > http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/share/mk/?only_with_tag=MAIN > to get an idea of what they could look like. > > Anybody up for reviewing these patches and sharing their comments? > > Thanks! > > -- > Julio Merino / @jmmv -- Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 05:28:22 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 39CF7B51 for ; Tue, 15 Oct 2013 05:28:22 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B8A6B221E for ; Tue, 15 Oct 2013 05:28:21 +0000 (UTC) Received: by mail-wg0-f43.google.com with SMTP id b13so4035963wgh.10 for ; Mon, 14 Oct 2013 22:28:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=iORvd9XabN3ARaYZBgdEL9kgYWY3caZu5KG1cqQMHhk=; b=TY/CiyVAkQKqMr4rNLK/azSw/eXH2al7oNh/xNuIgup/8OKH75SiGZHshgL8aRdrAl tJJ9hjX/OCMB/3mauZBQSy0y0BSNWBoXbZnkk94e/B2wHKPLcoXPwZjfOyPhAkw1tAeQ dQCTEJBmZP0sVdoZs00Zgy3W9BcRL0w0xp6O/8UdR2/rA2gfQPgknt9Ds+WDiqWmLysw ax4PNrVRT4IS7NEBP+xIjJr7xtirArqcJIB4yvkIuINUiWgNcFkfCztOS/pliV6yDIAA 7TtHgm9bLbZo2SJ9cOFdhHIDqG+UjQQtKGe9vKPFEUcvG+Ypa5s37UauyIBNlpX+vdRi Zpjw== MIME-Version: 1.0 X-Received: by 10.180.126.101 with SMTP id mx5mr17000989wib.63.1381814899805; Mon, 14 Oct 2013 22:28:19 -0700 (PDT) Received: by 10.227.121.68 with HTTP; Mon, 14 Oct 2013 22:28:19 -0700 (PDT) In-Reply-To: References: Date: Mon, 14 Oct 2013 22:28:19 -0700 Message-ID: Subject: Re: Refactoring of *.test.mk From: Rui Paulo To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-testing@freebsd.org, Rui Paulo , Garrett Cooper , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 05:28:22 -0000 On 14 October 2013 20:24, Julio Merino wrote: > [Fixing Garrett's address...] > > On Mon, Oct 14, 2013 at 10:44 PM, Julio Merino wrote: > > Hello all, > > > > [ CCing a few people directly that are probably interested. ] > > > > After the recent removal of the deprecated ATF tools from the tree, I > > have been working on a refactoring of the FreeBSD *.test.mk Makefiles > > to add various features that spilled from that change. > > > > The highlights of this work so far: > > > > - Clearly documented the interface of each *.test.mk file by putting > > the list of public variables at the top and adding comments where due. > > > > - Split the logic to build framework-less test programs into a new > > plain.test.mk file. This is in accordance with atf.test.mk, but also > > helps clarifying the code, defining a proper interfac in bsd.test.mk > > and makes some of the other changes simpler. > > > > - Added support to generate both Atffile and Kyuafile files. > > > > - Added support to run the tests using both atf-run and kyua (from > ports). > > > > You can find the relevant patches in: > > http://portal.meroh.net/~jmmv/freebsd-testing/ . > > > > The patches have been generated using quilt so take a look at the > > contents of the 'series' file to see in which order they would need to > > be applied. > > > > Each patch has a description at the top which can later be used as the > > commit log. > > > > Note that, at the moment, this can only be exercised in > > lib/libcrypt/tests/ . I actually wanted to write some tests for > > *.test.mk themselves (!) but I wouldn't want to spend time on that > > before agreeing that this structure is sensible. If it is, I might put > > the effort ;-) See the ones here > > > http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/share/mk/?only_with_tag=MAIN > > to get an idea of what they could look like. > > > > Anybody up for reviewing these patches and sharing their comments? > Looks simple enough. I don't see any problems with it. -- Rui Paulo From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 06:08:31 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 063ED3E4 for ; Tue, 15 Oct 2013 06:08:31 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from ch1outboundpool.messaging.microsoft.com (ch1ehsobe003.messaging.microsoft.com [216.32.181.183]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B8605246B for ; Tue, 15 Oct 2013 06:08:29 +0000 (UTC) Received: from mail187-ch1-R.bigfish.com (10.43.68.243) by CH1EHSOBE018.bigfish.com (10.43.70.68) with Microsoft SMTP Server id 14.1.225.22; Tue, 15 Oct 2013 06:08:20 +0000 Received: from mail187-ch1 (localhost [127.0.0.1]) by mail187-ch1-R.bigfish.com (Postfix) with ESMTP id 088FC3E01FF; Tue, 15 Oct 2013 06:08:20 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.224.50; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF01-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 2 X-BigFish: VPS2(zz1432Izz1f42h208ch1ee6h1de0h1fdah2073h1202h1e76h1d1ah1d2ah1fc6h1082kzz8275ch17326ah1de097h186068hz2fh2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h1155h) Received-SPF: pass (mail187-ch1: domain of juniper.net designates 66.129.224.50 as permitted sender) client-ip=66.129.224.50; envelope-from=sjg@juniper.net; helo=P-EMF01-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail187-ch1 (localhost.localdomain [127.0.0.1]) by mail187-ch1 (MessageSwitch) id 1381817298325009_6956; Tue, 15 Oct 2013 06:08:18 +0000 (UTC) Received: from CH1EHSMHS024.bigfish.com (snatpool1.int.messaging.microsoft.com [10.43.68.253]) by mail187-ch1.bigfish.com (Postfix) with ESMTP id 4A3C3180041; Tue, 15 Oct 2013 06:08:18 +0000 (UTC) Received: from P-EMF01-SAC.jnpr.net (66.129.224.50) by CH1EHSMHS024.bigfish.com (10.43.70.24) with Microsoft SMTP Server (TLS) id 14.16.227.3; Tue, 15 Oct 2013 06:08:17 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Mon, 14 Oct 2013 23:08:16 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id r9F68GL02044; Mon, 14 Oct 2013 23:08:16 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id C4DFC5807E; Mon, 14 Oct 2013 23:08:15 -0700 (PDT) To: Julio Merino Subject: Re: Refactoring of *.test.mk In-Reply-To: References: Comments: In-reply-to: Julio Merino message dated "Mon, 14 Oct 2013 22:44:25 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Mon, 14 Oct 2013 23:08:15 -0700 Message-ID: <20131015060815.C4DFC5807E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: freebsd-testing@freebsd.org, Rui Paulo , "yaneurabeya ." , sjg@juniper.net X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 06:08:31 -0000 Thanks Julio, will take a look. We have made a number of changes internally to *.test.mk so that it works well in our build env. >You can find the relevant patches in: >http://portal.meroh.net/~jmmv/freebsd-testing/ . > >The patches have been generated using quilt so take a look at the >contents of the 'series' file to see in which order they would need to >be applied. > >Each patch has a description at the top which can later be used as the >commit log. From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 06:23:01 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 97C09543 for ; Tue, 15 Oct 2013 06:23:01 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe005.messaging.microsoft.com [216.32.180.31]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 51851250B for ; Tue, 15 Oct 2013 06:23:00 +0000 (UTC) Received: from mail223-va3-R.bigfish.com (10.7.14.252) by VA3EHSOBE003.bigfish.com (10.7.40.23) with Microsoft SMTP Server id 14.1.225.23; Tue, 15 Oct 2013 06:22:53 +0000 Received: from mail223-va3 (localhost [127.0.0.1]) by mail223-va3-R.bigfish.com (Postfix) with ESMTP id A03BA1400C5; Tue, 15 Oct 2013 06:22:53 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.224.54; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF01-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 3 X-BigFish: VPS3(zzzz1f42h208ch1ee6h1de0h1fdah2073h1202h1e76h1d1ah1d2ah1fc6h1082kzz1de097hz2fh2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h1155h) Received-SPF: pass (mail223-va3: domain of juniper.net designates 66.129.224.54 as permitted sender) client-ip=66.129.224.54; envelope-from=sjg@juniper.net; helo=P-EMF01-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail223-va3 (localhost.localdomain [127.0.0.1]) by mail223-va3 (MessageSwitch) id 1381818172282177_17029; Tue, 15 Oct 2013 06:22:52 +0000 (UTC) Received: from VA3EHSMHS011.bigfish.com (unknown [10.7.14.246]) by mail223-va3.bigfish.com (Postfix) with ESMTP id 40E29400049; Tue, 15 Oct 2013 06:22:52 +0000 (UTC) Received: from P-EMF01-SAC.jnpr.net (66.129.224.54) by VA3EHSMHS011.bigfish.com (10.7.99.21) with Microsoft SMTP Server (TLS) id 14.16.227.3; Tue, 15 Oct 2013 06:22:50 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Mon, 14 Oct 2013 23:21:40 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id r9F6LeL07699; Mon, 14 Oct 2013 23:21:40 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id DAD595807E; Mon, 14 Oct 2013 23:21:39 -0700 (PDT) To: Julio Merino Subject: Re: Refactoring of *.test.mk In-Reply-To: References: Comments: In-reply-to: Julio Merino message dated "Mon, 14 Oct 2013 23:24:39 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Mon, 14 Oct 2013 23:21:39 -0700 Message-ID: <20131015062139.DAD595807E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: freebsd-testing@freebsd.org, Rui Paulo , Garrett Cooper X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 06:23:01 -0000 Thanks Julio, looks good. One nit; in hook-atf-run.diff and hook-kyua.diff I see: +TESTS_ENV+= ATF_BUILD_CC=${DESTDIR}/usr/bin/cc etc. It would be safer to allow ATF_BUILD_CC etc to be controlled eg. TESTS_ENV+= ATF_BUILD_CC=${ATF_BUILD_CC:U${DESTDIR}/usr/bin/cc} .. or ATF_BUILD_CC?= ${DESTDIR}/usr/bin/cc .. TESTS_ENV+= ATF_BUILD_CC=${ATF_BUILD_CC} .. On Mon, 14 Oct 2013 23:24:39 -0400, Julio Merino writes: >[Fixing Garrett's address...] From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 13:54:42 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3B628DC4 for ; Tue, 15 Oct 2013 13:54:42 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-lb0-f174.google.com (mail-lb0-f174.google.com [209.85.217.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BBDF525D8 for ; Tue, 15 Oct 2013 13:54:41 +0000 (UTC) Received: by mail-lb0-f174.google.com with SMTP id w6so6807083lbh.19 for ; Tue, 15 Oct 2013 06:54:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=KoGSJ13VeflQOZCJPXnwF4phMr3KOAPdqyHpE4tcrmk=; b=mnhLg/SwGXawadaU/9dPHoO9GB7TwzmI6dJMGXnL1sfi165gv5p1Uoyc+ApwPSUBfv amCqL7+wMsR4tkjBBtH32rdHNTWtIimoaQZUX8oyunhr17XnnF/VU7T4UK+Oyh6LFD/7 qpoyQFng2e0Z1ixfuks3blzKzQP13UMM9dqBHy+La02J0bDSt5qlpGWOy99QCevBbjMj Yj7pA+GtGkAeDM4s3Tke7Z1E3Awxd/kYb0nC9yX+RgxDRRKdzVDQhpaN4cEYBEJu7EHA G2EtFp4vpRQnk/ZNMT6D0iNRZNcGKB17rx4hbAHVowu520UXKRy82Pm/94Vt/shyjMyR +gvA== X-Gm-Message-State: ALoCoQmnntlVU0dRAy5/aM52txmbB3Nbn6Sfs7HF+rwyFnDnyboQz188dVUGPdzcnPO31XRtAE1n X-Received: by 10.112.52.225 with SMTP id w1mr6890488lbo.31.1381845274053; Tue, 15 Oct 2013 06:54:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.132.135 with HTTP; Tue, 15 Oct 2013 06:54:14 -0700 (PDT) X-Originating-IP: [2620:0:1003:1007:f060:8d4:3979:1541] In-Reply-To: <20131015062139.DAD595807E@chaos.jnpr.net> References: <20131015062139.DAD595807E@chaos.jnpr.net> From: Julio Merino Date: Tue, 15 Oct 2013 09:54:14 -0400 Message-ID: Subject: Re: Refactoring of *.test.mk To: "Simon J. Gerraty" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-testing@freebsd.org, Rui Paulo , Garrett Cooper X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 13:54:42 -0000 On Tue, Oct 15, 2013 at 2:21 AM, Simon J. Gerraty wrote: > Thanks Julio, > > looks good. Thanks for the reviews Simon and Rui. > One nit; in hook-atf-run.diff and hook-kyua.diff I see: [...] > ATF_BUILD_CC?= ${DESTDIR}/usr/bin/cc > .. > TESTS_ENV+= ATF_BUILD_CC=${ATF_BUILD_CC} > .. OK, I've done this. And, at the same time, improved the way TESTS_ENV is constructed by leaving the atf-specific bits in atf.test.mk and moving the path handling to bsd.test.mk. PTAL. I suspect that with 10 branched, we don't need approval from re@ for submission at this point; is that correct? That said, I think it's worth to merge this into 10 (now or after release, doesn't matter much). Given that this touches yet-unused files, it shouldn't be a major concern for re@. The reason for merging into 10 is that it'd be nice if any future tests written in HEAD for regressions encountered in 10 could be pulled up into the branch with ease along the actual fixes. Cheers, -- Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 15:35:27 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 381F4BC2 for ; Tue, 15 Oct 2013 15:35:27 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from db9outboundpool.messaging.microsoft.com (mail-db9lp0248.outbound.messaging.microsoft.com [213.199.154.248]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BBB712FE7 for ; Tue, 15 Oct 2013 15:35:26 +0000 (UTC) Received: from mail83-db9-R.bigfish.com (10.174.16.242) by DB9EHSOBE036.bigfish.com (10.174.14.99) with Microsoft SMTP Server id 14.1.225.22; Tue, 15 Oct 2013 15:35:23 +0000 Received: from mail83-db9 (localhost [127.0.0.1]) by mail83-db9-R.bigfish.com (Postfix) with ESMTP id BE32F8006B; Tue, 15 Oct 2013 15:35:23 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.224.52; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF01-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 3 X-BigFish: VPS3(zzzz1f42h208ch1ee6h1de0h1fdah2073h1202h1e76h1d1ah1d2ah1fc6h1082kzzz2fh2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h1155h) Received-SPF: pass (mail83-db9: domain of juniper.net designates 66.129.224.52 as permitted sender) client-ip=66.129.224.52; envelope-from=sjg@juniper.net; helo=P-EMF01-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail83-db9 (localhost.localdomain [127.0.0.1]) by mail83-db9 (MessageSwitch) id 1381851321968669_17711; Tue, 15 Oct 2013 15:35:21 +0000 (UTC) Received: from DB9EHSMHS028.bigfish.com (unknown [10.174.16.249]) by mail83-db9.bigfish.com (Postfix) with ESMTP id D52CD3A0075; Tue, 15 Oct 2013 15:35:21 +0000 (UTC) Received: from P-EMF01-SAC.jnpr.net (66.129.224.52) by DB9EHSMHS028.bigfish.com (10.174.14.38) with Microsoft SMTP Server (TLS) id 14.16.227.3; Tue, 15 Oct 2013 15:35:20 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Tue, 15 Oct 2013 08:35:18 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id r9FFZ4L29825; Tue, 15 Oct 2013 08:35:09 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id C6EDF5807E; Tue, 15 Oct 2013 08:35:03 -0700 (PDT) To: Julio Merino Subject: Re: Refactoring of *.test.mk In-Reply-To: References: <20131015062139.DAD595807E@chaos.jnpr.net> Comments: In-reply-to: Julio Merino message dated "Tue, 15 Oct 2013 09:54:14 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Tue, 15 Oct 2013 08:35:03 -0700 Message-ID: <20131015153503.C6EDF5807E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: freebsd-testing@freebsd.org, Rui Paulo , Garrett Cooper , sjg@juniper.net X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 15:35:27 -0000 On Tue, 15 Oct 2013 09:54:14 -0400, Julio Merino writes: >I suspect that with 10 branched, we don't need approval from re@ for >submission at this point; is that correct? head was restricted until yesterday I think. >That said, I think it's worth to merge this into 10 (now or after >release, doesn't matter much). Given that this touches yet-unused >files, it shouldn't be a major concern for re@. The reason for merging >into 10 is that it'd be nice if any future tests written in HEAD for >regressions encountered in 10 could be pulled up into the branch with >ease along the actual fixes. Sounds reasonable. From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 16:21:07 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1C20AD33 for ; Tue, 15 Oct 2013 16:21:07 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E7E5A23F4 for ; Tue, 15 Oct 2013 16:21:06 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id kq14so9222686pab.40 for ; Tue, 15 Oct 2013 09:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=JL5P6IUzLvJeZgagRvM2T5p3ukOBSKbPTdpL83I+TeY=; b=CzdkN9qGwUevBFD2/Uz6T8EEyd62AcutwXFiYXP6qRu2FxWFddvUdoY/ewLroR87HJ DNNV3yiR+XahRs3DgPTuKpd3GdwT26cSmjrkPzTaRJBTduCI0SlfVrgqVOgfLwT8beiS jIN/jTGNfm7RttLhBUnxjbFwzG/0BIWBcK+Aoq2WHs8IHXTX0ItRngWxDonBtYqkc/Lk jozHz7epYJgw9JI7+3oLzFrp8gy6PSdRBi3lf0185Moda3kSPlcyWHJTor3VAkd6mI2j uN1Z9je4Lm0cVCdpPmWfE9iZeMnzuCEGk/HcCn6D6RKuUkBeAPLZ1ceTUv4l1HYkv3Qu qwGw== X-Received: by 10.68.172.36 with SMTP id az4mr42929363pbc.48.1381854065966; Tue, 15 Oct 2013 09:21:05 -0700 (PDT) Received: from [10.0.1.114] ([64.14.143.130]) by mx.google.com with ESMTPSA id hz10sm85502523pbc.36.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Oct 2013 09:21:04 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <0017182B-9793-4A39-BC3F-7FF684AA1224@gmail.com> X-Mailer: iPhone Mail (11A501) From: Garrett Cooper Subject: Re: Refactoring of *.test.mk Date: Tue, 15 Oct 2013 09:21:03 -0700 To: Julio Merino Cc: "freebsd-testing@freebsd.org" , Rui Paulo , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 16:21:07 -0000 > On Oct 14, 2013, at 20:24, Julio Merino wrote: > > [Fixing Garrett's address...] > >> On Mon, Oct 14, 2013 at 10:44 PM, Julio Merino wrote: >> Hello all, >> >> [ CCing a few people directly that are probably interested. ] >> >> After the recent removal of the deprecated ATF tools from the tree, I >> have been working on a refactoring of the FreeBSD *.test.mk Makefiles >> to add various features that spilled from that change. >> >> The highlights of this work so far: >> >> - Clearly documented the interface of each *.test.mk file by putting >> the list of public variables at the top and adding comments where due. >> >> - Split the logic to build framework-less test programs into a new >> plain.test.mk file. This is in accordance with atf.test.mk, but also >> helps clarifying the code, defining a proper interfac in bsd.test.mk >> and makes some of the other changes simpler. >> >> - Added support to generate both Atffile and Kyuafile files. >> >> - Added support to run the tests using both atf-run and kyua (from ports). >> >> You can find the relevant patches in: >> http://portal.meroh.net/~jmmv/freebsd-testing/ . >> >> The patches have been generated using quilt so take a look at the >> contents of the 'series' file to see in which order they would need to >> be applied. >> >> Each patch has a description at the top which can later be used as the >> commit log. >> >> Note that, at the moment, this can only be exercised in >> lib/libcrypt/tests/ . I actually wanted to write some tests for >> *.test.mk themselves (!) but I wouldn't want to spend time on that >> before agreeing that this structure is sensible. If it is, I might put >> the effort ;-) See the ones here >> http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/share/mk/?only_with_tag=MAIN >> to get an idea of what they could look like. >> >> Anybody up for reviewing these patches and sharing their comments? >> >> Thanks! >> >> -- >> Julio Merino / @jmmv > > > > -- > Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 16:25:16 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E1D52EC3 for ; Tue, 15 Oct 2013 16:25:16 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B9C402440 for ; Tue, 15 Oct 2013 16:25:16 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id v10so9158524pde.10 for ; Tue, 15 Oct 2013 09:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=0erblrAbiroPLDmN8kNaFFJRpInAT2LLK6lA5riDeEc=; b=a7P+wzVqQ7BknPqY9PV+UAU8IjYKuZK8a/vB2W6sBCGqJe0q3kdtUPTLdZ/dg1VLS2 pYhld8Y9xLnlV0zRoHXjrgbIhuyoVmlTj6QwMvt3UpHfBCYhedVK92loBTXsjo/KVoJf o0pZGoKgWU1vGc9rwN+gzh/g+7CQBZSa0/VsXPW4PMPDO20279PYM4+NaoMMIKeeYVz8 32iMDv6EFeacfnPQXmWBYGWP7OxAOOkwK5ebbW5yvWZlhxZRmSf4BhonBVgPed+GQceP 6i/JnmeYrXyn6d9VFt6hXSDOkSQ15YWuNOmayr5rP5RbqGrjKYOsB1eNjusmtKZo2nLN Z25Q== X-Received: by 10.66.7.68 with SMTP id h4mr45045094paa.0.1381854316400; Tue, 15 Oct 2013 09:25:16 -0700 (PDT) Received: from [10.0.1.114] ([64.14.143.130]) by mx.google.com with ESMTPSA id bn4sm39394951pbc.11.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Oct 2013 09:25:15 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <4CE11789-6889-438B-89DF-CEBE4ACEC841@gmail.com> X-Mailer: iPhone Mail (11A501) From: Garrett Cooper Subject: Re: Refactoring of *.test.mk Date: Tue, 15 Oct 2013 09:25:14 -0700 To: Julio Merino Cc: "freebsd-testing@freebsd.org" , Rui Paulo , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 16:25:16 -0000 On Oct 14, 2013, at 20:24, Julio Merino wrote: >=20 > [Fixing Garrett's address...] (Sorry for the empty reply. Still getting used to iOS 7..) I'll look at this sometime before the end of the week. I have some minor con= cerns about the proposed layout and how things will integrate in, but these s= hould be easy to resolve. Also, did you look at my code on GitHub under share/mk? It's quite a bit dif= ferent from the stuff in FreeBSD head.. Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 20:50:13 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2DD35149 for ; Tue, 15 Oct 2013 20:50:13 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E2CB8279F for ; Tue, 15 Oct 2013 20:50:12 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id w8so3547328qac.11 for ; Tue, 15 Oct 2013 13:50:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=rXXmmef0xxkgebA4KyJdgDMCh7kg62AuuaCCOJL0wd0=; b=xP4tvfjusJqyutUUCjruE/4faqWYsY9xCiGtvSwevYiIqJb+Ij85GRn/wPZIQxm3WD CFj0vSI83vC9Yhcn7JiBfKsgxM/kBUM67okm2qOo1fsa7VFtQkFkr9CPB6D4/SzlxqPE bU316YcMvrlBZ923mp25zXPGtkx2O2KkI3tB5dTx9M8T2bozuN1gEQFRW9McYvqaBAKh 0b/Vsja08PZtGRd3JtAljwgEBHIFLU/q2IdneEG2Q1jFh5jkXeL+n1/IoOTyPi/w1457 Ev3uRirBx9GfTuFMZGRWMlVSpA/jhiVsSZvCzJMc6LRgnJZBxyiyIfawAsg7eR8x1z2d qoyA== MIME-Version: 1.0 X-Received: by 10.224.14.79 with SMTP id f15mr5526574qaa.113.1381870212104; Tue, 15 Oct 2013 13:50:12 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.49.39.97 with HTTP; Tue, 15 Oct 2013 13:50:12 -0700 (PDT) In-Reply-To: <4CE11789-6889-438B-89DF-CEBE4ACEC841@gmail.com> References: <4CE11789-6889-438B-89DF-CEBE4ACEC841@gmail.com> Date: Tue, 15 Oct 2013 14:50:12 -0600 X-Google-Sender-Auth: W-Mr7yB2zrvNM-5d-aA4yGWT4FQ Message-ID: Subject: Re: Refactoring of *.test.mk From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Rui Paulo , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 20:50:13 -0000 On Tue, Oct 15, 2013 at 10:25 AM, Garrett Cooper wrote: > On Oct 14, 2013, at 20:24, Julio Merino wrote: >> >> [Fixing Garrett's address...] > > (Sorry for the empty reply. Still getting used to iOS 7..) > > I'll look at this sometime before the end of the week. I have some minor concerns about the proposed layout and how things will integrate in, but these should be easy to resolve. > > Also, did you look at my code on GitHub under share/mk? It's quite a bit different from the stuff in FreeBSD head.. I'm currently trying to merge Garrett's stuff from GitHub. So far, I only see one thing that I would like to change. It would be really nice to be able to override the test_suite variable. That way, we could use bsd.test.mk to install ATF tests from Ports and use separate variable namespaces for them. -Alan > > Thanks! > -Garrett > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 21:11:30 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 63E53C84 for ; Tue, 15 Oct 2013 21:11:30 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E26A22915 for ; Tue, 15 Oct 2013 21:11:29 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id ea20so7303360lab.41 for ; Tue, 15 Oct 2013 14:11:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=NZnGVRk66aces74z+aQ7KvgBvyK8ORHwgjJfhfuVas0=; b=Ze/VUXTejj0JjznJIu50WslWrU+jt7TuzWqa2O5tTIE9RE1AevARyl/l7olX+f0/N+ 6WFTYvOTfxbCtPDnmFd9eZ7Ebn0PZtihjXLtXD/8OqNUSIb5X3L78B/oagLFXO5SJz2I G7CHxsuh5LVSJLeNogVCv0OocFBbQ8o8J9b17XVwSigHstDWBAOSYVzcIB0csmPUXnoT Yry0iZ98Qyeu0x3WmGQMWejwdKy3HmOjSWUYBJt2VodWoG5nHYRz2Qw3fn0RF9srC51b sn+3wi409NsJULxE+53hYO+AeMCD00tisdFtLjN9rB75BGvR3hZACGNOrXqsLvU+6ysF KpWA== X-Gm-Message-State: ALoCoQlnFiqTaVPHdI3hiAIt47Tem/vO7COfb/zG1W06Z8JeoKlm4hhVKEZFH+yHyxS+C6UqScAT X-Received: by 10.112.182.7 with SMTP id ea7mr13623lbc.63.1381871487833; Tue, 15 Oct 2013 14:11:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.132.135 with HTTP; Tue, 15 Oct 2013 14:11:07 -0700 (PDT) X-Originating-IP: [2620:0:1003:1007:f060:8d4:3979:1541] In-Reply-To: References: <4CE11789-6889-438B-89DF-CEBE4ACEC841@gmail.com> From: Julio Merino Date: Tue, 15 Oct 2013 17:11:07 -0400 Message-ID: Subject: Re: Refactoring of *.test.mk To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Rui Paulo , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 21:11:30 -0000 On Tue, Oct 15, 2013 at 4:50 PM, Alan Somers wrote: > On Tue, Oct 15, 2013 at 10:25 AM, Garrett Cooper wrote: >> On Oct 14, 2013, at 20:24, Julio Merino wrote: >>> >>> [Fixing Garrett's address...] >> >> (Sorry for the empty reply. Still getting used to iOS 7..) >> >> I'll look at this sometime before the end of the week. I have some minor concerns about the proposed layout and how things will integrate in, but these should be easy to resolve. >> >> Also, did you look at my code on GitHub under share/mk? It's quite a bit different from the stuff in FreeBSD head.. > > I'm currently trying to merge Garrett's stuff from GitHub. So far, I > only see one thing that I would like to change. It would be really > nice to be able to override the test_suite variable. Good idea. Updated patches accordingly. -- Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 21:30:03 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 17C5F583; Tue, 15 Oct 2013 21:30:03 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qe0-x22e.google.com (mail-qe0-x22e.google.com [IPv6:2607:f8b0:400d:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BB45229F9; Tue, 15 Oct 2013 21:30:02 +0000 (UTC) Received: by mail-qe0-f46.google.com with SMTP id s14so5519079qeb.19 for ; Tue, 15 Oct 2013 14:30:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=oajEWTLWHZyXzdrgKFk4e22BvHGDuN6G7Z1bkU/abOg=; b=e/b52L06vXtzVAOT1/yGzJEP+aEgRAqN5jiCEiJyZla5ZhMzd5QoKGKRVW7vyxQ8tA U2iYR38F28GQebv8/6Px5QLc+2tiB1q65jFwbUWueKh3vV76Ki7TD6SIsmbUhe43IcJQ yQZtkeI79t/oMFKC/HgTFvpA/7aZekjpZR3pFmhn+bCDScvUZyySULATQvjHTSFNhQNE c2kEaS3r/gyWngIa1lsiHWqT1UgubAZlGpdRFaj6Fe35j1yfc2GABwXAxeCNS56m4xPJ Qb1QKzNEWDHLAIC6heWpA6MxyhH5SZT/TX0ZzTGfxRWO8fZlohBvRudEASrd97XoRVTK tZSw== MIME-Version: 1.0 X-Received: by 10.224.147.84 with SMTP id k20mr6043257qav.99.1381872601886; Tue, 15 Oct 2013 14:30:01 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.49.39.97 with HTTP; Tue, 15 Oct 2013 14:30:01 -0700 (PDT) In-Reply-To: References: <4CE11789-6889-438B-89DF-CEBE4ACEC841@gmail.com> Date: Tue, 15 Oct 2013 15:30:01 -0600 X-Google-Sender-Auth: dV-stqqVyIbUJ73nIV0z7Yld8ro Message-ID: Subject: Re: Refactoring of *.test.mk From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Rui Paulo , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 21:30:03 -0000 On Tue, Oct 15, 2013 at 3:11 PM, Julio Merino wrote: > On Tue, Oct 15, 2013 at 4:50 PM, Alan Somers wrote: >> On Tue, Oct 15, 2013 at 10:25 AM, Garrett Cooper wrote: >>> On Oct 14, 2013, at 20:24, Julio Merino wrote: >>>> >>>> [Fixing Garrett's address...] >>> >>> (Sorry for the empty reply. Still getting used to iOS 7..) >>> >>> I'll look at this sometime before the end of the week. I have some minor concerns about the proposed layout and how things will integrate in, but these should be easy to resolve. >>> >>> Also, did you look at my code on GitHub under share/mk? It's quite a bit different from the stuff in FreeBSD head.. >> >> I'm currently trying to merge Garrett's stuff from GitHub. So far, I >> only see one thing that I would like to change. It would be really >> nice to be able to override the test_suite variable. > > Good idea. Updated patches accordingly. Suite^H^H^H^Hweet. > > -- > Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Sat Oct 19 15:49:43 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5694BD8F for ; Sat, 19 Oct 2013 15:49:43 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-la0-f51.google.com (mail-la0-f51.google.com [209.85.215.51]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DA71820E6 for ; Sat, 19 Oct 2013 15:49:42 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id ea20so294785lab.38 for ; Sat, 19 Oct 2013 08:49:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-type; bh=22Nl+oX7EiyM9ebe9/pOPZ62pYSfCqT9lYOzp2hTHUI=; b=dMmqYUITNbjhYfuQ8TC111nwOMJbw2FVZmHhMoR1aZ/BLa/aEdrf6d2MaXS0PYhOm1 ppiY5BcaCki0gCaCPgZSVK4p1qs4cJgTc3ShN1WLH4F2MfMEmPAZg75oHvrRRDgyOJkM X1pm8CW+e739P7J65E2h5wbXICU4EvIMLl8gSXhwlM9EqRO5JI2uHhAnt01+zJuZXXHy QgmqW/Tub48LXrO7q1v13G7ulNVnkFl0n/Iodi6vm+A1MeUgG1hmLOtpSFhNZh86JSRu Kdg1Q4liMka/7ypIqRvg1LkrowLwH+RLPLko8GV7Qsf+QxPtl2OxtSk2mFJ1XhwrqRj/ h3vg== X-Gm-Message-State: ALoCoQn4myIuq3mJfZm+2EVzdRqQEedZwucEgLZc+hsl0def1gHxK2sHzr4/oLst14CUyG40h+W3 X-Received: by 10.152.28.7 with SMTP id x7mr5936984lag.26.1382197775448; Sat, 19 Oct 2013 08:49:35 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.132.135 with HTTP; Sat, 19 Oct 2013 08:49:15 -0700 (PDT) X-Originating-IP: [172.26.105.74] From: Julio Merino Date: Sat, 19 Oct 2013 11:49:15 -0400 Message-ID: Subject: Populating /usr/tests To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Rui Paulo , "yaneurabeya ." , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2013 15:49:43 -0000 Hello, [ CCing same interested parties as before ] Here comes the next round of patches. This time the goal is to start populating /usr/tests (which already exists) with the only test program currently available in the tree: src/lib/libcrypt/tests/*. With this boilerplate in place, adding new test programs to the base system should be straightforward (which is the goal here!). I'll be happy to provide a tutorial later if we agree to this scheme, similar to the one we have for NetBSD (http://wiki.netbsd.org/tutorials/atf/), detailing the procedure to add both atf-based and "old-style" test programs. The README file added in add-tests-hierarchy.diff and the tests.7 page added in add-tests-7.diff might answer some of the questions you might have. Not repasting their contents here. As before, the new set of patches is in http://portal.meroh.net/~jmmv/freebsd-testing/ . The series file in it lists the order in which they need to be read/applied and each patch has a header usable as the commit log. I have successfully tested this both in amd64 and powerpc with update and clean builds/installs. However, please review carefully. Thoughts? Thanks, -- Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Sat Oct 19 15:51:27 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3F7F8DA2 for ; Sat, 19 Oct 2013 15:51:27 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-la0-f48.google.com (mail-la0-f48.google.com [209.85.215.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C10EA2114 for ; Sat, 19 Oct 2013 15:51:26 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id er20so1727000lab.21 for ; Sat, 19 Oct 2013 08:51:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=zwPY3knTvFe5EpiQvr7138YzNp4x8Nfhr4LhFVlnvWA=; b=A2AK6n4rc6JFFGOwTixrOjmtAZV03GbTwYiR/UfKOa/1Enouthd+tEcABEuoXCq8o4 07uJIiV9EUU4Q9cDujE6r2yP1zmVBbUHkcb7eBz9YYW5Q4+V2oOpfOBOQTuFUt2/XNzV Z0CtVsNzfvwjiRZW5F134jXe3v1ni4FFXuqhU4jjPfYeC5WsKodRIgMLsebMINIdu8ur VsJUrVkYP1/yuU8aYoMs3DVjPfcwEy/cgcAMDPAmG26GP/zMl7pxStMGqA6fYyE6VfIP 3wyCButpzv9pFJNQ96+JS8WYuVyH8PmfiWQa4OQeTPG+BOjZiyo6IeLEa5rWCXnPKTmb kAJg== X-Gm-Message-State: ALoCoQnLXSY/Hd4Soq0U0DHCCNZ6OYgCorqarJU6hTNgTuBOfwB3dUnKmnvi1XbR74mFVkMOGDeb X-Received: by 10.112.51.166 with SMTP id l6mr6536168lbo.5.1382197879329; Sat, 19 Oct 2013 08:51:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.132.135 with HTTP; Sat, 19 Oct 2013 08:50:59 -0700 (PDT) X-Originating-IP: [172.26.105.74] In-Reply-To: References: From: Julio Merino Date: Sat, 19 Oct 2013 11:50:59 -0400 Message-ID: Subject: Re: Populating /usr/tests To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Rui Paulo , "yaneurabeya ." , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2013 15:51:27 -0000 [ I'll not learn this... fixing Garret's address. ] On Sat, Oct 19, 2013 at 11:49 AM, Julio Merino wrote: > Hello, > > [ CCing same interested parties as before ] > > Here comes the next round of patches. This time the goal is to start > populating /usr/tests (which already exists) with the only test > program currently available in the tree: src/lib/libcrypt/tests/*. > > With this boilerplate in place, adding new test programs to the base > system should be straightforward (which is the goal here!). I'll be > happy to provide a tutorial later if we agree to this scheme, similar > to the one we have for NetBSD (http://wiki.netbsd.org/tutorials/atf/), > detailing the procedure to add both atf-based and "old-style" test > programs. > > The README file added in add-tests-hierarchy.diff and the tests.7 page > added in add-tests-7.diff might answer some of the questions you might > have. Not repasting their contents here. > > As before, the new set of patches is in > http://portal.meroh.net/~jmmv/freebsd-testing/ . The series file in > it lists the order in which they need to be read/applied and each > patch has a header usable as the commit log. > > I have successfully tested this both in amd64 and powerpc with update > and clean builds/installs. However, please review carefully. > > Thoughts? > > Thanks, > > -- > Julio Merino / @jmmv -- Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Sat Oct 19 16:41:32 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3FD7FACD for ; Sat, 19 Oct 2013 16:41:32 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe004.messaging.microsoft.com [216.32.180.14]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ECBFA23CA for ; Sat, 19 Oct 2013 16:41:31 +0000 (UTC) Received: from mail146-va3-R.bigfish.com (10.7.14.246) by VA3EHSOBE008.bigfish.com (10.7.40.28) with Microsoft SMTP Server id 14.1.225.22; Sat, 19 Oct 2013 16:41:24 +0000 Received: from mail146-va3 (localhost [127.0.0.1]) by mail146-va3-R.bigfish.com (Postfix) with ESMTP id 19FFA4E00A0; Sat, 19 Oct 2013 16:41:24 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.224.50; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF01-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 3 X-BigFish: VPS3(zzzz1f42h208ch1ee6h1de0h1fdah2073h1202h1e76h1d1ah1d2ah1fc6h1082kzz8275dh1de097hz2fh2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h1155h) Received-SPF: pass (mail146-va3: domain of juniper.net designates 66.129.224.50 as permitted sender) client-ip=66.129.224.50; envelope-from=sjg@juniper.net; helo=P-EMF01-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail146-va3 (localhost.localdomain [127.0.0.1]) by mail146-va3 (MessageSwitch) id 1382200882469266_18396; Sat, 19 Oct 2013 16:41:22 +0000 (UTC) Received: from VA3EHSMHS020.bigfish.com (unknown [10.7.14.246]) by mail146-va3.bigfish.com (Postfix) with ESMTP id 6E0E3C003F; Sat, 19 Oct 2013 16:41:22 +0000 (UTC) Received: from P-EMF01-SAC.jnpr.net (66.129.224.50) by VA3EHSMHS020.bigfish.com (10.7.99.30) with Microsoft SMTP Server (TLS) id 14.16.227.3; Sat, 19 Oct 2013 16:41:21 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Sat, 19 Oct 2013 09:41:19 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id r9JGfIL79211; Sat, 19 Oct 2013 09:41:19 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id A06E25807E; Sat, 19 Oct 2013 09:41:18 -0700 (PDT) To: Julio Merino Subject: Re: Populating /usr/tests In-Reply-To: References: Comments: In-reply-to: Julio Merino message dated "Sat, 19 Oct 2013 11:50:59 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Sat, 19 Oct 2013 09:41:18 -0700 Message-ID: <20131019164118.A06E25807E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: freebsd-testing@freebsd.org, Rui Paulo , "yaneurabeya ." , sjg@juniper.net X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2013 16:41:32 -0000 Thanks Julio, looks good. I don't see anything that would cause us grief. Internally we are using the meta mode build (svn.freebsd.org/base/projects/bmake) so no tree walks, multiple passes etc. We have all the tests from netbsd building with the tests located with the relevant apps/libs as in the libcrypt case. We've not yet finished fixing them all to pass on freebsd though. From owner-freebsd-testing@FreeBSD.ORG Sat Oct 19 20:49:42 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 15FBE450 for ; Sat, 19 Oct 2013 20:49:42 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E10F22FE8 for ; Sat, 19 Oct 2013 20:49:41 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id p10so3717780pdj.39 for ; Sat, 19 Oct 2013 13:49:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=SGoQwmf7qlW9iB2hlBbnIdHVrr73e9Ht0Wc7tdKZLko=; b=RluOZ17NRu66rU0MB/XtpS8m1Ma6z7x39GaiXow/ZqXJh7jBVAYD7QkLQXQe01oZ+j fZyz5IisoEXYVZbY0/zm+2CnfdTFbCCwkVS7GOxaotYcJoQ8pfdv1WIu0E9w8YbO7wXm oACpw+0ic8dLuMeQdkUuDJ5cGgZu6b5fC1qoCk5/1Jg18GbframDKm6k1Zl5BQKONyYJ bND+waSPFurj9b6zGq8GIMqkqpfGf44oGl7P5hKolHdlNqBIm7/rx2Na3CjCXPsCtFq1 G1vQMhZfRO9WIOUCMBo1hHLD210ARS6IFf6ktwtTaCphLW5kIASeEIF6z2E6ysOf/s9C zHyA== X-Received: by 10.66.217.166 with SMTP id oz6mr10161009pac.22.1382215781512; Sat, 19 Oct 2013 13:49:41 -0700 (PDT) Received: from [10.3.73.48] (mobile-166-147-083-033.mycingular.net. [166.147.83.33]) by mx.google.com with ESMTPSA id wd6sm13883535pab.3.2013.10.19.13.49.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 19 Oct 2013 13:49:40 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <6DC6C1E8-7AEF-439E-9250-D223A56EF432@gmail.com> X-Mailer: iPhone Mail (11A501) From: Garrett Cooper Subject: Re: Populating /usr/tests Date: Sat, 19 Oct 2013 13:49:36 -0700 To: Julio Merino Cc: "freebsd-testing@freebsd.org" , Rui Paulo , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2013 20:49:42 -0000 You're missing a lot of code in etc/, some in Makefile.inc1, and other piece= s in share/mk as well with bsd.own.mk. Also, your approach with tests/Makefi= le is ok, but it doesn't partition well with prove.test.mk on my branch, etc= . Thanks, -Garrett PS No worries about the email addresses; I still use yanegomi, but mainly as= a spam catch address nowadays. > On Oct 19, 2013, at 8:49, Julio Merino wrote: >=20 > Hello, >=20 > [ CCing same interested parties as before ] >=20 > Here comes the next round of patches. This time the goal is to start > populating /usr/tests (which already exists) with the only test > program currently available in the tree: src/lib/libcrypt/tests/*. >=20 > With this boilerplate in place, adding new test programs to the base > system should be straightforward (which is the goal here!). I'll be > happy to provide a tutorial later if we agree to this scheme, similar > to the one we have for NetBSD (http://wiki.netbsd.org/tutorials/atf/), > detailing the procedure to add both atf-based and "old-style" test > programs. >=20 > The README file added in add-tests-hierarchy.diff and the tests.7 page > added in add-tests-7.diff might answer some of the questions you might > have. Not repasting their contents here. >=20 > As before, the new set of patches is in > http://portal.meroh.net/~jmmv/freebsd-testing/ . The series file in > it lists the order in which they need to be read/applied and each > patch has a header usable as the commit log. >=20 > I have successfully tested this both in amd64 and powerpc with update > and clean builds/installs. However, please review carefully. >=20 > Thoughts? >=20 > Thanks, >=20 > --=20 > Julio Merino / @jmmv > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org"=