From owner-freebsd-testing@freebsd.org Mon Aug 6 19:14:14 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D2B41062797 for ; Mon, 6 Aug 2018 19:14:14 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-pl0-x235.google.com (mail-pl0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 87073853DE; Mon, 6 Aug 2018 19:14:13 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-pl0-x235.google.com with SMTP id x6-v6so6031236plv.10; Mon, 06 Aug 2018 12:14:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=ipfLJ315KLYWNHdhYIs/giaDLf6dm2TjkiQ9kE5wXCk=; b=p2Tuw2A4Me/D2EtpnGRE+b/Av8DvZlilifkkY2HcpW/hZ9Fih7cgAooVaLezU0XKcs zwbxGnEZsG5v9RSVB4E+Ml9nsYzdl7DcTieZr8e9R6t++e5l7ivDyAul3Y1ZDXnbfoE7 Z4L0Z3Jj3i5zjPveCLr+iVAdkF/+YhJJyH39KOu4qC54+Q9CWMyqeqtevy24X0mmo0AS owfow6JZ7jIM0DDic7tmZbSMSVQeBNzm02MybOgRIUYHfcgLMIdRHrUFvuII3PTdGhK6 TB4roX6Guq+TecVhZ395GOQIA/MLbeMaLAWfyltLTqdB5Jig+O/TYM2ny+VJ3tVJLEKL 6tkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=ipfLJ315KLYWNHdhYIs/giaDLf6dm2TjkiQ9kE5wXCk=; b=qBrHHpPNzyEDIha7+m2r7ty0+p6CozngMDO0pCigVRLcbFQdLTUYpL/fW6daUjXW8t 114R7M98LTrc7l2CfCxQlyJmGic5s1d/q0BKzlyh7sKDRaBZjrpoxgYJCeVIWY4683/C O+fbEQv5GZxYTvVDTRxnrGqQDA60IRaHhDzsu/o5XyAx9FFCrn6esIP6hRT5kCaq0tIb xAVH1BVeXNp2x7sdMPUZG1it/JWUppRXjXHWomU4ot7EbVF1g/XSYrDVttUwUrV7byX8 tcillv6Z+49oE35+PN9CAtUGdEenCItTWkm/iR/OgsZze8kNokASfU/8JhY2WV6EmvDX YC6g== X-Gm-Message-State: AOUpUlEjDHqowfaVjkixMetX40ivQWJlHj0p6Qp3AxIRLrctLd0/p+9e ddBrr0BzfIgZZd3McZRXzvnL7yZG X-Google-Smtp-Source: AAOMgpfmV5ysQJEuO/ODmpgBtzqxXQlxqKQ8EGUkO1TaMm470hKehpckOowudQaK0e4VnOPjLE30pQ== X-Received: by 2002:a17:902:aa8f:: with SMTP id d15-v6mr15165631plr.64.1533582852230; Mon, 06 Aug 2018 12:14:12 -0700 (PDT) Received: from raichu (toroon0560w-lp130-09-70-52-224-239.dsl.bell.ca. [70.52.224.239]) by smtp.gmail.com with ESMTPSA id y63-v6sm20746846pgd.94.2018.08.06.12.14.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Aug 2018 12:14:11 -0700 (PDT) Sender: Mark Johnston Date: Mon, 6 Aug 2018 15:14:06 -0400 From: Mark Johnston To: Alan Somers Cc: "freebsd-testing@freebsd.org" Subject: Re: Skipping tests that are unimplemented in 32-bit emulation Message-ID: <20180806191406.GA77150@raichu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2018 19:14:14 -0000 On Sun, Jul 29, 2018 at 11:23:33AM -0600, Alan Somers wrote: > I recently tried running the i386 test suite in a chroot on an amd64 > system. 162 tests failed, and 33 were broken. Some of the failures were > due to system calls that haven't been implemented in 32-bit emulation. > setfib(2) is an example. I think it's unlikely that anybody will ever need > 32-bit emulation for setfib(2), so perhaps we should just skip the test? > What's the best way to do that? I can come up with two ways: > > 1) At runtime, check the hw.machine sysctl and see if it matches some > compile time preprocessor constant. I don't know what constant to use, > though. Checking __amd64__ would only work for i386 binaries on amd64 > kernels, and not something else like mips binaries on mips64 kernels (I > don't know if we support that, but I don't want to rule it out). > > 2) At buildtime, put an "allowed_architectures=i386" metadata property into > the Kyuafile for that test program. This would require support in > /usr/share/mk/bsd.test.mk. It would also require patching Kyua itself, > because currently "Kyua config" returns the architecture for which it was > built, not the one on which it's running. > > Thoughts? I don't have any particular suggestions, but I'd personally rather avoid a solution that requires tests to opt-in to running under 32-bit emulation, which I think excludes 2). I'd be happy to help annotate any failing tests as required. It bugs me that the test suite currently doesn't cover such relatively complicated functions as freebsd32_copy_msg_out(). From owner-freebsd-testing@freebsd.org Mon Aug 6 20:18:38 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2BB71064364 for ; Mon, 6 Aug 2018 20:18:37 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com [IPv6:2a00:1450:4864:20::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5081389479; Mon, 6 Aug 2018 20:18:37 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-x235.google.com with SMTP id r13-v6so11603286ljg.10; Mon, 06 Aug 2018 13:18:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=5Qh4wpbGdACigbSquMVhk4om3HqzjY8vnNN61W2sNA0=; b=IIrWrNOvGCodQyqynElESy3A+D2Yf4cVYSskoSuCjyBSVyhG2V8mnv3Bpr9j1rSxRO mgvGVlUFA+nFgCY7wAviKX3FGs85eg2QygXrKUiDK2S+DtwwwKaydmRA1D076TtS1Xxy cVZduuO3EazeuDFcGbOoqWLGaC8LUb/OG0he8rYGZnSw1NS/aOgJaGFoeDsyRRpuZ8q5 qryyWndwWjIS/hHCzLxBfJtmKl8hxZxvaZlfyTA+Q+JCRY6dxt0JHFFLbgVzrdsv7kD0 viLQACo2fj16l12RtyBfc3HZ97vS/hJVn21/ffZbvADDBTDobGfzbo2d6yVwsFQ7r74W lFvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=5Qh4wpbGdACigbSquMVhk4om3HqzjY8vnNN61W2sNA0=; b=erJA07FY7ECeWR0KyiGBfZq9JhTYhuMcUYIjt4blaU1jEh80olKMuYOXX1Od/J15kU i8uTKg5UI9631tWIOjdl70j0EMQrLUeentmDB8tbEn9lF9ICgzcc7GBtS8KJvelN7655 YPF8T7rdhOTjQaxZnJfS+OVE2TmwSxA1KE9FfbOmUu+YcuuwH/Ca2ubRcaFVzYycNf72 9VOlO4skm2DB5+J0134cG+SvvVa6EzqzlIIlSRfVyNY7KerC53gtEyugtwBkulptQfA+ j8U0FgulT4Lye0P0YtPtoJckLZQfLRmktvX8UkKowo776NV59b1H5AwTUnpY2rJP+Maf 3Yrg== X-Gm-Message-State: AOUpUlGdnK0EJtM2ZJvr2rtjO95rB7PgnFTlPhKFNeasT4/yuW/TktWj DAHnQeqL2LJh7eZpmQChLbyZhJ6J6uVbXgrC29/whw== X-Google-Smtp-Source: AAOMgpegurR5t/reEb4vtdKB3Pi1QjUJ6NOKE6GOM6bbcWHtU3UceFfMVIa7oo54ACm14p+Xy4Od//c+8gFSB8YWs1s= X-Received: by 2002:a2e:87da:: with SMTP id v26-v6mr13252935ljj.69.1533586715631; Mon, 06 Aug 2018 13:18:35 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:7851:0:0:0:0:0 with HTTP; Mon, 6 Aug 2018 13:18:35 -0700 (PDT) In-Reply-To: <20180806191406.GA77150@raichu> References: <20180806191406.GA77150@raichu> From: Alan Somers Date: Mon, 6 Aug 2018 14:18:35 -0600 X-Google-Sender-Auth: -whSdT8MAp_594-UiTXQv7eMDtI Message-ID: Subject: Re: Skipping tests that are unimplemented in 32-bit emulation To: Mark Johnston Cc: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Aug 2018 20:18:38 -0000 On Mon, Aug 6, 2018 at 1:14 PM, Mark Johnston wrote: > On Sun, Jul 29, 2018 at 11:23:33AM -0600, Alan Somers wrote: > > I recently tried running the i386 test suite in a chroot on an amd64 > > system. 162 tests failed, and 33 were broken. Some of the failures were > > due to system calls that haven't been implemented in 32-bit emulation. > > setfib(2) is an example. I think it's unlikely that anybody will ever > need > > 32-bit emulation for setfib(2), so perhaps we should just skip the test? > > What's the best way to do that? I can come up with two ways: > > > > 1) At runtime, check the hw.machine sysctl and see if it matches some > > compile time preprocessor constant. I don't know what constant to use, > > though. Checking __amd64__ would only work for i386 binaries on amd64 > > kernels, and not something else like mips binaries on mips64 kernels (I > > don't know if we support that, but I don't want to rule it out). > > > > 2) At buildtime, put an "allowed_architectures=i386" metadata property > into > > the Kyuafile for that test program. This would require support in > > /usr/share/mk/bsd.test.mk. It would also require patching Kyua itself, > > because currently "Kyua config" returns the architecture for which it was > > built, not the one on which it's running. > > > > Thoughts? > > I don't have any particular suggestions, but I'd personally rather avoid > a solution that requires tests to opt-in to running under 32-bit > emulation, which I think excludes 2). I'd be happy to help annotate > any failing tests as required. It bugs me that the test suite currently > doesn't cover such relatively complicated functions as > freebsd32_copy_msg_out(). > I don't think that 2 would necessarily be opt-in, because an undefined value for allowed_architectures is interpreted as meaning "all". It could be opt-out instead. But it could still be a little awkward. Option 1 could be accomplished for atf-c testcases by comparing the value of __LP64__ to a hardcoded list of known 64-bit processors as returned by uname(3). But I don't know how to implement 1 for atf-sh programs. An atf-sh program would need to know the architecture of any binary that it might invoke. Is there anything in /etc indicating what architecture the image was built for? Should we just use "file /lib/libc.so.*"? From owner-freebsd-testing@freebsd.org Tue Aug 7 15:03:55 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C6581061778 for ; Tue, 7 Aug 2018 15:03:55 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B94173563; Tue, 7 Aug 2018 15:03:55 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-pg1-x52b.google.com with SMTP id n7-v6so7982142pgq.4; Tue, 07 Aug 2018 08:03:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=0NgdCZdDIPgbFzQ5GxEi9usqxrm1i8SgUeWC/hJowr0=; b=fpgGJoqNrH+6vZ+UxFo+4D8wyiRgRZATEEkKAkMsdTr0CN/ZvWyi6UIWWKdTTjy4CH 7yuyoV7DESR2oEyNTKL43DrDvvhQ4PSvoecDsJom1w9AJTrV6UnwMvmBaPaF9zJGZDaV F9RM2fh2dU0ht2s5FnVtLa7x96vJUEOWeYawl8SbWmSnCB8I41TV6mD/IoTXoJle5pEW 0rPC2DOulfgudz/g8Gg3uRd/H9dJvy2Oho04HmAfpa0C3YZkdLoWtpPOubhKZSCTHONS hEPU+leDUWDe6zYuNgmdMn+G9y1gCCSmxg+3EQXPJJDVKAilN9ECNbGH6rwmZ26Wo+0z dvWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=0NgdCZdDIPgbFzQ5GxEi9usqxrm1i8SgUeWC/hJowr0=; b=OAWws5jtFoIilNIW7S/gMpGl1r1MPcsJsNHAUHhvuuyCPVJvmvEl0zDEBYFrjG3ijQ BljbpZ2C1NeToiWnD1AQb3rtsB7wwKycly0Bay7KFBeO/8Tyk2JPv7At+xESBA77F3u5 evSXVOzAb9cAnV2ZUwmWa1JFsPqrPEU3ghv2IHkAkBJbl/8+eZNSKmNAuIZpYsTIMJwR /YQeb/+rUFJD7POC4CbKGnTI5tWOMxfS6A5w2J3s/5Wsu4jKuaaGsHfQfq00eINMx2HS WH6/nWeJfUDUBJz0DDk4i75Mbxe8Fn0BGl9+MRSjZ8aFG9Ion9gKJj6O9+QNtjZkz6z4 v2Ag== X-Gm-Message-State: AOUpUlFyE/nm+cPffdt6dbP+iNiQbzYj33wGQFSVljTS18Hc8LLoIPpm D32R5Qb8kDJMPcyRLITFt+tzpWco X-Google-Smtp-Source: AAOMgpdJFS++Typw+UYggUHg+mI5SADFWIePApufWa4kdTqY+8e4zp5WHgP49Z65rbLx3VEvWM4VNw== X-Received: by 2002:a63:4b5a:: with SMTP id k26-v6mr18569182pgl.384.1533654232995; Tue, 07 Aug 2018 08:03:52 -0700 (PDT) Received: from raichu (toroon0560w-lp130-09-70-52-224-239.dsl.bell.ca. [70.52.224.239]) by smtp.gmail.com with ESMTPSA id a20-v6sm6122577pfc.14.2018.08.07.08.03.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 08:03:52 -0700 (PDT) Sender: Mark Johnston Date: Tue, 7 Aug 2018 11:03:49 -0400 From: Mark Johnston To: Alan Somers Cc: "freebsd-testing@freebsd.org" Subject: Re: Skipping tests that are unimplemented in 32-bit emulation Message-ID: <20180807150349.GF77150@raichu> References: <20180806191406.GA77150@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 15:03:55 -0000 On Mon, Aug 06, 2018 at 02:18:35PM -0600, Alan Somers wrote: > On Mon, Aug 6, 2018 at 1:14 PM, Mark Johnston wrote: > > > On Sun, Jul 29, 2018 at 11:23:33AM -0600, Alan Somers wrote: > > > I recently tried running the i386 test suite in a chroot on an amd64 > > > system. 162 tests failed, and 33 were broken. Some of the failures were > > > due to system calls that haven't been implemented in 32-bit emulation. > > > setfib(2) is an example. I think it's unlikely that anybody will ever > > need > > > 32-bit emulation for setfib(2), so perhaps we should just skip the test? > > > What's the best way to do that? I can come up with two ways: > > > > > > 1) At runtime, check the hw.machine sysctl and see if it matches some > > > compile time preprocessor constant. I don't know what constant to use, > > > though. Checking __amd64__ would only work for i386 binaries on amd64 > > > kernels, and not something else like mips binaries on mips64 kernels (I > > > don't know if we support that, but I don't want to rule it out). > > > > > > 2) At buildtime, put an "allowed_architectures=i386" metadata property > > into > > > the Kyuafile for that test program. This would require support in > > > /usr/share/mk/bsd.test.mk. It would also require patching Kyua itself, > > > because currently "Kyua config" returns the architecture for which it was > > > built, not the one on which it's running. > > > > > > Thoughts? > > > > I don't have any particular suggestions, but I'd personally rather avoid > > a solution that requires tests to opt-in to running under 32-bit > > emulation, which I think excludes 2). I'd be happy to help annotate > > any failing tests as required. It bugs me that the test suite currently > > doesn't cover such relatively complicated functions as > > freebsd32_copy_msg_out(). > > I don't think that 2 would necessarily be opt-in, because an undefined > value for allowed_architectures is interpreted as meaning "all". It could > be opt-out instead. But it could still be a little awkward. Option 1 > could be accomplished for atf-c testcases by comparing the value of > __LP64__ to a hardcoded list of known 64-bit processors as returned by > uname(3). But I don't know how to implement 1 for atf-sh programs. An > atf-sh program would need to know the architecture of any binary that it > might invoke. Is there anything in /etc indicating what architecture the > image was built for? Should we just use "file /lib/libc.so.*"? Could we instead build and install a /usr/tests32 suite on systems that can support it, and use a top-level Kyuafile and kyua.conf that overrides the "architecture" variable? From owner-freebsd-testing@freebsd.org Tue Aug 7 15:08:42 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38EFD1061A2F for ; Tue, 7 Aug 2018 15:08:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com [IPv6:2a00:1450:4864:20::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F283738D9; Tue, 7 Aug 2018 15:08:41 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-x231.google.com with SMTP id p6-v6so13739216ljc.5; Tue, 07 Aug 2018 08:08:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=MK0/1VM7FC2VD1cc2a2tnKsYHERN4RIuWhOe33Mt31Y=; b=QMoaZ1ag/vC1j3UDMJyOycv9iDx/vQIZz7UV+RpnMNjgRGxW+ZvkVo0/HJimzrMMpk gEBrSCe9L5/kvH4bH7hLRcCRUV4SIYcY/fRuJ9FBAxWxEvGoje/rGhJxDM0EoIETddgF AUbLFDd6w3NVJ2EuhPM0DG4EuRBb73DVCnKN+OyG8Sxdu5NOJpDGS53+XNAw2LX07/OZ agdZFoCi1jhRSppG7OxzXuLGx+x0ahAz2/EdaJOFhDVDr+ohM8VuDLLZ/LE1QKaH/Us/ fJ1rfkdl5iEAArC/UCVu8adR2tdkdAj01yJPAgKrVAc3bytv12oeadbM7Tfjli8178fF rqzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=MK0/1VM7FC2VD1cc2a2tnKsYHERN4RIuWhOe33Mt31Y=; b=oxhtOzg2XU7wKr4CNlIhXKP48xYlS1pBF3FMWQ3dIy94B+XQwgmJDtIJa5VNNVPmad 8eTelWn+hhffDLjPspPFLqmAGm2Gff3seDPMDI3LL9xILcz8gfR6UiNXIUJb9kHkhEc1 wrSSOT5Kx5Ktvf2GhvDM/ApXervJ3jwLysDzYp+fPdlYm/wpNi5jWrbAhDeW56YAJRNB LqLxWaceiArWxeJb24+ju916QVNssxkoMGTlxLwHX3QnGBkAlgAtjlDqACpCzqAhXxbP LgTlWAsnZ349HGFoaDS2SY6x8RoBVpuS2qyAFdhxhfK5xvzVf0gA10kF37MaZa45Dtgc a1WQ== X-Gm-Message-State: AOUpUlEozRCfMUkuDLfxwC/3SnIn70Gz+A2A31bHAWd9c6ViQ90e9QBT fG1ddqxUfDgPwYFK42eOs1JPZAg/fI1pwSpYJHT8mg== X-Google-Smtp-Source: AAOMgpdcqO2fvIHkdAWTkap+c58yfMt2oRNPmDoGMODfL9xxWCZtbrqolswR4RRwcO5zTHa6ncafJ1E9Db6nDpBp3DY= X-Received: by 2002:a2e:4401:: with SMTP id r1-v6mr17020189lja.21.1533654519723; Tue, 07 Aug 2018 08:08:39 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:7851:0:0:0:0:0 with HTTP; Tue, 7 Aug 2018 08:08:39 -0700 (PDT) In-Reply-To: <20180807150349.GF77150@raichu> References: <20180806191406.GA77150@raichu> <20180807150349.GF77150@raichu> From: Alan Somers Date: Tue, 7 Aug 2018 09:08:39 -0600 X-Google-Sender-Auth: 4lFQR0DqG1jKjjOOMiDPAghg5lU Message-ID: Subject: Re: Skipping tests that are unimplemented in 32-bit emulation To: Mark Johnston Cc: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 15:08:42 -0000 On Tue, Aug 7, 2018 at 9:03 AM, Mark Johnston wrote: > On Mon, Aug 06, 2018 at 02:18:35PM -0600, Alan Somers wrote: > > On Mon, Aug 6, 2018 at 1:14 PM, Mark Johnston wrote: > > > > > On Sun, Jul 29, 2018 at 11:23:33AM -0600, Alan Somers wrote: > > > > I recently tried running the i386 test suite in a chroot on an amd64 > > > > system. 162 tests failed, and 33 were broken. Some of the failures > were > > > > due to system calls that haven't been implemented in 32-bit > emulation. > > > > setfib(2) is an example. I think it's unlikely that anybody will > ever > > > need > > > > 32-bit emulation for setfib(2), so perhaps we should just skip the > test? > > > > What's the best way to do that? I can come up with two ways: > > > > > > > > 1) At runtime, check the hw.machine sysctl and see if it matches some > > > > compile time preprocessor constant. I don't know what constant to > use, > > > > though. Checking __amd64__ would only work for i386 binaries on > amd64 > > > > kernels, and not something else like mips binaries on mips64 kernels > (I > > > > don't know if we support that, but I don't want to rule it out). > > > > > > > > 2) At buildtime, put an "allowed_architectures=i386" metadata > property > > > into > > > > the Kyuafile for that test program. This would require support in > > > > /usr/share/mk/bsd.test.mk. It would also require patching Kyua > itself, > > > > because currently "Kyua config" returns the architecture for which > it was > > > > built, not the one on which it's running. > > > > > > > > Thoughts? > > > > > > I don't have any particular suggestions, but I'd personally rather > avoid > > > a solution that requires tests to opt-in to running under 32-bit > > > emulation, which I think excludes 2). I'd be happy to help annotate > > > any failing tests as required. It bugs me that the test suite > currently > > > doesn't cover such relatively complicated functions as > > > freebsd32_copy_msg_out(). > > > > I don't think that 2 would necessarily be opt-in, because an undefined > > value for allowed_architectures is interpreted as meaning "all". It > could > > be opt-out instead. But it could still be a little awkward. Option 1 > > could be accomplished for atf-c testcases by comparing the value of > > __LP64__ to a hardcoded list of known 64-bit processors as returned by > > uname(3). But I don't know how to implement 1 for atf-sh programs. An > > atf-sh program would need to know the architecture of any binary that it > > might invoke. Is there anything in /etc indicating what architecture the > > image was built for? Should we just use "file /lib/libc.so.*"? > > Could we instead build and install a /usr/tests32 suite on systems that > can support it, and use a top-level Kyuafile and kyua.conf that overrides > the "architecture" variable? > It's not enough to build and install the atf-c tests in 32-bit mode. We would need an entire 32-bit chroot in order to run the atf-sh tests in 32-bit mode. From owner-freebsd-testing@freebsd.org Tue Aug 7 15:24:08 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA7201062002 for ; Tue, 7 Aug 2018 15:24:07 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6647C741B1; Tue, 7 Aug 2018 15:24:07 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-pg1-x52b.google.com with SMTP id y5-v6so8018312pgv.1; Tue, 07 Aug 2018 08:24:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=30/n1eWTxYjKoFZ2NHURqhKa+Gy24I8QZcUJoV5dRFE=; b=AxHj0jW3Tr1ztyq/hhOtq8dST03IMKVgtPw3gQajfwUT2ufgc1/+ID/Rn2p/MDHPAz 1DnVlMFNvZxByEFmTfTaP/FD4H+5yn7lziYNixDSCrwz/9NjO0G1lC0n91F+m7rFKfd4 OP+vjzBbfok22DTRlxk9IPqx219hTKN0q6WRGYSkVY4Kfgs85tOOEK/J6Wx1cgNdGuZp 2edAb3pDKEtr2XLAoSp07birADZigCEAsSSTx/HTW5Ry4hAqNfO1TBgSpRaK4BomRFJ7 hLdqJdG3DTFWmOpDccF90pzxDU1afEOBLT6tHe4qVegK+pHa7bGj4sEfLAR5erun6IhY SHNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=30/n1eWTxYjKoFZ2NHURqhKa+Gy24I8QZcUJoV5dRFE=; b=ilsT5IBhVtRkSBmZLe9sVx1JL2vF6PlclWe+g2t4t4M2BWzoJvjBDSktsRqf9UwKx4 vsZg3/n1xonEB1ZR+M4mMmhFUuJuNZPBpf5cPDF7V7YGBoGSiZa6xXIyvn3qGJJeMfp+ vAdvIB8Z8QYboGRQBd3VN0LAfvGsMI6MhAu9O1DZybqJmue8PKiFX2d2lbXfM861N/DP iNUzM2PmrfhVKfYeI1s9j/HRPxYRS/Wm9brOdquC25voqr9d7WZyA0/iqMz32lgz82US agHK0VWVR2BoWS9Rh6tiEQxS8WAZBpauRsNNUU0VOXmVmYlBtS1cMIyv/tpGKAUwETgZ 1Ktg== X-Gm-Message-State: AOUpUlEdItSrrXLKlzxL829FlCfNYxhwIwsNBKsUyCCq6ZkPIY0KRV/T +GiU0odhLcl+8SbFUjmlZKM7wjG+ X-Google-Smtp-Source: AAOMgpeioyPmvo5wmgYDF/tm2PECEk1OwSc+j7uJzRHc6vQLrb+SR4K8i6oy0ekB3D8pR8uouNzQZg== X-Received: by 2002:a62:1647:: with SMTP id 68-v6mr22188096pfw.6.1533655445954; Tue, 07 Aug 2018 08:24:05 -0700 (PDT) Received: from raichu (toroon0560w-lp130-09-70-52-224-239.dsl.bell.ca. [70.52.224.239]) by smtp.gmail.com with ESMTPSA id s66-v6sm1906761pgc.67.2018.08.07.08.24.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 08:24:04 -0700 (PDT) Sender: Mark Johnston Date: Tue, 7 Aug 2018 11:24:02 -0400 From: Mark Johnston To: Alan Somers Cc: "freebsd-testing@freebsd.org" Subject: Re: Skipping tests that are unimplemented in 32-bit emulation Message-ID: <20180807152402.GG77150@raichu> References: <20180806191406.GA77150@raichu> <20180807150349.GF77150@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 15:24:08 -0000 On Tue, Aug 07, 2018 at 09:08:39AM -0600, Alan Somers wrote: > On Tue, Aug 7, 2018 at 9:03 AM, Mark Johnston wrote: > > > On Mon, Aug 06, 2018 at 02:18:35PM -0600, Alan Somers wrote: > > > On Mon, Aug 6, 2018 at 1:14 PM, Mark Johnston wrote: > > > > > > > On Sun, Jul 29, 2018 at 11:23:33AM -0600, Alan Somers wrote: > > > > > I recently tried running the i386 test suite in a chroot on an amd64 > > > > > system. 162 tests failed, and 33 were broken. Some of the failures > > were > > > > > due to system calls that haven't been implemented in 32-bit > > emulation. > > > > > setfib(2) is an example. I think it's unlikely that anybody will > > ever > > > > need > > > > > 32-bit emulation for setfib(2), so perhaps we should just skip the > > test? > > > > > What's the best way to do that? I can come up with two ways: > > > > > > > > > > 1) At runtime, check the hw.machine sysctl and see if it matches some > > > > > compile time preprocessor constant. I don't know what constant to > > use, > > > > > though. Checking __amd64__ would only work for i386 binaries on > > amd64 > > > > > kernels, and not something else like mips binaries on mips64 kernels > > (I > > > > > don't know if we support that, but I don't want to rule it out). > > > > > > > > > > 2) At buildtime, put an "allowed_architectures=i386" metadata > > property > > > > into > > > > > the Kyuafile for that test program. This would require support in > > > > > /usr/share/mk/bsd.test.mk. It would also require patching Kyua > > itself, > > > > > because currently "Kyua config" returns the architecture for which > > it was > > > > > built, not the one on which it's running. > > > > > > > > > > Thoughts? > > > > > > > > I don't have any particular suggestions, but I'd personally rather > > avoid > > > > a solution that requires tests to opt-in to running under 32-bit > > > > emulation, which I think excludes 2). I'd be happy to help annotate > > > > any failing tests as required. It bugs me that the test suite > > currently > > > > doesn't cover such relatively complicated functions as > > > > freebsd32_copy_msg_out(). > > > > > > I don't think that 2 would necessarily be opt-in, because an undefined > > > value for allowed_architectures is interpreted as meaning "all". It > > could > > > be opt-out instead. But it could still be a little awkward. Option 1 > > > could be accomplished for atf-c testcases by comparing the value of > > > __LP64__ to a hardcoded list of known 64-bit processors as returned by > > > uname(3). But I don't know how to implement 1 for atf-sh programs. An > > > atf-sh program would need to know the architecture of any binary that it > > > might invoke. Is there anything in /etc indicating what architecture the > > > image was built for? Should we just use "file /lib/libc.so.*"? > > > > Could we instead build and install a /usr/tests32 suite on systems that > > can support it, and use a top-level Kyuafile and kyua.conf that overrides > > the "architecture" variable? > > > > It's not enough to build and install the atf-c tests in 32-bit mode. We > would need an entire 32-bit chroot in order to run the atf-sh tests in > 32-bit mode. Oops, right. For 2), then, couldn't we define test_suite("FreeBSD32") in the kyuafiles for the 32-bit test suite and set a test_suite.FreeBSD32.compat32=true variable in kyua.conf? Then a test which uses, say, setfib(1) can query that variable and skip if it's set. From owner-freebsd-testing@freebsd.org Tue Aug 7 16:11:48 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1EE91063066 for ; Tue, 7 Aug 2018 16:11:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com [IPv6:2a00:1450:4864:20::130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0FAD275D5F; Tue, 7 Aug 2018 16:11:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf1-x130.google.com with SMTP id f135-v6so12017158lfg.10; Tue, 07 Aug 2018 09:11:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=kQyZ/tYX7TDpdBu3H/VVxMWS4qKBQmviKiUKtuToI0g=; b=V+JUmP0fJkwd9tmYfn+HAtewWi6fsozwTZqMd2nrYUXCmN8Y1xTbQUof/rmRZ6NMRH i9Mhq5Wn9KJr/uBAXkrFw6m6pJdp9WASwNnyCRALpNPInLcnODyP4D9l6sADOwxeO8sp kAm1PtoanugiEuIjxnKH/aZDnDi4eRPCSDsTzmc7pGKFoH9RHz50WPT2eZtfzBpxJDaG IZVRg+3k5JKHV9qoypdd+B30cXNek2B3uY9K4C/538eWTTW8UIPudY/KpG28OWFWVg7l TlGSwEmQMYG+xQ+/xG7P/M9WX+fMJ8dPxfTXvoc09TfSeB0J+1OWoIXpQhxioPSMv10+ BSxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=kQyZ/tYX7TDpdBu3H/VVxMWS4qKBQmviKiUKtuToI0g=; b=jX7DSWJ00EaJb73W+AgMNkvoToPRZR1Vfa1DOfy+QoWHFfY573yz+2LZyztrK+MPiu o6ELQES2GzZ72AYLvv5zXzSwL3K83Z4snvMwLvsAYHkxL3Qw1P/Sq2pP1Op0D04qTgIj GIkb1aog5DCBdFmfqT45JSvbVWiSuQ3zWOqd+eueEMRWO8PJDLPDFzYvnxi2CQjjtBu6 z6eXPH3QTiV5aUl0iRf/WCsjJwRmkelhIq5HzaeEDyCDJwNGlQmpf9kZrAMwCkN6NMzP 3Y1DxJ6Eo8aaY2yAwamDDqrUQ77oyJGlBbKr7N6G5uNNgOqr+2wdJiLrydgySYTYZPZl zelQ== X-Gm-Message-State: AOUpUlFPuFtAqeutvq9rk7bGyGYnTsd3PcxKpYJWF2x19Thvabp7cmnj Idat9OtDCM27NWXThp0dCkhK13m+L9gOBw/k+yIiVOM7 X-Google-Smtp-Source: AAOMgpevtnL5tYCaJvconujFf/LVdZwHmcrXzyjikDGeDYEIjvtsDsz4NjzOzTYwcxnZFbFCv3i47m/qM8DWPts4NLM= X-Received: by 2002:a19:1003:: with SMTP id f3-v6mr15236837lfi.116.1533658306222; Tue, 07 Aug 2018 09:11:46 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:7851:0:0:0:0:0 with HTTP; Tue, 7 Aug 2018 09:11:45 -0700 (PDT) In-Reply-To: <20180807152402.GG77150@raichu> References: <20180806191406.GA77150@raichu> <20180807150349.GF77150@raichu> <20180807152402.GG77150@raichu> From: Alan Somers Date: Tue, 7 Aug 2018 10:11:45 -0600 X-Google-Sender-Auth: GCSTX_p8dzHvQlP2NWMYpwevGgo Message-ID: Subject: Re: Skipping tests that are unimplemented in 32-bit emulation To: Mark Johnston Cc: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 16:11:48 -0000 On Tue, Aug 7, 2018 at 9:24 AM, Mark Johnston wrote: > On Tue, Aug 07, 2018 at 09:08:39AM -0600, Alan Somers wrote: > > On Tue, Aug 7, 2018 at 9:03 AM, Mark Johnston wrote: > > > > > On Mon, Aug 06, 2018 at 02:18:35PM -0600, Alan Somers wrote: > > > > On Mon, Aug 6, 2018 at 1:14 PM, Mark Johnston > wrote: > > > > > > > > > On Sun, Jul 29, 2018 at 11:23:33AM -0600, Alan Somers wrote: > > > > > > I recently tried running the i386 test suite in a chroot on an > amd64 > > > > > > system. 162 tests failed, and 33 were broken. Some of the > failures > > > were > > > > > > due to system calls that haven't been implemented in 32-bit > > > emulation. > > > > > > setfib(2) is an example. I think it's unlikely that anybody will > > > ever > > > > > need > > > > > > 32-bit emulation for setfib(2), so perhaps we should just skip > the > > > test? > > > > > > What's the best way to do that? I can come up with two ways: > > > > > > > > > > > > 1) At runtime, check the hw.machine sysctl and see if it matches > some > > > > > > compile time preprocessor constant. I don't know what constant > to > > > use, > > > > > > though. Checking __amd64__ would only work for i386 binaries on > > > amd64 > > > > > > kernels, and not something else like mips binaries on mips64 > kernels > > > (I > > > > > > don't know if we support that, but I don't want to rule it out). > > > > > > > > > > > > 2) At buildtime, put an "allowed_architectures=i386" metadata > > > property > > > > > into > > > > > > the Kyuafile for that test program. This would require support > in > > > > > > /usr/share/mk/bsd.test.mk. It would also require patching Kyua > > > itself, > > > > > > because currently "Kyua config" returns the architecture for > which > > > it was > > > > > > built, not the one on which it's running. > > > > > > > > > > > > Thoughts? > > > > > > > > > > I don't have any particular suggestions, but I'd personally rather > > > avoid > > > > > a solution that requires tests to opt-in to running under 32-bit > > > > > emulation, which I think excludes 2). I'd be happy to help > annotate > > > > > any failing tests as required. It bugs me that the test suite > > > currently > > > > > doesn't cover such relatively complicated functions as > > > > > freebsd32_copy_msg_out(). > > > > > > > > I don't think that 2 would necessarily be opt-in, because an > undefined > > > > value for allowed_architectures is interpreted as meaning "all". It > > > could > > > > be opt-out instead. But it could still be a little awkward. Option > 1 > > > > could be accomplished for atf-c testcases by comparing the value of > > > > __LP64__ to a hardcoded list of known 64-bit processors as returned > by > > > > uname(3). But I don't know how to implement 1 for atf-sh programs. > An > > > > atf-sh program would need to know the architecture of any binary > that it > > > > might invoke. Is there anything in /etc indicating what > architecture the > > > > image was built for? Should we just use "file /lib/libc.so.*"? > > > > > > Could we instead build and install a /usr/tests32 suite on systems that > > > can support it, and use a top-level Kyuafile and kyua.conf that > overrides > > > the "architecture" variable? > > > > > > > It's not enough to build and install the atf-c tests in 32-bit mode. We > > would need an entire 32-bit chroot in order to run the atf-sh tests in > > 32-bit mode. > > Oops, right. For 2), then, couldn't we define test_suite("FreeBSD32") > in the kyuafiles for the 32-bit test suite and set a > test_suite.FreeBSD32.compat32=true variable in kyua.conf? Then a test > which uses, say, setfib(1) can query that variable and skip if it's set. > Ok, but what will set the test_suite.FreeBSD32.compat32 variable? It can't be set as part of the image build, because then it will have the wrong value in jails and chroots. It needs to be set at runtime. From owner-freebsd-testing@freebsd.org Tue Aug 7 16:34:11 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDE7E10636FC for ; Tue, 7 Aug 2018 16:34:10 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-pl0-x235.google.com (mail-pl0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 525A376715; Tue, 7 Aug 2018 16:34:10 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-pl0-x235.google.com with SMTP id j8-v6so7323124pll.12; Tue, 07 Aug 2018 09:34:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=gO920L9XURYMFAWUYN9BuSb9TRwfns66ug16b0WEr1Q=; b=t16lgpxtdTM3xxtODBF9ePh2DxH+bweSec+8ymHzLsHRtO8NRQxrrA0uRW9/9EHN4O v8vV2fImvRahY8x74pc2+s1inU3pcRJUTb64n+Gfyl0wcwrpz+xiADMKcpEz/clkR/6R 2puegIplrnav/va0Va2+rnCSqV5oJ+e2XS5zKZt0kYjS/LLojTQATw0tlxLFU5Kz8+TS tRPGAnc4fKMIG2xqBPBE+LRaql2PAMcD3mqDcBu9amlHAFdZRNepH/AAxb4kdmDCvf9h CJCX1eDLivKYJK3zzjLYAJxBpvigyOQLREJD3T0Yn5dbrxf2khjzPT9gG/3oeumvc1SQ m4ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=gO920L9XURYMFAWUYN9BuSb9TRwfns66ug16b0WEr1Q=; b=hnod0zzUfI1kbjWuiPYj/Jd6wem7MvjPCOyyjHEXAMxc8K4Nfma7U6cL09oWEbZxuZ +6/3uAFZp0ZNWrxsWCULg1b+1Br546q+Q1VhGrocLI/NYcFaXPFFhLvz7YjgAmiFp27a X5gD8kcWLiOoRlSzU6KUkaE4jk90XxMFPvMYoy1cYeVJ/eRwrXg6RqRm6fDRWbxm/xLR PVHNSuBbW66IAkwkfpsG2yH+E+ijuHKhRAACX9bzt30Lqw9MinxW0JNwyJnJlXLlUtBc TUk8CA0kiLpTndmFL0ndZKG0JisutynHwJRvYUSqSIUyGHqHAE/hboc8Ny+zl7dD0GrU YhhQ== X-Gm-Message-State: AOUpUlGvM/BItAwOz3W0Fvv/yrZODOh8zlpxtqhoEHKSjkG6wMJNLFxn W2xCCCIzgoRmPrIXIlWxyOqK3BXQ X-Google-Smtp-Source: AAOMgpcTKq25vdYJcloX3ne/S9ZMXnKbO3xfVHCwOx3verGF2ltYf5EOyGZ4DKj4nsvB6en454lX2w== X-Received: by 2002:a17:902:123:: with SMTP id 32-v6mr18338832plb.181.1533659648947; Tue, 07 Aug 2018 09:34:08 -0700 (PDT) Received: from raichu (toroon0560w-lp130-09-70-52-224-239.dsl.bell.ca. [70.52.224.239]) by smtp.gmail.com with ESMTPSA id y20-v6sm1931614pgv.31.2018.08.07.09.34.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:34:07 -0700 (PDT) Sender: Mark Johnston Date: Tue, 7 Aug 2018 12:34:05 -0400 From: Mark Johnston To: Alan Somers Cc: "freebsd-testing@freebsd.org" Subject: Re: Skipping tests that are unimplemented in 32-bit emulation Message-ID: <20180807163405.GH77150@raichu> References: <20180806191406.GA77150@raichu> <20180807150349.GF77150@raichu> <20180807152402.GG77150@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 16:34:11 -0000 On Tue, Aug 07, 2018 at 10:11:45AM -0600, Alan Somers wrote: > On Tue, Aug 7, 2018 at 9:24 AM, Mark Johnston wrote: > > Oops, right. For 2), then, couldn't we define test_suite("FreeBSD32") > > in the kyuafiles for the 32-bit test suite and set a > > test_suite.FreeBSD32.compat32=true variable in kyua.conf? Then a test > > which uses, say, setfib(1) can query that variable and skip if it's set. > > > > Ok, but what will set the test_suite.FreeBSD32.compat32 variable? It can't > be set as part of the image build, because then it will have the wrong > value in jails and chroots. It needs to be set at runtime. For my suggestion to work, the 32-bit test suite must be built with TESTSUITE=FreeBSD32 instead of the default in suite.test.mk. This could be accomplished in a few ways: 1) Modify the amd64 build to additionally build a 32-bit copy of the test suite with TESTSUITE=FreeBSD32 and install it to /usr/tests32. Users that wish to run it would have to copy it into their chroot/jail. 2) When building a 32-bit image for a chroot/jail, just remember to specify TESTSUITE=FreeBSD32 in the make(1) invocation. (I didn't actually verify that this does what I expect.) 3) Always define the test suite name to be FreeBSD32 or FreeBSD64 depending on the target architecture. Then, tests that need to be skipped when run under COMPAT32 would need to check for the "compat32" variable _and_ determine if the host kernel is 64-bit (e.g., by looking at the hw.machine sysctl). If you do 2) and just set test_suite.FreeBSD32.compat32 statically in /usr/local/etc/kyua.conf, then I don't see why you need any special runtime logic. Any tests that aren't supported by COMPAT32 can just be skipped when the compat32 kyua variable is set. From owner-freebsd-testing@freebsd.org Tue Aug 7 16:53:02 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A37F61064240 for ; Tue, 7 Aug 2018 16:53:02 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com [IPv6:2a00:1450:4864:20::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17C5677710; Tue, 7 Aug 2018 16:53:02 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-x234.google.com with SMTP id v9-v6so13990494ljk.4; Tue, 07 Aug 2018 09:53:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=xE5dnbHoi0Ukr2O/BBdgtBXuhwI3jLW/ttj++dby7S0=; b=ERS7OSKmsQY4M88Mh1lKbmJc6+4Et5aFUZJfkZfs+98jzH49+6sUVF6EqxxB7cmTF6 UbFFm1U0xIhOUM6dpMbyd0fN4Oq+LlYV5U9GsNdLfr4e4XnAC2iOFqmyDU5Md6n8pMp5 OYtcTtG1dMf3isCpB0xZJRveiA07/bLEDTBi+TyXi1do/J8N2BEa10X0gQw//rYAFzVq 4TRVYpk36Ml+4paTu5tWMCf9e682eu27T91/0qlpmABGT6yMHa7C7DKscvmb0DvyvXSM QsoFdfdrS7VlEUpK4o2aUGQlFskiZ+haorvhsphoIJrNXB7AWqKA03qpOTQjbrtXx1ya OpOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=xE5dnbHoi0Ukr2O/BBdgtBXuhwI3jLW/ttj++dby7S0=; b=rHvESW6rUjv2HPOcMPYZCVNgwhWopMh5jSmio6q+7FtI5ZTPfVbwXn8CZza9tqtXpU n4ya1gU5d4MjGFdgwLgZWiDKwve7ovfa4ySHb5yW8nBQbl/em7cW8XUIWUx0EUqdfXWW GFFSjBJFkZuFiTDVbbc14pO3E+EBcQ9dJtqoY3JCvQTYk126dSGOr+TNDBaxku9ryh8c KEZzRhGvtB+Zd5G0xqKhaF7mVecMaKsQx9zPTeNdmf8TCZkIoxjhi6naIhW7RE7k0SRt xmFahLKjNqWSdYAJdE45Yum9GBaKvbJXk+L//8vpTEScnZpxCKGsxNK0Idm8FPJiztqb Sw3Q== X-Gm-Message-State: AOUpUlFwQ6nCGcAz9T+zt+FVUVCtRUNHc3pnyuEUhvJv5elCeoRFx6y3 FM1Mdj5Rrwi0AP6T6aBbUsX7mIj4OdISw7NseYrnMQ== X-Google-Smtp-Source: AAOMgpdcLkGXtj7SyRgQEFoNBKaxVXevIapMAValNCFYqBIOobCj7R6cHFkxLTSPLKEJ07eUYqnXlyYgc7pcPzwHWgs= X-Received: by 2002:a2e:87da:: with SMTP id v26-v6mr15897506ljj.69.1533660780404; Tue, 07 Aug 2018 09:53:00 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:7851:0:0:0:0:0 with HTTP; Tue, 7 Aug 2018 09:52:59 -0700 (PDT) In-Reply-To: <20180807163405.GH77150@raichu> References: <20180806191406.GA77150@raichu> <20180807150349.GF77150@raichu> <20180807152402.GG77150@raichu> <20180807163405.GH77150@raichu> From: Alan Somers Date: Tue, 7 Aug 2018 10:52:59 -0600 X-Google-Sender-Auth: RNto7bU5hCosYjb0gYaQa_0xRP0 Message-ID: Subject: Re: Skipping tests that are unimplemented in 32-bit emulation To: Mark Johnston Cc: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 16:53:03 -0000 On Tue, Aug 7, 2018 at 10:34 AM, Mark Johnston wrote: > On Tue, Aug 07, 2018 at 10:11:45AM -0600, Alan Somers wrote: > > On Tue, Aug 7, 2018 at 9:24 AM, Mark Johnston wrote: > > > Oops, right. For 2), then, couldn't we define test_suite("FreeBSD32") > > > in the kyuafiles for the 32-bit test suite and set a > > > test_suite.FreeBSD32.compat32=true variable in kyua.conf? Then a test > > > which uses, say, setfib(1) can query that variable and skip if it's > set. > > > > > > > Ok, but what will set the test_suite.FreeBSD32.compat32 variable? It > can't > > be set as part of the image build, because then it will have the wrong > > value in jails and chroots. It needs to be set at runtime. > > For my suggestion to work, the 32-bit test suite must be built with > TESTSUITE=FreeBSD32 instead of the default in suite.test.mk. This could > be accomplished in a few ways: > 1) Modify the amd64 build to additionally build a 32-bit copy of the > test suite with TESTSUITE=FreeBSD32 and install it to /usr/tests32. > Users that wish to run it would have to copy it into their > chroot/jail. > 2) When building a 32-bit image for a chroot/jail, just remember to > specify TESTSUITE=FreeBSD32 in the make(1) invocation. (I didn't > actually verify that this does what I expect.) > 3) Always define the test suite name to be FreeBSD32 or FreeBSD64 > depending on the target architecture. Then, tests that need to be > skipped when run under COMPAT32 would need to check for the > "compat32" variable _and_ determine if the host kernel is 64-bit > (e.g., by looking at the hw.machine sysctl). > > If you do 2) and just set test_suite.FreeBSD32.compat32 statically in > /usr/local/etc/kyua.conf, then I don't see why you need any special > runtime logic. Any tests that aren't supported by COMPAT32 can just be > skipped when the compat32 kyua variable is set. > Option 2) may avoid additional runtime logic, but at the expense of creating an image that won't work correctly when run natively. Plus, you would also have to copy all of test_suite.FreeBSD's Kyua variables to test_suite.FreeBSD32. For example, test_suites.FreeBSD.disks. I still think the simplest option is to build a single test suite, where the few tests that can't run under emulation will call a helper function that compares the build architecture to the current kernel and skip on a mismatch. It eliminates extra configuration, uses just a single image, and doesn't modify a large number of files. Are there any emulations besides amd64/i386 that we support? Like mips64/mips for example? -Alan From owner-freebsd-testing@freebsd.org Tue Aug 7 17:09:51 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D34101064C6A for ; Tue, 7 Aug 2018 17:09:50 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 31DF977EBE; Tue, 7 Aug 2018 17:09:50 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-pg1-x544.google.com with SMTP id x5-v6so8137011pgp.7; Tue, 07 Aug 2018 10:09:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=FueDaCPyobJgE+4SZ+pXxVAwr0gzczhuEskBELo3Rxw=; b=ZIl7mfpNk+A8g8zrGHSqAFm+bCA8uQ0riv3pmNzLvhHmsl3ze2TUvYAS36coQORPMY irGGKxTXTyREMzj3yts2UgJM0CsEZU1bUqew/otblbKkB33JDu2k9SzhvF3h9MHJzOvt zx4Uo1BWQg0RPenlX/lEYl2kD5pOeYjRiZmOr2R9S0j30GRmn4VLra34mQuQ3YQ57Zkk 5WigxgmWXc+Chv6dAZWXjzbP8y3vd4b2TCV1J8l0OJZrO0/W3PqMjzDviQ27ifQJfFgB e/xV5wsKu9WlkzQkq/hPClFJKk1mfe+jT3TEIeW0beqowUzkZjCvgeh+yv5r5Kz3aRaA t7iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=FueDaCPyobJgE+4SZ+pXxVAwr0gzczhuEskBELo3Rxw=; b=sfKV6cSAglXi9HGFy7e75VjASxJ3sOZdnCey16uBUf1ZFPfQdYYyepGl+nbh4zLfDH gS7Ic5sWHhSgxqihiwfkhnExQWL6s7d/ab9LpGggQUUr9yORuX1nvCBNr9YIOmKJxq8T N42oGWRH0EHNaPXnvVd9TL0X+taQHZufl5g8QvK+Lj8ymds42AujMaS11v+gdljXGdph ZTqJXsOs6ywqD8/KWfK6+MP0Kb6nKBqP7egK8HEG7SAoZ1xekiYPjj7c2JDAlcaUynME RQJLFtb9FhIWAiNSwiX3pGn/LPIkNeB+iLUkV7v94m6pnY4IBYUb91AKSuzpccIbN505 +pJA== X-Gm-Message-State: AOUpUlEbs2llLMyowe4JUHqi3eem7dK00CxcpRJhVzYlODJj8Nprtb6K DHC+XC5sFhTryrnsFkXYGh6rMaV6 X-Google-Smtp-Source: AAOMgpeqgCJs7ofjxzGuEOfV6f7CtEQ2Nfm2LXswDZnsDzYjEcxjMDrko9bc6ekVaQRZv04Li3NzTg== X-Received: by 2002:a62:4ece:: with SMTP id c197-v6mr22866465pfb.240.1533661788856; Tue, 07 Aug 2018 10:09:48 -0700 (PDT) Received: from raichu (toroon0560w-lp130-09-70-52-224-239.dsl.bell.ca. [70.52.224.239]) by smtp.gmail.com with ESMTPSA id y86-v6sm3541400pfk.84.2018.08.07.10.09.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 10:09:47 -0700 (PDT) Sender: Mark Johnston Date: Tue, 7 Aug 2018 13:09:44 -0400 From: Mark Johnston To: Alan Somers Cc: "freebsd-testing@freebsd.org" Subject: Re: Skipping tests that are unimplemented in 32-bit emulation Message-ID: <20180807170944.GJ77150@raichu> References: <20180806191406.GA77150@raichu> <20180807150349.GF77150@raichu> <20180807152402.GG77150@raichu> <20180807163405.GH77150@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 17:09:51 -0000 On Tue, Aug 07, 2018 at 10:52:59AM -0600, Alan Somers wrote: > On Tue, Aug 7, 2018 at 10:34 AM, Mark Johnston wrote: > > > On Tue, Aug 07, 2018 at 10:11:45AM -0600, Alan Somers wrote: > > > On Tue, Aug 7, 2018 at 9:24 AM, Mark Johnston wrote: > > > > Oops, right. For 2), then, couldn't we define test_suite("FreeBSD32") > > > > in the kyuafiles for the 32-bit test suite and set a > > > > test_suite.FreeBSD32.compat32=true variable in kyua.conf? Then a test > > > > which uses, say, setfib(1) can query that variable and skip if it's > > set. > > > > > > > > > > Ok, but what will set the test_suite.FreeBSD32.compat32 variable? It > > can't > > > be set as part of the image build, because then it will have the wrong > > > value in jails and chroots. It needs to be set at runtime. > > > > For my suggestion to work, the 32-bit test suite must be built with > > TESTSUITE=FreeBSD32 instead of the default in suite.test.mk. This could > > be accomplished in a few ways: > > 1) Modify the amd64 build to additionally build a 32-bit copy of the > > test suite with TESTSUITE=FreeBSD32 and install it to /usr/tests32. > > Users that wish to run it would have to copy it into their > > chroot/jail. > > > 2) When building a 32-bit image for a chroot/jail, just remember to > > specify TESTSUITE=FreeBSD32 in the make(1) invocation. (I didn't > > actually verify that this does what I expect.) > > 3) Always define the test suite name to be FreeBSD32 or FreeBSD64 > > depending on the target architecture. Then, tests that need to be > > skipped when run under COMPAT32 would need to check for the > > "compat32" variable _and_ determine if the host kernel is 64-bit > > (e.g., by looking at the hw.machine sysctl). > > > > If you do 2) and just set test_suite.FreeBSD32.compat32 statically in > > /usr/local/etc/kyua.conf, then I don't see why you need any special > > runtime logic. Any tests that aren't supported by COMPAT32 can just be > > skipped when the compat32 kyua variable is set. > > > > Option 2) may avoid additional runtime logic, but at the expense of > creating an image that won't work correctly when run natively. Plus, you > would also have to copy all of test_suite.FreeBSD's Kyua variables to > test_suite.FreeBSD32. For example, test_suites.FreeBSD.disks. > I still think the simplest option is to build a single test suite, where > the few tests that can't run under emulation will call a helper function > that compares the build architecture to the current kernel and skip on a > mismatch. It eliminates extra configuration, uses just a single image, and > doesn't modify a large number of files. > > Are there any emulations besides amd64/i386 that we support? Like > mips64/mips for example? There is COMPAT_FREEBSD32 glue for mips and powerpc64 at least. I'm not sure how well-supported that is though.