From owner-svn-src-head@FreeBSD.ORG Thu Feb 12 21:57:36 2015 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 874AE14F; Thu, 12 Feb 2015 21:57:36 +0000 (UTC) Received: from mail-pd0-f173.google.com (mail-pd0-f173.google.com [209.85.192.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 517261B2; Thu, 12 Feb 2015 21:57:36 +0000 (UTC) Received: by pdbfl12 with SMTP id fl12so14801662pdb.2; Thu, 12 Feb 2015 13:57:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=3xXYgmKBLDu4oSr5u//CnIZUej2MDHyXyweSpqNhRcI=; b=jCvKHfqPJwqcPWlFBTKHTzgtGxWiMVA197H7m6HjpZTB5tsRZY0Z3vKMyNrf85LePf 8KIT87ABrfl35zdEvDjdUSQACi9zmpieP955BbjZ5J3pE06ujZUX0+ioV5ec0b3InQ90 +zhJwQycj5reudxrxJfhNBuOIb2KAiGsWzX6OHz0BwGmZ2G3n0UZM//faczeWoGKehn8 k+loxcms2gjKvwYkySYCUzR9k6wcuFxSbmSjFlQZet2xg0QEGZE3k6LGD805Y4gksmqq frnCpKePHsJ+zn4uQiUmiH83ua37hsABWcRjXUO0nLROJT5OZG+MODCkMwMA+2FJzmgW yxoA== X-Received: by 10.70.37.202 with SMTP id a10mr10071444pdk.108.1423778255187; Thu, 12 Feb 2015 13:57:35 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:5de4:da0f:932c:57df? ([2601:8:ab80:7d6:5de4:da0f:932c:57df]) by mx.google.com with ESMTPSA id n8sm4668376pdm.62.2015.02.12.13.57.34 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 12 Feb 2015 13:57:34 -0800 (PST) Content-Type: multipart/signed; boundary="Apple-Mail=_178185C1-3F5E-47FB-8470-97460255EB0C"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: svn commit: r278323 - in head: etc/rc.d usr.sbin/jail From: Garrett Cooper In-Reply-To: Date: Thu, 12 Feb 2015 13:57:31 -0800 Message-Id: <42AB9765-E63E-494B-A3FE-D9FF696921D9@gmail.com> References: <201502061754.t16HssXU042750@svn.freebsd.org> <343803A3-CFA3-4766-8294-139A6D8C0235@gmail.com> To: James Gritton X-Mailer: Apple Mail (2.1878.6) Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers , jenkins-admin@freebsd.org X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Feb 2015 21:57:36 -0000 --Apple-Mail=_178185C1-3F5E-47FB-8470-97460255EB0C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Feb 12, 2015, at 13:12, Garrett Cooper wrote: > On Feb 9, 2015, at 18:51, James Gritton wrote: >=20 >> On 2015-02-06 22:23, Garrett Cooper wrote: >>> On Feb 6, 2015, at 18:38, James Gritton wrote: >>>> On 2015-02-06 19:23, Garrett Cooper wrote: >>>>> I think you broke the Jenkins tests runs, and potentially jail = support >>>>> in some edgecases: >>>>> https://jenkins.freebsd.org/job/FreeBSD_HEAD-tests2/651/ >>>> Where do I go from here? There error you refer to certainly seems = jail-related, which leads me to guess at something disconnected between = the matching rc.d/jail and jail(8) change (i.e. using the new rc file = with the old jail program). But that's really just a wild guess. Is = there somewhere I look for more information? For example, where does = Jenkins actually do its thing? >>>> Sorry for being so stupid in this - Jenkins has only been on the = very edge of my awareness until now. >>> I honestly don=92t think it=92s Jenkins because Jenkins runs in = bhyve. I >>> think you accidentally broke option handling in the jail = configuration >>> (please see my other reply about added =93break;=94 statements). >>> ... >>> You can verify your changes by doing: >>> % (cd /usr/tests/bin/pkill; sudo kyua test) >>=20 >> After some testing and looking around, I've decided the problem = definitely isn't in rc.d where I thought it might be. I've also decided = it's probably not in my patch either. >>=20 >> I've run this kyua test on a 10 system (don't have current handy for = such things at the moment), and sometimes I would see a failure and = sometimes I wouldn't. This was whether I was using the new or old jail = code. Later in the day, when the box was less loaded, it seemed to = always pass. Looking at the pkill-j_test script, I see jails being = created with sleep commands both inside and outside the jail around its = creation. I'm guessing this script is very sensitive to timing issues = that could be cause by (among other things) system load. The jail = commands in this script were also very simple, with the only parameters = used being: path, name, ip4.addr, and command. This isn't some kind of = esoteric exercising of the jail(8) options, and I would expect if it = works at one time it would work at another. I've "hand-run" these = particular jail commands and couldn't get them to fail (and the actual = content of the jail(8) changes were tests already). >>=20 >> I looked at the freebsd-current (I think) list where the Jenkins = errors are posted, and it's true it started failing the pkill-j test at = the time I made my change. But it's also true that it had failed that = test once the day before my change, and then started passing it again. = This particular test just seems to be fragile. >>=20 >> So I don't have anywhere else to go with this. I'm going to assume = jail(8) isn't the problem here. >=20 > The tests are racy and make some interesting assumptions. It appears = that WITNESS plays a part in it, and I bet VIMAGE (something that I = don=92t have in my kernel config) plays a part in it too. I say this = because I just ran into the issue when running the tests in a tight loop = on my VMware workstation 7 instance with code from r278636. >=20 > Doesn=92t surprise me because before r272305, it was failing = consistently on head, so what Craig did in that commit helped, but it = didn=92t fully fix the raciness of the tests. >=20 > I=92m going to recompile my system with VIMAGE and see if that impacts = performance of the tests, and if so, I=92ll adjust the sleep between = setting up the jailed instances, and waiting for them to be fully = formed. >=20 > Thanks! >=20 > $ while : ; do sudo prove -rv pgrep-j_test.sh || break; done > pgrep-j_test.sh ..=20 > 1..3 > usage: pgrep [-LSfilnoqvx] [-d delim] [-F pidfile] [-G gid] [-M core] = [-N system] > [-P ppid] [-U uid] [-c class] [-g pgrp] [-j jid] > [-s sid] [-t tty] [-u euid] pattern ... > not ok 1 - pgrep -j # pgrep output: '', pidfile output: '74275 = 74278' > ok 2 - pgrep -j any > ok 3 - pgrep -j none > Failed 1/3 subtests=20 >=20 > Test Summary Report > ------------------- > pgrep-j_test.sh (Wstat: 0 Tests: 3 Failed: 1) > Failed test: 1 > Files=3D1, Tests=3D3, 5 wallclock secs ( 0.04 usr 0.02 sys + 0.02 = cusr 0.55 csys =3D 0.63 CPU) > Result: FAIL This Jenkins run is interesting: = https://jenkins.freebsd.org/job/FreeBSD_HEAD-tests2/686/testReport/junit/b= in.pkill/pgrep-j_test/main/ . The first run passed, but the second one = didn=92t (more output than expected). This error shouldn=92t occur after = r278636, but it definitely confirms the fact that the test is racy, in = other ways. --Apple-Mail=_178185C1-3F5E-47FB-8470-97460255EB0C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJU3SHMAAoJEMZr5QU6S73eKSQH/06dNtSuW9SPvq4rTTwfQi9o UnPDeG5F4FUaPYaD0o6J5NKs+6GGcJy03cMBhThbHlJo3TUJU7bZCOUlSX25s4V8 BaR65UonwhkiqUh1GarUMHS1kCAWIwhGTfrx5A23pSt6EHA7CjDv9qxaLnGRwhln i9xWFcDl1yo1/ihxYF/gT1KBhA7GMr1f0ZmqpXGDR6bSplPN2y5INJKuRQ+fXexg rQbDFxoVSmk7A/2W1ln8k6x/TNHhW78KO8mdyvd5Fixw0vt8jfnALS8giHoNW6my RFSqnvytXtrFw98O+Cl1FerCa5RcFBhT7s+WfV+Rgj+HGzgZACGAyt851e6gAGk= =cjq3 -----END PGP SIGNATURE----- --Apple-Mail=_178185C1-3F5E-47FB-8470-97460255EB0C--