Date: Mon, 29 Apr 2024 16:16:58 -0600 From: Warner Losh <imp@bsdimp.com> To: Brooks Davis <brooks@freebsd.org> Cc: Christian Weisgerber <naddy@mips.inka.de>, freebsd-hackers@freebsd.org Subject: Re: How to run tests without installing? Message-ID: <CANCZdfqz2AVyYNh34irA93jhVtbLKzBJNW5g4R7wL4WSX%2BT6BA@mail.gmail.com> In-Reply-To: <ZjAHcluVI2SGh8Ul@spindle.one-eyed-alien.net> References: <Zi93yxSsEnttr3Ay@lorvorc.mips.inka.de> <ZjAHcluVI2SGh8Ul@spindle.one-eyed-alien.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000525a9b0617439d65 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Apr 29, 2024 at 4:03=E2=80=AFPM Brooks Davis <brooks@freebsd.org> w= rote: > On Mon, Apr 29, 2024 at 12:34:51PM +0200, Christian Weisgerber wrote: > > How can I run the regression tests on my work-in-progress without > > installing it first? > > > > Say I'm changing something in sh. Can I run the tests on the > > compiled sh in /usr/obj, without having to install my potentially > > broken work into the system? Running "make tests" in src/bin/sh > > doesn't seem to actually test anything. > > Generally speaking you can't. This is one of the problematic things > about the current test framework. The best you can do for something > like sh where you really don't want to install a broken one is probably > installing in a jail and running tests there. > For library tests, though, you can run the individual test just like kyua would, but all bets are off if it relies on anything other than the code in the binary (and associated shared libraries). Even then, it's a royal pain. I usually stop one step short of a jail when I've needed to do this: I just make installworld DESTDIR=3D$HOME/mumble and then do sudo chroot $HOME/mumble. Though it doesn't work at all well for some tests (they hate being run in a chroot), but they tend to be networking tests anyway... But the same tests that don't work well in a chroot work even less well in a jail. There's a crazy lot of them that require different /dev/ entries, which is why I don't do the chroot thing that often... tl;dr: It's easier not to test, and that's actually a problem. Warner --000000000000525a9b0617439d65 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Mon, Apr 29, 2024 at 4:03=E2=80=AF= PM Brooks Davis <<a href=3D"mailto:brooks@freebsd.org">brooks@freebsd.or= g</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin= :0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"= >On Mon, Apr 29, 2024 at 12:34:51PM +0200, Christian Weisgerber wrote:<br> > How can I run the regression tests on my work-in-progress without<br> > installing it first?<br> > <br> > Say I'm changing something in sh.=C2=A0 Can I run the tests on the= <br> > compiled sh in /usr/obj, without having to install my potentially<br> > broken work into the system?=C2=A0 Running "make tests" in s= rc/bin/sh<br> > doesn't seem to actually test anything.<br> <br> Generally speaking you can't.=C2=A0 This is one of the problematic thin= gs<br> about the current test framework.=C2=A0 The best you can do for something<b= r> like sh where you really don't want to install a broken one is probably= <br> installing in a jail and running tests there.<br></blockquote><div><br></di= v><div>For library tests, though, you can run the individual test just like= kyua</div><div>would, but all bets are off if it relies on anything other = than the code in</div><div>the binary (and associated shared libraries). Ev= en then, it's a royal</div><div>pain.</div><div><br></div><div>I usuall= y stop one step short of a jail when I've needed to do this:</div><div>= I just make installworld DESTDIR=3D$HOME/mumble and then do</div><div>sudo = chroot $HOME/mumble. Though it doesn't work at all well</div><div>for s= ome tests (they hate being run in a chroot), but they tend to</div><div>be = networking tests anyway... But the same tests that don't work</div><div= >well in a chroot work even less well in a jail. There's a crazy lot</d= iv><div>of them that require different /dev/ entries, which is why I don= 9;t do</div><div>the chroot thing that often...</div><div><br></div><div>tl= ;dr: It's easier not to test, and that's actually a problem.</div><= div><br></div><div>Warner</div></div></div> --000000000000525a9b0617439d65--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfqz2AVyYNh34irA93jhVtbLKzBJNW5g4R7wL4WSX%2BT6BA>