Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 29 Apr 2024 16:16:58 -0600
From:      Warner Losh <imp@bsdimp.com>
To:        Brooks Davis <brooks@freebsd.org>
Cc:        Christian Weisgerber <naddy@mips.inka.de>, freebsd-hackers@freebsd.org
Subject:   Re: How to run tests without installing?
Message-ID:  <CANCZdfqz2AVyYNh34irA93jhVtbLKzBJNW5g4R7wL4WSX%2BT6BA@mail.gmail.com>
In-Reply-To: <ZjAHcluVI2SGh8Ul@spindle.one-eyed-alien.net>
References:  <Zi93yxSsEnttr3Ay@lorvorc.mips.inka.de> <ZjAHcluVI2SGh8Ul@spindle.one-eyed-alien.net>

next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000525a9b0617439d65
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Apr 29, 2024 at 4:03=E2=80=AFPM Brooks Davis <brooks@freebsd.org> w=
rote:

> On Mon, Apr 29, 2024 at 12:34:51PM +0200, Christian Weisgerber wrote:
> > How can I run the regression tests on my work-in-progress without
> > installing it first?
> >
> > Say I'm changing something in sh.  Can I run the tests on the
> > compiled sh in /usr/obj, without having to install my potentially
> > broken work into the system?  Running "make tests" in src/bin/sh
> > doesn't seem to actually test anything.
>
> Generally speaking you can't.  This is one of the problematic things
> about the current test framework.  The best you can do for something
> like sh where you really don't want to install a broken one is probably
> installing in a jail and running tests there.
>

For library tests, though, you can run the individual test just like kyua
would, but all bets are off if it relies on anything other than the code in
the binary (and associated shared libraries). Even then, it's a royal
pain.

I usually stop one step short of a jail when I've needed to do this:
I just make installworld DESTDIR=3D$HOME/mumble and then do
sudo chroot $HOME/mumble. Though it doesn't work at all well
for some tests (they hate being run in a chroot), but they tend to
be networking tests anyway... But the same tests that don't work
well in a chroot work even less well in a jail. There's a crazy lot
of them that require different /dev/ entries, which is why I don't do
the chroot thing that often...

tl;dr: It's easier not to test, and that's actually a problem.

Warner

--000000000000525a9b0617439d65
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Mon, Apr 29, 2024 at 4:03=E2=80=AF=
PM Brooks Davis &lt;<a href=3D"mailto:brooks@freebsd.org">brooks@freebsd.or=
g</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin=
:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"=
>On Mon, Apr 29, 2024 at 12:34:51PM +0200, Christian Weisgerber wrote:<br>
&gt; How can I run the regression tests on my work-in-progress without<br>
&gt; installing it first?<br>
&gt; <br>
&gt; Say I&#39;m changing something in sh.=C2=A0 Can I run the tests on the=
<br>
&gt; compiled sh in /usr/obj, without having to install my potentially<br>
&gt; broken work into the system?=C2=A0 Running &quot;make tests&quot; in s=
rc/bin/sh<br>
&gt; doesn&#39;t seem to actually test anything.<br>
<br>
Generally speaking you can&#39;t.=C2=A0 This is one of the problematic thin=
gs<br>
about the current test framework.=C2=A0 The best you can do for something<b=
r>
like sh where you really don&#39;t want to install a broken one is probably=
<br>
installing in a jail and running tests there.<br></blockquote><div><br></di=
v><div>For library tests, though, you can run the individual test just like=
 kyua</div><div>would, but all bets are off if it relies on anything other =
than the code in</div><div>the binary (and associated shared libraries). Ev=
en then, it&#39;s a royal</div><div>pain.</div><div><br></div><div>I usuall=
y stop one step short of a jail when I&#39;ve needed to do this:</div><div>=
I just make installworld DESTDIR=3D$HOME/mumble and then do</div><div>sudo =
chroot $HOME/mumble. Though it doesn&#39;t work at all well</div><div>for s=
ome tests (they hate being run in a chroot), but they tend to</div><div>be =
networking tests anyway... But the same tests that don&#39;t work</div><div=
>well in a chroot work even less well in a jail. There&#39;s a crazy lot</d=
iv><div>of them that require different /dev/ entries, which is why I don&#3=
9;t do</div><div>the chroot thing that often...</div><div><br></div><div>tl=
;dr: It&#39;s easier not to test, and that&#39;s actually a problem.</div><=
div><br></div><div>Warner</div></div></div>

--000000000000525a9b0617439d65--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfqz2AVyYNh34irA93jhVtbLKzBJNW5g4R7wL4WSX%2BT6BA>