Date: Tue, 16 Jan 2024 18:55:51 -0700 From: Warner Losh <imp@bsdimp.com> To: Kyle Evans <kevans@freebsd.org> Cc: Mark Johnston <markj@freebsd.org>, freebsd-arch@freebsd.org Subject: Re: orch(1) for tty testing Message-ID: <CANCZdfoyx28YocZ1jva6aCCU8iJOomMGJiCa6-F8C%2BsCOFqpzA@mail.gmail.com> In-Reply-To: <be058507-b1c9-418e-bf5f-88b986f39f82@FreeBSD.org> References: <478cf238-ab2e-4185-a5eb-4230b5efe735@FreeBSD.org> <ZabXgs4kWzVBFlBT@nuc> <be058507-b1c9-418e-bf5f-88b986f39f82@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000096979e060f1a8cc2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Jan 16, 2024 at 6:27=E2=80=AFPM Kyle Evans <kevans@freebsd.org> wro= te: > On 1/16/24 13:22, Mark Johnston wrote: > > On Mon, Jan 15, 2024 at 01:54:54PM -0600, Kyle Evans wrote: > >> Hi, > >> > >> I wrote a tool for tty testing w/ lua scripts, inspired by expect(1) a= nd > >> with some similarities, but explicitly without the goal of trying to b= e > >> compatible: > >> > >> https://git.kevans.dev/kevans/orch > >> > >> An example of its use is here: > >> > >> https://git.kevans.dev/kevans/tty-tests/src/branch/main/test_canon.orc= h > >> > >> Each spawn() creates a new pts(4) and allows us to drive I/O to the > process > >> with write/match statements. I have one or two tests written with it > so far > >> and the end result is, IMO, much less painful looking and less error > prone > >> than the equivalent in C. > >> > >> It still needs a bit of polish (e.g., improved diagnostics when a matc= h > >> fails), but I'd like to consider importing this somewhere for use in t= he > >> near future for tty testing given how much it simplifies. I can push > it to > >> ports, but we have a private liblua in base so it could be built and > >> banished off into, e.g., /usr/tests/sys/kern/tty/, to make my life jus= t > a > >> tad easier re: getting these to run in CI. It's pretty lightweight; t= he > >> orch binary on my amd64 system clocks in at ~27K, and orch.lua at 13K. > >> > >> Any objections to importing this new tool into /usr/tests or maybe eve= n > >> /usr/libexec? > > > > This looks quite handy. I have some lua scripts which I use to checkou= t > > a src tree, build a VM image, boot the image and run tests, then > > download the resulting kyua DB, all with a single CLI command. I > > currently run the hypervisor under expect(1) and send commands over a > > pipe, but that was an expedient hack and I'd rather use something > > implemented natively. There are some existing lua expect ports, but > > orch looks simpler and I'll try it out to see if it's missing anything > > for my use-case. More generally, we lack a test framework which lets > > one boot and interact with VMs, and orch might be a useful component of > > that. > > That seems like a good exercise for it- feel free to get in touch. I > wouldn't mind porting whatever you've written in expect(1) over and take > the opportunity to fill in the feature gaps, regardless of whether you > use the end result or not. > Last summer's gsoc student wrote a bunch of Lua to do similar things with testing boot loaders against a kernel and minimal userland... Would be nice to have those in-tree and maybe using this. Warner --00000000000096979e060f1a8cc2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Tue, Jan 16, 2024 at 6:27=E2=80=AF= PM Kyle Evans <<a href=3D"mailto:kevans@freebsd.org">kevans@freebsd.org<= /a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0= px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">O= n 1/16/24 13:22, Mark Johnston wrote:<br> > On Mon, Jan 15, 2024 at 01:54:54PM -0600, Kyle Evans wrote:<br> >> Hi,<br> >><br> >> I wrote a tool for tty testing w/ lua scripts, inspired by expect(= 1) and<br> >> with some similarities, but explicitly without the goal of trying = to be<br> >> compatible:<br> >><br> >> <a href=3D"https://git.kevans.dev/kevans/orch" rel=3D"noreferrer" = target=3D"_blank">https://git.kevans.dev/kevans/orch</a><br>; >><br> >> An example of its use is here:<br> >><br> >> <a href=3D"https://git.kevans.dev/kevans/tty-tests/src/branch/main= /test_canon.orch" rel=3D"noreferrer" target=3D"_blank">https://git.kevans.d= ev/kevans/tty-tests/src/branch/main/test_canon.orch</a><br> >><br> >> Each spawn() creates a new pts(4) and allows us to drive I/O to th= e process<br> >> with write/match statements.=C2=A0 I have one or two tests written= with it so far<br> >> and the end result is, IMO, much less painful looking and less err= or prone<br> >> than the equivalent in C.<br> >><br> >> It still needs a bit of polish (e.g., improved diagnostics when a = match<br> >> fails), but I'd like to consider importing this somewhere for = use in the<br> >> near future for tty testing given how much it simplifies.=C2=A0 I = can push it to<br> >> ports, but we have a private liblua in base so it could be built a= nd<br> >> banished off into, e.g., /usr/tests/sys/kern/tty/, to make my life= just a<br> >> tad easier re: getting these to run in CI.=C2=A0 It's pretty l= ightweight; the<br> >> orch binary on my amd64 system clocks in at ~27K, and orch.lua at = 13K.<br> >><br> >> Any objections to importing this new tool into /usr/tests or maybe= even<br> >> /usr/libexec?<br> > <br> > This looks quite handy.=C2=A0 I have some lua scripts which I use to c= heckout<br> > a src tree, build a VM image, boot the image and run tests, then<br> > download the resulting kyua DB, all with a single CLI command.=C2=A0 I= <br> > currently run the hypervisor under expect(1) and send commands over a<= br> > pipe, but that was an expedient hack and I'd rather use something<= br> > implemented natively.=C2=A0 There are some existing lua expect ports, = but<br> > orch looks simpler and I'll try it out to see if it's missing = anything<br> > for my use-case.=C2=A0 More generally, we lack a test framework which = lets<br> > one boot and interact with VMs, and orch might be a useful component o= f<br> > that.<br> <br> That seems like a good exercise for it- feel free to get in touch. I <br> wouldn't mind porting whatever you've written in expect(1) over and= take <br> the opportunity to fill in the feature gaps, regardless of whether you <br> use the end result or not.<br></blockquote><div><br></div><div>Last summer&= #39;s gsoc student wrote a bunch of Lua to do similar things with</div><div= >testing boot loaders against a kernel and minimal userland... Would be nic= e</div><div>to have those in-tree and maybe using this.<br></div><div><br><= /div><div>Warner <br></div></div></div> --00000000000096979e060f1a8cc2--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfoyx28YocZ1jva6aCCU8iJOomMGJiCa6-F8C%2BsCOFqpzA>