From owner-freebsd-testing@freebsd.org Mon Oct 7 01:08:07 2019 Return-Path: Delivered-To: freebsd-testing@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3C5E6139884 for ; Mon, 7 Oct 2019 01:08:07 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46mj5f70dtz44Pv; Mon, 7 Oct 2019 01:08:06 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 10BB419D22; Mon, 7 Oct 2019 01:08:04 +0000 (UTC) To: Alan Somers Cc: "freebsd-testing@freebsd.org" References: <0cd4df2a-dedd-5cb0-da71-37247fd7bfb4@freebsd.org> From: Allan Jude Openpgp: preference=signencrypt Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= mQINBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABtCJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+iQI/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6a5Ag0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAGJAiUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Subject: Re: Revisiting the ZFS test suite Message-ID: <34dfdd18-cc44-833a-5632-89e54182e973@freebsd.org> Date: Sun, 6 Oct 2019 21:07:50 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CHM4sQDLgVZ844ABGFxz9tUOLDN7pBrMO" X-Rspamd-Queue-Id: 46mj5f70dtz44Pv X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Oct 2019 01:08:07 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --CHM4sQDLgVZ844ABGFxz9tUOLDN7pBrMO Content-Type: multipart/mixed; boundary="W7qAUZkZLkagLrlWudYhU4Jm09wxB7RqS"; protected-headers="v1" From: Allan Jude To: Alan Somers Cc: "freebsd-testing@freebsd.org" Message-ID: <34dfdd18-cc44-833a-5632-89e54182e973@freebsd.org> Subject: Re: Revisiting the ZFS test suite References: <0cd4df2a-dedd-5cb0-da71-37247fd7bfb4@freebsd.org> In-Reply-To: --W7qAUZkZLkagLrlWudYhU4Jm09wxB7RqS Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2019-10-06 18:29, Alan Somers wrote: > On Sun, Oct 6, 2019 at 10:20 AM Allan Jude > wrote: >=20 > On 2019-10-04 12:23, Alan Somers wrote: > > My employment status changed last week, and I find myself once ag= ain > > working on ZFS.=C2=A0 It's hard to have confidence in any changes= without a > > solid test suite, so my main task right now is to fix the problem= s in > > FreeBSD's ZFS test suite.=C2=A0 Our most recent run had 64 failed= tests > and 99 > > skipped ones; let's get those down to 0 and something small, > respectively. > > > > > https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/3880/testRep= ort/ > > > > Would anybody be interested to review my changes to the test > suite?=C2=A0 It's > > not terrible if I have to commit without review; I can't possibly= > break > > anything but the test suite itself. > > > > -Alan > > _______________________________________________ > > freebsd-testing@freebsd.org > mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > > To unsubscribe, send any mail to > "freebsd-testing-unsubscribe@freebsd.org > " > > >=20 > Have you followed the work here: >=20 > https://github.com/zfsonfreebsd/ZoF >=20 > That has ported the improved ZFS test suite from ZFSonLinux back to= > FreeBSD (as part of the effort to upstream FreeBSD support into tha= t > newer version of ZFS) >=20 > --=20 > Allan Jude >=20 >=20 > Oh, boy, I didn't know about that fork of the test suite.=C2=A0 When I = made > FreeBSD's, I didn't try for portability.=C2=A0 Merging the two will not= be > easy.=C2=A0 There are > 60,000 lines of diffs between the two, plus=C2=A0= > 1,000 > files that don't exist on both sides.=C2=A0 Worse, there are some major= > structural differences: > 1) FreeBSD's test suite runs as root and drops privileges selectively, > but ZoL's runs as an unprivileged user and uses sudo selectively to gai= n > privileges > 2) FreeBSD's test suite uses ATF, just like everything else, but ZoL's > test suite uses a bespoke python program > 3) FreeBSD's test suite runs setup and cleanup once for every test file= , > but ZoL does it once for every directory of test files. > 4) FreeBSD's test suite uses FreeBSD make, but ZoL's uses autotools and= > gmake >=20 > Of those item 3 is the most troubling.=C2=A0 I wonder if ZoL could be > convinced to switch to ATF and Kyua?=C2=A0 They're pretty portable. > -Alan Yeah, so I think the ZoF project is just porting the ZoL test suite to FreeBSD, not necessarily pushing anything from ours back. It'd be good to get to keep more of our tests. The goal was really to be able to take advantage of the existing CI that ZFSonLinux has, and to require all future pull-requests against the new OpenZFS repo require passing the ZTS on both OSes to be able to be merged= =2E So the more FreeBSD stuff we can get in there the better. --=20 Allan Jude --W7qAUZkZLkagLrlWudYhU4Jm09wxB7RqS-- --CHM4sQDLgVZ844ABGFxz9tUOLDN7pBrMO Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJdmo/zAAoJEBmVNT4SmAt+nrkQAJ7YYx1f0MM0PthkZmNMWKr+ 52NUMgAILbYkPW10Ya8xbM46yd+WQ/DvXHAzxh0/YmGxNi+BVlPkUDgRjLFqPX0H jJ0vzyVDnKQycN9tJhOpcMDTlMbWyYxiEVRCWtXKDCFbfJBUJZFPHl+H2CH6/HxK JGGczAdOr4T87opxGhVD4SAyjVNgrq8ONlWNQVC3tv3Cx86nahFz/OnKT3RUqAHD 8RLKWAP1XhWHKDCkOiZtjqrAAPX9Yp4FnNVc/iwyuKGdPHOpyXE9EPiwfxd7+5zf aC5KlJlacmdRKxkzqpdyLhiwSVI9ZUuOPO/u7B+PhWWzVAFn1WI8zxe95Tv0kxyn BrkhMIku36ygL87OdM88VaYrRLGz4aPcb6Op5189+iG1hD2PHgI6WwBG1sGf8OTv QWU2raFUgBcrDihEhv9ZHbNhcTEn0H1QkrfUowN0itk1U43FAreVgAsYzWO486Dw 3D82zD2td1f2ZuWgOAEK6D7LLllDKe5YXJC+ef/J3aRUh0azILJvMhjF+sI8XHY7 R1k+4n3OIKhKTTRqOK7T/sc+5XvrDJwiyTQ9Th87OA7JyMHwSPt57Trxpp8auwZ1 xWWJgYcVQjtzgXSKEZi714OQla1yI7ZLCNkHjl8tz66CmyTMU4JCnGElGNQx7xkM Ibrl1j6MmBC72oKbH3De =cGzo -----END PGP SIGNATURE----- --CHM4sQDLgVZ844ABGFxz9tUOLDN7pBrMO--