From owner-freebsd-testing@freebsd.org Fri Sep 11 01:03:46 2020 Return-Path: Delivered-To: freebsd-testing@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7DD263E8968 for ; Fri, 11 Sep 2020 01:03:46 +0000 (UTC) (envelope-from lg49152@outlook.com) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2020.outbound.protection.outlook.com [40.92.19.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bncvh4vlYz4sdm for ; Fri, 11 Sep 2020 01:03:44 +0000 (UTC) (envelope-from lg49152@outlook.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ON3eOA4QaZZb56WqiA40xAg3W/b7lb9ptx5rbxqWgQ8/zz7NAfKsqe2EZOSQuUT18Rv9Padt9AuauXfcr55EkmVNIAkop0/msaA4424N9FiqGfCkeUJidy9DxaunlA4fTzQb7dC78CCJbJ7SDtTLNeeSeI8td4EPRLPkgvH/AE+OguB3odDIMrEZEqBAGvJVhPdmucxTtAHMbbgJaWD7hvFdjODa8hCunt+neQI+Jw8sWXVGUJaa3ls2epyQvpdjMJBZwHBKJUSZ1QzHMvNIZ6V6sV+EaInjlgnM0JB9/ziqIJP6B5H3RgTGBpWS2i0CVutbCRB8F7GOAG14eHJl4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8tqol931xDIomd8FNXQE0Abfz2jsjIObmCI+XZNHyIo=; b=NDy90TUDk+8bk0pOt+pULYxaq58Pz13bXs8ldiAOihFfh13L3bvmOdPrvNLwfsz5OI1aXTV8TZIUik7Vj5RfUig9DPfdWF86IYcZ9tcT3Sf+SR7NZt1Nh/fHNI2bHNy3avs2rzG8YDFbC1IQSKf6fairhsvQ/zP6DLHgK7tLceTJK7BOkLIqYQW6cQYpunPYn2c33pijxiITF17vzbrMTmzw95T+TdrT0wPf37VM18ui9V+MU2G+vjS9QrSNlyoBTz90CbU60btfncxOYXp8hgy4Bm3vwbrpOtC9Ggc1CERhR4U8dEURZ97hiiuQb8QFOxKVll5l64OnHGdh12Nz5g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8tqol931xDIomd8FNXQE0Abfz2jsjIObmCI+XZNHyIo=; b=prja7rMFkXbQaNkVUj/9sZGLg2woUkDsGXhZd9+cbW2CDfcomAyRF8+4k2q1AeuaqAvrigitGI+p3eBx2xYqXeCRTSjGvSUAL7Lhh7Ww/lxqfeMAxVFDArRhOuryt9cyjZq7oHxoE2ZiFkeTBAoEJOof/KBJdXFXU804Nj+ptpXsfOtQYMXWPZI8pf7WUm8Con51/XkEe/xnZzZmCI4s430PSFsc7tbRv8rbEuIu/mXQbrAOLlho7D/ezY3xBCEQ5km3VHrWcrkkvqtLceRJYerAsw0PzDKxEu0Bo0LaPlfJSjstOHf9PQxz8nO5i8odHX8BmV+3gQyMxG7Xdyt4bw== Received: from BN8NAM11FT054.eop-nam11.prod.protection.outlook.com (2a01:111:e400:fc4b::42) by BN8NAM11HT020.eop-nam11.prod.protection.outlook.com (2a01:111:e400:fc4b::414) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3370.16; Fri, 11 Sep 2020 01:03:43 +0000 Received: from DM6PR04MB5484.namprd04.prod.outlook.com (2a01:111:e400:fc4b::42) by BN8NAM11FT054.mail.protection.outlook.com (2a01:111:e400:fc4b::358) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3370.16 via Frontend Transport; Fri, 11 Sep 2020 01:03:43 +0000 Received: from DM6PR04MB5484.namprd04.prod.outlook.com ([fe80::e927:93b0:247b:ae50]) by DM6PR04MB5484.namprd04.prod.outlook.com ([fe80::e927:93b0:247b:ae50%5]) with mapi id 15.20.3348.019; Fri, 11 Sep 2020 01:03:43 +0000 From: Lawrence Greening To: "freebsd-testing@freebsd.org" Subject: Developers' Handbook: Chapter on testing Thread-Topic: Developers' Handbook: Chapter on testing Thread-Index: AQHWh9SRuCiD5IclKUayMIdlHK/L+w== Date: Fri, 11 Sep 2020 01:03:43 +0000 Message-ID: Accept-Language: en-CA, en-US Content-Language: en-CA X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:6E1BB85419DF58CC926A0FA5679FF04973E45AE700509C442004806A41816683; UpperCasedChecksum:80FC1057036B56AB35772A3E5A51B247D8AFD00C05E070C7774F9ED78E6C3E64; SizeAsReceived:6699; Count:42 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [oVACv3Nv4JsrbzDA2LvBs8Gbb3LYV4xH] x-ms-publictraffictype: Email x-incomingheadercount: 42 x-eopattributedmessage: 0 x-ms-office365-filtering-correlation-id: cd9381a8-75ad-4bb3-06b6-08d855ee8726 x-ms-traffictypediagnostic: BN8NAM11HT020: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5zjhw3FM+EUtH1AbwTnotZIriOeZFxLiIvicqrYYVQaWpHz/iK2JOz2oHFdXWR7/4dRYvUHlrGwPhXrkEGSpmOsBHW0ZxAE/Lfy5U6ymIoP84tOO54WUF/KUADQu+/5eXQCk3MhI/ubrBsbFeH5qemxb2t0OFE+tq9jGOQHOLBEa9kmyVyML6/IvlJbyyBlCcL/Ob56Ayu9d2Eg0katAGA== x-ms-exchange-antispam-messagedata: IW3lkfdDQOsz2hG+WpjDjJ7/IH+zG3okvLdy+odtvtQyKVPlCfNbKVfC/gjejoszJRdcTdvvJAM4yLvjeHH021OunaMTIWVcMZtEh1SmoIRiqlG7M+9cNAraZKDf2OWP8T1PgJPYcM124wh1L51Irw== x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT054.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: cd9381a8-75ad-4bb3-06b6-08d855ee8726 X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Sep 2020 01:03:43.0429 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8NAM11HT020 X-Rspamd-Queue-Id: 4Bncvh4vlYz4sdm X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=outlook.com header.s=selector1 header.b=prja7rMF; dmarc=pass (policy=none) header.from=outlook.com; spf=pass (mx1.freebsd.org: domain of lg49152@outlook.com designates 40.92.19.20 as permitted sender) smtp.mailfrom=lg49152@outlook.com X-Spamd-Result: default: False [-4.50 / 15.00]; NEURAL_HAM_MEDIUM(-0.95)[-0.954]; R_DKIM_ALLOW(-0.20)[outlook.com:s=selector1]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.92.19.20:from]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[outlook.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; R_SPF_ALLOW(-0.20)[+ip4:40.92.0.0/15]; NEURAL_HAM_LONG(-0.99)[-0.994]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[outlook.com:+]; DMARC_POLICY_ALLOW(-0.50)[outlook.com,none]; RCVD_IN_DNSWL_NONE(0.00)[40.92.19.20:from]; NEURAL_HAM_SHORT(-0.55)[-0.552]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[outlook.com]; ASN(0.00)[asn:8075, ipnet:40.80.0.0/12, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-testing]; DWL_DNSWL_NONE(0.00)[outlook.com:dkim] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Sep 2020 01:03:46 -0000 Hi. I am writing a chapter on testing for the FreeBSD Developers' Handbook. The following topics are expected to be covered: * FreeBSD Test Suite * Kyua * Automated Testing Framework (ATF) * Organization of current FreeBSD tests. * Using automation and virtualization to reduce the overhead in setting up = the test infrastructure. * Examples of standard types of testing, e.g. unit tests, functional testin= g, load testing, etc. * Reproducibility in building and testing. However, this list is just a starting point. The scope and organization wil= l evolve as the chapter is written. After reading the chapter, the reader should be able to do at least the fol= lowing: * Install and configure the necessary testing tools, test suites, etc. for = the purposes of testing FreeBSD. * Run tests. * Write tests. * Submit tests. The chapter will include walkthroughs and guided or cumulative examples. At this preliminary stage, I am interested in the following feedback: (1) What material do you want to see covered? (2) What material do you want to see prioritized? (3) What general approach(es) do you want to see taken towards the subject = matter? (4) Are there good example tests or areas of the code base that deserve foc= us, either because of their importance, usefulness as examples, history of = regressions, etc.? (5) Anything else? Cheers, Lawrence