From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 17:33:02 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 598B5EE9 for ; Mon, 3 Mar 2014 17:33:02 +0000 (UTC) Received: from mail-qc0-f181.google.com (mail-qc0-f181.google.com [209.85.216.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 19C60283 for ; Mon, 3 Mar 2014 17:33:01 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id c9so4001251qcz.40 for ; Mon, 03 Mar 2014 09:33:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=xYEoNP6CS7YVCKWosuYu0m0hbnMK+cXpbWWg52QdZ+s=; b=fk9Vsw9Jr7YMakgIldqaCVlKYMBrUbjIeyUftuprf21AOoJRa78EEp6fpznuKYNpSv MBuj7Oe3ETX/vALZnmUqDsNWogOPf8i2b4J1McZ4RkrhdBf9OMJeSvCgp6e6qPdRBDgj ODGANr2ORZuYn8YuCaN3csODpYbcxzMYszwtYSwwe/T7TEOfbXjkpVnYrlSREy9pCJLv 0+9Jw6kv6csUx60TUrFL0AJbZZcbcBYv6TRZswbd6yVwK29YevUgIjqU1X7u86agQNL2 4InLGapr+Zp14Z6EUHOCEbEuis8R0DTClH63oD/BmRyE3JFlvYf5O5qth7mpqr64707Q FM+w== X-Gm-Message-State: ALoCoQmJMiKoWyNg0yqq5jTf3lEXTD9J0IvigAaFGKSbPJ9CTvbvWgEa3wN+up7mje5QdWih/piZ X-Received: by 10.224.157.7 with SMTP id z7mr24972956qaw.37.1393867980741; Mon, 03 Mar 2014 09:33:00 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Mon, 3 Mar 2014 09:32:40 -0800 (PST) X-Originating-IP: [2620:0:1003:1007:69c9:8c4b:609d:3fea] In-Reply-To: <20140303165226.GK85115@kiwi> References: <20140303165226.GK85115@kiwi> From: Julio Merino Date: Mon, 3 Mar 2014 12:32:40 -0500 X-Google-Sender-Auth: nN26zLCXerK9bcPKMwBbn629KuM Message-ID: Subject: Re: atf-c for kernel function unit testing To: "R. Tyler Croy" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 17:33:02 -0000 On Mon, Mar 3, 2014 at 11:52 AM, R. Tyler Croy wrote: > > I'm wondering if anybody's explored using atf-c for adding unit tests to kernel > level functions or driver code. We do this extensively in NetBSD: we have a mechanism (rump) to build unmodified kernel-level code as user-space binaries and we later use atf to unit-test the code in controlled tiny environments. (You'd think about this as super-lightweight virtualization of the kernel.) Achieving this for any non-trivial kernel-level code is hard however, but I believe that this approach has huge benefits. For example: we are able to unit-test all file system and networking code in user-space without any chances of crashing the system; there is also the benefit of being able to control the test environment fully without relying on any state of the running kernel. For more details see: http://en.wikipedia.org/wiki/Anykernel http://wiki.netbsd.org/rumpkernel/ If the code you are interested in testing can easily be built without ties to the kernel, you can relatively easily do the above. (In fact, we used to do this in NetBSD for some tiny areas of the code before we got rump.) At some point I also pondered the idea to write an interface and a kernel module to allow writing unit tests within the kernel. This might be interesting for some special cases (like testing live hardware maybe) but it's a very fragile setup that should not be pursued if other safer solutions suffice.