From owner-freebsd-testing@FreeBSD.ORG Fri Dec 19 15:30:27 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4061E9E for ; Fri, 19 Dec 2014 15:30:27 +0000 (UTC) Received: from mail-qg0-f48.google.com (mail-qg0-f48.google.com [209.85.192.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F037C2C7F for ; Fri, 19 Dec 2014 15:30:26 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id f51so774379qge.21 for ; Fri, 19 Dec 2014 07:30:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=RpNjHEewIVv1sIXyUuLC+H4AsfwKf1ARDjOQvnGoSWI=; b=UWbgzWXza228Y8bhhGou6xbOlrdJ1Ao5pxqiEodHOSrnAR7Wh2U+q47c0/HFbxwMAl 7Lld6otXtYUYmstBW+Nh40gVRTDonwQ9c5PWZysDB/PyKD6EUKMgjwRZ4WW14Uyqzlw7 raicYacP6q5J4e1JdDW327S5u+464u7WZH3xeRSa41oqF67P2Qx9S9C5yhhUz/fti7Lm DK8D/OMCMGQExN+JSF63gr7cXSUe53LzNliLOlJy1/lvvzZOYczLnyOgZqTOjZa0cw2N vYgHIiLBdUvnMIznjanCbtwfkiHUtiYgDR7UjUwAEsdmggd2BV5dlFGVXPiMDQBkatNm Y4mg== X-Gm-Message-State: ALoCoQlIdBgkrP+7Vdee5CR4/1z6abpIf75gN86TVrHX2nttmGB7uEesbf3aWjaAaBrJKST7mrP8 X-Received: by 10.140.31.36 with SMTP id e33mr13861084qge.36.1419003019774; Fri, 19 Dec 2014 07:30:19 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.154.4 with HTTP; Fri, 19 Dec 2014 07:29:59 -0800 (PST) X-Originating-IP: [2620:0:1003:1007:51a6:ae44:98bd:55f] In-Reply-To: References: From: Julio Merino Date: Fri, 19 Dec 2014 10:29:59 -0500 X-Google-Sender-Auth: X1JYSG2IdTF2fWyAOyne7N4oKkY Message-ID: Subject: Re: Deltas between FreeBSD and NetBSD and POSIX conformance (was "[Bug 194828] [test] lib.libc.sys.getcontext_test.setcontext_link fails on amd64, not i386") To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Dec 2014 15:30:27 -0000 On Fri, Nov 14, 2014 at 12:34 PM, Garrett Cooper wrote: > > On Nov 14, 2014, at 2:28, Justin Cormack wrote: > >> On Fri, Nov 14, 2014 at 7:35 AM, wrote: >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194828 >>> >>> --- Comment #12 from Garrett Cooper,425-314-3911 --- >>> In a perfect world I would like for everything to be consistent between FreeBSD >>> and NetBSD, but that's not the way it should be, and that's a silly ideal to >>> hope for :). >> >> NetBSD dev here... I would rather that tests actually test behaviour >> as defined in standards, especially for eg libc tests, and indeed I >> have bunch of stuff to add more tests on standards compliance. So if >> stuff is testing implementation internals it should go away, and if >> NetBSDs behaviour is incorrect please file an issue, or if the test >> setup is not eg standards compliant please file an issue >> http://www.netbsd.org/cgi-bin/sendpr.cgi?gndb=netbsd That's a nice ideal... but in practice, the standard does sometimes leave details up to the implementation and/or the implementation provides custom extensions. Both need tests, and such tests will be implementation-dependent. So how could we deal with this? Tests for the standard should really be unified across both codebases, but there should be a way to supplement them with implementation-specific tests. Not sure what the best organization scheme for this would be, nor how we'd maintain the "common code".