From owner-svn-src-head@FreeBSD.ORG Thu Feb 12 21:12:36 2015 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF18D33C; Thu, 12 Feb 2015 21:12:36 +0000 (UTC) Received: from mail-ig0-x22b.google.com (mail-ig0-x22b.google.com [IPv6:2607:f8b0:4001:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 981E8D25; Thu, 12 Feb 2015 21:12:36 +0000 (UTC) Received: by mail-ig0-f171.google.com with SMTP id h15so6564844igd.4; Thu, 12 Feb 2015 13:12:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=9gAItp56aNt5gwvRMfLqPifFdvexFEJ+Ywvb8nRD37I=; b=x5W15BUgCXlLfMjlwZaU+fC5rQ5Q2oMnO1l6XcV1+znAvGd9e1Nbmp6tQ1lMFhYnrj mGXgZnMAUPBQvqMSLwoTVOuGzcPM4Xgi1/ne3skJT9o48IhleIuL2Y4/MYmfWU3I5YIi EdE3Vv4jzS+F3d5VMJaAnDf5GmMbjSQzz/Ym5A0RvSGwPUhql8pOBPO+0hHBFsYvotkm bakPCH8VEXykLDkr20sJUmlvZ1MmQAl43SHwaxaed5PxAN5i4s4+Ar1s8uLDioB794mu OZCFmem8CodmclXt3+OvBsdP8Y/slH1KLJ9gFv8MjptuGVQmQZjc7pBH1kxXxvNpFPOa Z5jQ== X-Received: by 10.42.77.9 with SMTP id g9mr10917405ick.78.1423775556112; Thu, 12 Feb 2015 13:12:36 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:5de4:da0f:932c:57df? ([2601:8:ab80:7d6:5de4:da0f:932c:57df]) by mx.google.com with ESMTPSA id sd7sm1894115igb.20.2015.02.12.13.12.35 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 12 Feb 2015 13:12:35 -0800 (PST) Content-Type: multipart/signed; boundary="Apple-Mail=_2D9F087C-03E5-4A8D-AC83-1C2FBCA6D12D"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: svn commit: r278323 - in head: etc/rc.d usr.sbin/jail From: Garrett Cooper In-Reply-To: Date: Thu, 12 Feb 2015 13:12:33 -0800 Message-Id: References: <201502061754.t16HssXU042750@svn.freebsd.org> <343803A3-CFA3-4766-8294-139A6D8C0235@gmail.com> To: James Gritton X-Mailer: Apple Mail (2.1878.6) Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers , jenkins-admin@freebsd.org X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Feb 2015 21:12:37 -0000 --Apple-Mail=_2D9F087C-03E5-4A8D-AC83-1C2FBCA6D12D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Feb 9, 2015, at 18:51, James Gritton wrote: > On 2015-02-06 22:23, Garrett Cooper wrote: >> On Feb 6, 2015, at 18:38, James Gritton wrote: >>> On 2015-02-06 19:23, Garrett Cooper wrote: >>>> I think you broke the Jenkins tests runs, and potentially jail = support >>>> in some edgecases: >>>> https://jenkins.freebsd.org/job/FreeBSD_HEAD-tests2/651/ >>> Where do I go from here? There error you refer to certainly seems = jail-related, which leads me to guess at something disconnected between = the matching rc.d/jail and jail(8) change (i.e. using the new rc file = with the old jail program). But that's really just a wild guess. Is = there somewhere I look for more information? For example, where does = Jenkins actually do its thing? >>> Sorry for being so stupid in this - Jenkins has only been on the = very edge of my awareness until now. >> I honestly don=92t think it=92s Jenkins because Jenkins runs in = bhyve. I >> think you accidentally broke option handling in the jail = configuration >> (please see my other reply about added =93break;=94 statements). >> ... >> You can verify your changes by doing: >> % (cd /usr/tests/bin/pkill; sudo kyua test) >=20 > After some testing and looking around, I've decided the problem = definitely isn't in rc.d where I thought it might be. I've also decided = it's probably not in my patch either. >=20 > I've run this kyua test on a 10 system (don't have current handy for = such things at the moment), and sometimes I would see a failure and = sometimes I wouldn't. This was whether I was using the new or old jail = code. Later in the day, when the box was less loaded, it seemed to = always pass. Looking at the pkill-j_test script, I see jails being = created with sleep commands both inside and outside the jail around its = creation. I'm guessing this script is very sensitive to timing issues = that could be cause by (among other things) system load. The jail = commands in this script were also very simple, with the only parameters = used being: path, name, ip4.addr, and command. This isn't some kind of = esoteric exercising of the jail(8) options, and I would expect if it = works at one time it would work at another. I've "hand-run" these = particular jail commands and couldn't get them to fail (and the actual = content of the jail(8) changes were tests already). >=20 > I looked at the freebsd-current (I think) list where the Jenkins = errors are posted, and it's true it started failing the pkill-j test at = the time I made my change. But it's also true that it had failed that = test once the day before my change, and then started passing it again. = This particular test just seems to be fragile. >=20 > So I don't have anywhere else to go with this. I'm going to assume = jail(8) isn't the problem here. The tests are racy and make some interesting assumptions. It appears = that WITNESS plays a part in it, and I bet VIMAGE (something that I = don=92t have in my kernel config) plays a part in it too. I say this = because I just ran into the issue when running the tests in a tight loop = on my VMware workstation 7 instance with code from r278636. Doesn=92t surprise me because before r272305, it was failing = consistently on head, so what Craig did in that commit helped, but it = didn=92t fully fix the raciness of the tests. I=92m going to recompile my system with VIMAGE and see if that impacts = performance of the tests, and if so, I=92ll adjust the sleep between = setting up the jailed instances, and waiting for them to be fully = formed. Thanks! $ while : ; do sudo prove -rv pgrep-j_test.sh || break; done pgrep-j_test.sh ..=20 1..3 usage: pgrep [-LSfilnoqvx] [-d delim] [-F pidfile] [-G gid] [-M core] = [-N system] [-P ppid] [-U uid] [-c class] [-g pgrp] [-j jid] [-s sid] [-t tty] [-u euid] pattern ... not ok 1 - pgrep -j # pgrep output: '', pidfile output: '74275 = 74278' ok 2 - pgrep -j any ok 3 - pgrep -j none Failed 1/3 subtests=20 Test Summary Report ------------------- pgrep-j_test.sh (Wstat: 0 Tests: 3 Failed: 1) Failed test: 1 Files=3D1, Tests=3D3, 5 wallclock secs ( 0.04 usr 0.02 sys + 0.02 = cusr 0.55 csys =3D 0.63 CPU) Result: FAIL --Apple-Mail=_2D9F087C-03E5-4A8D-AC83-1C2FBCA6D12D Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJU3RdBAAoJEMZr5QU6S73eKkUH/2HVmt2Mrte77j6J5Fj5fLsa VxbF+QHPmEhhLShRfJ7XC166WMZdt5Weh2sDbHJ8FgMVny6yYvAbCms5+eoPXgZp /6niDYG8maeUEczCY9PY6gqImmL4EXn1YpQgP4s+Mu5m44HIIZLL5MnZvrbaWFOw WZll/O6eIXow/NRpw+ra6G3mmkC7eDg8IVq0kA2vIjI0c0WdZM53rIf0RQs/5txX GLxyWIWVU6VzNO6jGlLFFDa1dzJE4otNqLJuOTHtHWLmdT0lEnCqntUizPdFL9Gh q/2VG+jptijtOqaGLCompp5hICAoL240WmXnmWhU8xDsshO5ZcHo17Nidb0GCIY= =q2EA -----END PGP SIGNATURE----- --Apple-Mail=_2D9F087C-03E5-4A8D-AC83-1C2FBCA6D12D--