From owner-freebsd-arch@FreeBSD.ORG Sun Dec 26 17:28:25 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A811C1065672 for ; Sun, 26 Dec 2010 17:28:25 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 2ABC38FC08 for ; Sun, 26 Dec 2010 17:28:24 +0000 (UTC) Received: from park.js.berklix.net (p5B22C6AD.dip.t-dialin.net [91.34.198.173]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id oBQHSKdF022912 for ; Sun, 26 Dec 2010 17:28:22 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id oBQHSPKa091577 for ; Sun, 26 Dec 2010 18:28:25 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id oBQHSK40032421 for ; Sun, 26 Dec 2010 18:28:25 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201012261728.oBQHSK40032421@fire.js.berklix.net> To: freebsd-arch@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Sat, 25 Dec 2010 20:17:44 +0100." Date: Sun, 26 Dec 2010 18:28:20 +0100 Sender: jhs@berklix.com Subject: Let's adopt a standard nomenclature for webs of patch trees etc. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Dec 2010 17:28:25 -0000 Was Subject: Re: Schedule for releases I changed it, as my reply takes it too far off that topic. Erik Cederstrand wrote: > Hi Mike, > Den 22/12/2010 kl. 18.45 skrev Mike Karels: > > > - Those of us doing backports could probably do a better job of > > sharing the results. On the other hand, I'm generally backporting > > to a specific release (currently 6.3 or 7.2) rather than -stable, > > and we're testing our software rather than FreeBSD. > > Thanks for taking the time to write your comments. What strikes me is = > that we may have lots of non-FreeBSD developers working to backport = > stuff in their own private trees. Possibly a lot of redundant work is = > being done. > > Even though you're backporting to a specific release, and even though = > you're only testing the work via your own software, would it not help = > others and possibly even yourself if this FreeBSD-specific work lands in = > the FreeBSD repository instead of your private tree? In my view you're = > just as much a FreeBSD developer as people with commit access that are = > scratching their own itches in CURRENT. > > Erik= Good point, Probably loads of fixes from non commiters never get sent back to FreeBSD. Many people will have motivation only to fix local problems, but no time to send back, especially deterred by clunky send-pr. Though I & many others have sent lots of send-pr, contributing even a spelling correction to FreeBSD is much harder than to eg http://wikipedia.org + a beginner has to bend their brain to send-pr + send-pr user should not be burdened exploring tree to find Maintainer to send-pr CC (which should be automaticly extracted from tree on a ports =MAINTAINER basis or eg a src/ .MAINTAINER per some sub directories where there is a volunteer or mail list) + send-pr user must spend time composing a diplomatic & attractive subject & body, to catch some gnats@ readers eye, to get them to stop browsing get interested, & commit. Many a potential contributor's attitude will be: I don't have time: Catch the diff or drop it, your loss ! So a lot of potential send-pr won't get filed, but I bet local users don't toss their fixes though, but keep local patch kits, till if ever they or others send-pr & something gets commited, (which might be days or years later). Those diff trees stored localy, users could easily export via rdist/rsync etc to their local webs, eg I do this: My diffs in a tree structure http://berklix.com/~jhs/src/bsd/fixes/FreeBSD My application script http://berklix.com/~jhs/bin/.csh/customise Those trees, FreeBSD could encourage users to keep in a standard format (path nomenclature etc) & we should reccomend, indexed from a common page on eg wiki.freebsd.org It would make a search tool &/or automatic periodic indexing for possible diffs so much better than any general purpose search engine. Index of uncommited patches ready for test, would be ideal for those currently stuck, & would assist more motivated testers corroborating good patches worth commiting. A standard format would increase chances patch kits are found, even if patch creator too busy to file send-pr etc. Let's adopt standards to make searches for potential patch trees easier: - Adopt a common path root & nomenclature for all our trees of local diffs, - Ask users to mirror local uncommited trees of diffs to thir local webs (until if when commited after send-pr, then they delete) - Ask authors of local patch kits to submit a single URL to a new wiki page, pointing to top automatically apply-able directory of patches Later we might also list a SOC project for a crawler indexer, - src/ directories could also Optionaly later adopt .MAINTAINER files (Subject of previous discussions, please dont let that distract from main proposal though) - ports/*/*/Makefile MAINTAINER = could also be used by a SOC tool Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, or HTML or base 64. Avoid top posting, it cripples itemised cumulative responses. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 26 20:42:35 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6BC91065673 for ; Sun, 26 Dec 2010 20:42:35 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 041DB8FC08 for ; Sun, 26 Dec 2010 20:42:34 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id oBQKgTbW062688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 26 Dec 2010 21:42:30 +0100 (CET) (envelope-from uqs@spoerlein.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1293396150; bh=0ycEfxLrUSMBP+/yDt8ENWJPt1nH4N/EayORkW2ytqw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=bCcW2FhhY2tKb2Uc8oUpizft7C56wDpXM0A0tB1Sf2XD5WX3tdsag2avC764877BO aADFG1cqvg/0/nVCZN2oe7apPfA7OklQfwb7ZGPmm4r94qs1ic+WBR06omwHiOQMR6 vwlbzP1X2fdRqB6D0jknFbZg7PPaH8p5NDWl5o7s= Date: Sun, 26 Dec 2010 21:42:29 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: "Julian H. Stacey" Message-ID: <20101226204229.GS23098@acme.spoerlein.net> Mail-Followup-To: "Julian H. Stacey" , freebsd-arch@freebsd.org References: <201012261728.oBQHSK40032421@fire.js.berklix.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201012261728.oBQHSK40032421@fire.js.berklix.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch@freebsd.org Subject: Re: Let's adopt a standard nomenclature for webs of patch trees etc. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Dec 2010 20:42:35 -0000 On Sun, 26.12.2010 at 18:28:20 +0100, Julian H. Stacey wrote: > Was Subject: Re: Schedule for releases > I changed it, as my reply takes it too far off that topic. > > Erik Cederstrand wrote: > > Hi Mike, > > Den 22/12/2010 kl. 18.45 skrev Mike Karels: > > > > > - Those of us doing backports could probably do a better job of > > > sharing the results. On the other hand, I'm generally backporting > > > to a specific release (currently 6.3 or 7.2) rather than -stable, > > > and we're testing our software rather than FreeBSD. > > > > Thanks for taking the time to write your comments. What strikes me is = > > that we may have lots of non-FreeBSD developers working to backport = > > stuff in their own private trees. Possibly a lot of redundant work is = > > being done. > > > > Even though you're backporting to a specific release, and even though = > > you're only testing the work via your own software, would it not help = > > others and possibly even yourself if this FreeBSD-specific work lands in = > > the FreeBSD repository instead of your private tree? In my view you're = > > just as much a FreeBSD developer as people with commit access that are = > > scratching their own itches in CURRENT. > > > > Erik= > > Good point, Probably loads of fixes from non commiters never get > sent back to FreeBSD. Many people will have motivation only to fix local > problems, but no time to send back, especially deterred by clunky send-pr. > > Though I & many others have sent lots of send-pr, > contributing even a spelling correction to FreeBSD > is much harder than to eg http://wikipedia.org > > + a beginner has to bend their brain to send-pr > > + send-pr user should not be burdened exploring tree to find > Maintainer to send-pr CC (which should be automaticly > extracted from tree on a ports =MAINTAINER basis > or eg a src/ .MAINTAINER per some sub directories > where there is a volunteer or mail list) > > + send-pr user must spend time composing a > diplomatic & attractive subject & body, to catch > some gnats@ readers eye, to get them to stop browsing > get interested, & commit. > > Many a potential contributor's attitude will be: I don't > have time: Catch the diff or drop it, your loss ! > > So a lot of potential send-pr won't get filed, but I bet local users > don't toss their fixes though, but keep local patch kits, till if > ever they or others send-pr & something gets commited, (which might > be days or years later). > > Those diff trees stored localy, users could easily export via > rdist/rsync etc to their local webs, eg I do this: > My diffs in a tree structure > http://berklix.com/~jhs/src/bsd/fixes/FreeBSD > My application script > http://berklix.com/~jhs/bin/.csh/customise > > Those trees, FreeBSD could encourage users to keep in a standard > format (path nomenclature etc) & we should reccomend, > indexed from a common page on eg wiki.freebsd.org > > It would make a search tool &/or automatic periodic indexing > for possible diffs so much better than any general purpose > search engine. > > Index of uncommited patches ready for test, would be ideal > for those currently stuck, & would assist more motivated > testers corroborating good patches worth commiting. > > A standard format would increase chances patch kits are found, > even if patch creator too busy to file send-pr etc. > > Let's adopt standards to make searches for potential patch trees easier: > - Adopt a common path root & nomenclature for all our trees of local diffs, > - Ask users to mirror local uncommited trees of diffs to thir local webs > (until if when commited after send-pr, then they delete) > - Ask authors of local patch kits to submit a single URL to a new wiki page, > pointing to top automatically apply-able directory of patches > > Later we might also list a SOC project for a crawler indexer, > - src/ directories could also Optionaly later adopt > .MAINTAINER files (Subject of previous discussions, please dont let that > distract from main proposal though) > - ports/*/*/Makefile MAINTAINER = could also be used by a SOC tool While this idea is good as a base, doing this with patch-trees is the worst possible move. Patchfiles lack comments or 'commit info' and they do not easily record the revision and branch they should be applied to. Stacking multiple patches together with comments on what they do, is exactly what revision control systems were made for. And while we cannot easily share svn access to random contributors, systems like git or mercurial are exactly what we need here. In other words, we need github for FreeBSD. I'm working on some basics for this at repos.freebsd.your.org, but had severe VM instabilities during the last weeks. Regards, Uli From owner-freebsd-arch@FreeBSD.ORG Sun Dec 26 21:11:03 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 340B21065679 for ; Sun, 26 Dec 2010 21:11:03 +0000 (UTC) (envelope-from roam@ringlet.net) Received: from praag.hoster.bg (praag.hoster.bg [77.77.142.10]) by mx1.freebsd.org (Postfix) with ESMTP id 956928FC08 for ; Sun, 26 Dec 2010 21:11:02 +0000 (UTC) Received: from middenheim.hoster.bg (middenheim.hoster.bg [77.77.142.11]) by praag.hoster.bg (Postfix) with ESMTP id EBE678C9E8 for ; Sun, 26 Dec 2010 22:50:44 +0200 (EET) Received: from straylight.ringlet.net (unknown [94.155.53.142]) (Authenticated sender: roam@hoster.bg) by mail.hoster.bg (Postfix) with ESMTP id 5F3655C3D5 for ; Sun, 26 Dec 2010 22:50:34 +0200 (EET) Received: from roam (uid 1000) (envelope-from roam@ringlet.net) id 416007 by straylight.ringlet.net (DragonFly Mail Agent) Sun, 26 Dec 2010 22:50:33 +0200 Date: Sun, 26 Dec 2010 22:50:33 +0200 From: Peter Pentchev To: "Julian H. Stacey" , freebsd-arch@freebsd.org Message-ID: <20101226205033.GA4135@straylight.ringlet.net> Mail-Followup-To: "Julian H. Stacey" , freebsd-arch@freebsd.org References: <201012261728.oBQHSK40032421@fire.js.berklix.net> <20101226204229.GS23098@acme.spoerlein.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="82I3+IH0IqGh5yIs" Content-Disposition: inline In-Reply-To: <20101226204229.GS23098@acme.spoerlein.net> User-Agent: Mutt/1.5.20 (2009-06-14) X-MailScanner-ID: 5F3655C3D5.71013 X-hoster-MailScanner: Found to be clean X-hoster-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=0.001, required 10, autolearn=disabled, UNPARSEABLE_RELAY 0.00) X-hoster-MailScanner-From: roam@ringlet.net X-hoster-MailScanner-To: freebsd-arch@freebsd.org X-Spam-Status: No Cc: Subject: Re: Let's adopt a standard nomenclature for webs of patch trees etc. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Dec 2010 21:11:03 -0000 --82I3+IH0IqGh5yIs Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Dec 26, 2010 at 09:42:29PM +0100, Ulrich Sp=C3=B6rlein wrote: > On Sun, 26.12.2010 at 18:28:20 +0100, Julian H. Stacey wrote: > > Was Subject: Re: Schedule for releases > > I changed it, as my reply takes it too far off that topic. > >=20 > > Erik Cederstrand wrote: > > > Hi Mike, > > > Den 22/12/2010 kl. 18.45 skrev Mike Karels: > > >=20 > > > > - Those of us doing backports could probably do a better job of > > > > sharing the results. On the other hand, I'm generally backporting > > > > to a specific release (currently 6.3 or 7.2) rather than -stable, > > > > and we're testing our software rather than FreeBSD. > > >=20 > > > Thanks for taking the time to write your comments. What strikes me is= =3D > > > that we may have lots of non-FreeBSD developers working to backport = =3D > > > stuff in their own private trees. Possibly a lot of redundant work is= =3D > > > being done. > > >=20 > > > Even though you're backporting to a specific release, and even though= =3D > > > you're only testing the work via your own software, would it not help= =3D > > > others and possibly even yourself if this FreeBSD-specific work lands= in =3D > > > the FreeBSD repository instead of your private tree? In my view you'r= e =3D > > > just as much a FreeBSD developer as people with commit access that ar= e =3D > > > scratching their own itches in CURRENT. > > >=20 > > > Erik=3D > >=20 > > Good point, Probably loads of fixes from non commiters never get > > sent back to FreeBSD. Many people will have motivation only to fix loc= al > > problems, but no time to send back, especially deterred by clunky send-= pr. > >=20 > > Though I & many others have sent lots of send-pr, =20 > > contributing even a spelling correction to FreeBSD > > is much harder than to eg http://wikipedia.org > > =09 > > + a beginner has to bend their brain to send-pr=20 > > =09 > > + send-pr user should not be burdened exploring tree to find=20 > > Maintainer to send-pr CC (which should be automaticly > > extracted from tree on a ports =3DMAINTAINER basis > > or eg a src/ .MAINTAINER per some sub directories > > where there is a volunteer or mail list) > > =09 > > + send-pr user must spend time composing a > > diplomatic & attractive subject & body, to catch > > some gnats@ readers eye, to get them to stop browsing > > get interested, & commit. > > =09 > > Many a potential contributor's attitude will be: I don't > > have time: Catch the diff or drop it, your loss ! > >=20 > > So a lot of potential send-pr won't get filed, but I bet local users > > don't toss their fixes though, but keep local patch kits, till if > > ever they or others send-pr & something gets commited, (which might > > be days or years later). > >=20 > > Those diff trees stored localy, users could easily export via > > rdist/rsync etc to their local webs, eg I do this: > > My diffs in a tree structure > > http://berklix.com/~jhs/src/bsd/fixes/FreeBSD > > My application script > > http://berklix.com/~jhs/bin/.csh/customise > >=20 > > Those trees, FreeBSD could encourage users to keep in a standard > > format (path nomenclature etc) & we should reccomend, > > indexed from a common page on eg wiki.freebsd.org > >=20 > > It would make a search tool &/or automatic periodic indexing > > for possible diffs so much better than any general purpose > > search engine. > >=20 > > Index of uncommited patches ready for test, would be ideal > > for those currently stuck, & would assist more motivated > > testers corroborating good patches worth commiting. > >=20 > > A standard format would increase chances patch kits are found, > > even if patch creator too busy to file send-pr etc. > >=20 > > Let's adopt standards to make searches for potential patch trees easier: > > - Adopt a common path root & nomenclature for all our trees of local di= ffs, > > - Ask users to mirror local uncommited trees of diffs to thir local webs > > (until if when commited after send-pr, then they delete) > > - Ask authors of local patch kits to submit a single URL to a new wiki = page, > > pointing to top automatically apply-able directory of patches > >=20 > > Later we might also list a SOC project for a crawler indexer, > > - src/ directories could also Optionaly later adopt=20 > > .MAINTAINER files (Subject of previous discussions, please dont let t= hat > > distract from main proposal though) > > - ports/*/*/Makefile MAINTAINER =3D could also be used by a SOC tool >=20 > While this idea is good as a base, doing this with patch-trees is the > worst possible move. Patchfiles lack comments or 'commit info' and they > do not easily record the revision and branch they should be applied to. >=20 > Stacking multiple patches together with comments on what they do, is > exactly what revision control systems were made for. And while we cannot > easily share svn access to random contributors, systems like git or > mercurial are exactly what we need here. >=20 > In other words, we need github for FreeBSD. I'm working on some basics > for this at repos.freebsd.your.org, but had severe VM instabilities > during the last weeks. I have to admit that this crossed my mind as soon as I read Julian's e-mail, especially as I've been keeping my local FreeBSD patches in a version-controlled tree for the past ten years - first CVS, then Subversion, and recently Git. Now, is there a reason we couldn't just use Gitorious? :) G'luck, Peter --=20 Peter Pentchev roam@space.bg roam@ringlet.net roam@FreeBSD.org PGP key: http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 If the meanings of 'true' and 'false' were switched, then this sentence wou= ldn't be false. --82I3+IH0IqGh5yIs Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iQIcBAEBCAAGBQJNF6qVAAoJEGUe77AlJ98TeacQAJQEVJJqR/3Lrx/ksypGOgLL sL8PR8bPEHxur6IBanpmhGuDidxym9xhj/VSsxqAiAEIiOKfX1tsE+fOLcA7Ygtg pusQTfhJD4KOCS86aFucXGL0r2gAEKkPUyJAwfORiZSaQsDjfVWKClEvZQNuBEOv wk4DeeZsPcKBJCTDiplF/MJLTLgPTHQT30Xjsq2Ci9/f2atqD630v+GicywRo1Ha jx9BF4wOF4o/1+XImB/eRWy8ZGAoHAwiFioFQwWSncEVbTPbnrHhiOZGBs2Ad57S iaD+qEDRrD+Pc/fOcrHEKxE6frk1dskl9TOYUGWXh/XOhrCA4B0I+J8Q7elTCsrB FFrsO2OTZOzWwTzSF/zLMsx8jAA/Flk7N86Fz5mf566P0GMn4dd/XR6oZXbAMasy FJOPi7SllmQeVwLERwwQgxKCVIUB2ofYN+76jMxM11qbpXlBD5wipMV/nOAULrSg vvj2Mi4d4aunUPy8J2AOYfc3RpqPRmDALXGz+g2ExFii2qN6qYxvrrxUa3PZHIu1 N1d5yZM5Mm0NoiJJtQOlSi51NKydL5wcNvpPKg4XWJXz3H7b+4fNXKY/D4O+lkBr jhC9OvWV3vzx/ME34GVjJAP3LT7E1ArWz0kyGQYEU5jzthPJd27L7vO+ftLJziVA yQRepT6Dh6BtD6SdIFCQ =o6Cb -----END PGP SIGNATURE----- --82I3+IH0IqGh5yIs-- From owner-freebsd-arch@FreeBSD.ORG Mon Dec 27 12:42:43 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9914B106566C for ; Mon, 27 Dec 2010 12:42:43 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 25B238FC19 for ; Mon, 27 Dec 2010 12:42:42 +0000 (UTC) Received: by wyf19 with SMTP id 19so8384224wyf.13 for ; Mon, 27 Dec 2010 04:42:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=G4n37NFPRmueH9T5Vi7hjMvNaQnNTg0DrjL/BeVg0sU=; b=OkP99bMJ4JWy4Ki41oivnTzGhSIX0klHSpcmiO6+ZAN05wNzhZ+z9l3VByX+5lNhjA flj0fQRps6IM9arHGtGeBXT0x6qSj0uLANDJVaQfsJzOat+F+gttK0tfXWf4O7kMBeXV 9EuwGk4Rr4iB6jcEGe+pqPjp6kGpJsia0coUk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=k07E58EZe3RTYzgP3RPHw0a55H3yzd/42RfcnECiJMcrv5f5o1TXZgQ3OUFdwPhYO7 lAhavL3t9JusZcmdZmwoSdvFMnK4WfVpbRHCKcJ3yq6jEeU20R/GS2+UEV9OgPVHpGai KE0XqCtQZfMWON/q//nIhRc0mZBYs8ay5etpE= MIME-Version: 1.0 Received: by 10.216.180.76 with SMTP id i54mr4933874wem.33.1293453761705; Mon, 27 Dec 2010 04:42:41 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.173.213 with HTTP; Mon, 27 Dec 2010 04:42:41 -0800 (PST) In-Reply-To: <20101226205033.GA4135@straylight.ringlet.net> References: <201012261728.oBQHSK40032421@fire.js.berklix.net> <20101226204229.GS23098@acme.spoerlein.net> <20101226205033.GA4135@straylight.ringlet.net> Date: Mon, 27 Dec 2010 20:42:41 +0800 X-Google-Sender-Auth: JIJskRoQPlc7-Xr5OBHpyPVcmuo Message-ID: From: Adrian Chadd To: "Julian H. Stacey" , freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: Let's adopt a standard nomenclature for webs of patch trees etc. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Dec 2010 12:42:43 -0000 It occasionally goes down. I/Bernard are sort of using it at the moment for our wireless hacking. I've not yet really had a shot at using it for collaborative work or trying to track -head, so I can't comment on that. Adrian On 27 December 2010 04:50, Peter Pentchev wrote: > On Sun, Dec 26, 2010 at 09:42:29PM +0100, Ulrich Sp=F6rlein wrote: >> On Sun, 26.12.2010 at 18:28:20 +0100, Julian H. Stacey wrote: >> > Was =A0Subject: Re: Schedule for releases >> > I changed it, as my reply takes it too far off that topic. >> > >> > Erik Cederstrand wrote: >> > > Hi Mike, >> > > Den 22/12/2010 kl. 18.45 skrev Mike Karels: >> > > >> > > > - Those of us doing backports could probably do a better job of >> > > > sharing the results. =A0On the other hand, I'm generally backporti= ng >> > > > to a specific release (currently 6.3 or 7.2) rather than -stable, >> > > > and we're testing our software rather than FreeBSD. >> > > >> > > Thanks for taking the time to write your comments. What strikes me i= s =3D >> > > that we may have lots of non-FreeBSD developers working to backport = =3D >> > > stuff in their own private trees. Possibly a lot of redundant work i= s =3D >> > > being done. >> > > >> > > Even though you're backporting to a specific release, and even thoug= h =3D >> > > you're only testing the work via your own software, would it not hel= p =3D >> > > others and possibly even yourself if this FreeBSD-specific work land= s in =3D >> > > the FreeBSD repository instead of your private tree? In my view you'= re =3D >> > > just as much a FreeBSD developer as people with commit access that a= re =3D >> > > scratching their own itches in CURRENT. >> > > >> > > Erik=3D >> > >> > Good point, Probably loads of fixes from non commiters never get >> > sent back to FreeBSD. =A0Many people will have motivation only to fix = local >> > problems, but no time to send back, especially deterred by clunky send= -pr. >> > >> > =A0 =A0 Though I & many others have sent lots of send-pr, >> > =A0 =A0 contributing even a spelling correction to FreeBSD >> > =A0 =A0 is much harder than to eg http://wikipedia.org >> > >> > =A0 =A0 + a beginner has to bend their brain to send-pr >> > >> > =A0 =A0 + send-pr user should not be burdened exploring tree to find >> > =A0 =A0 =A0 =A0 =A0 =A0 Maintainer to send-pr CC (which should be auto= maticly >> > =A0 =A0 =A0 =A0 =A0 =A0 extracted from tree on a ports =3DMAINTAINER b= asis >> > =A0 =A0 =A0 =A0 =A0 =A0 or eg a src/ .MAINTAINER per some sub director= ies >> > =A0 =A0 =A0 =A0 =A0 =A0 where there is a volunteer or mail list) >> > >> > =A0 =A0 + send-pr user must spend time composing a >> > =A0 =A0 =A0 =A0 =A0 =A0 diplomatic & attractive subject & body, to cat= ch >> > =A0 =A0 =A0 =A0 =A0 =A0 some gnats@ readers eye, to get them to stop b= rowsing >> > =A0 =A0 =A0 =A0 =A0 =A0 get interested, & commit. >> > >> > =A0 =A0 Many a potential contributor's attitude will be: I don't >> > =A0 =A0 have time: Catch the diff or drop it, your loss ! >> > >> > So a lot of potential send-pr won't get filed, but I bet local users >> > don't toss their fixes though, but keep local patch kits, till if >> > ever they or others send-pr & something gets commited, (which might >> > be days or years later). >> > >> > =A0 =A0 Those diff trees stored localy, users could easily export via >> > =A0 =A0 rdist/rsync etc to their local webs, eg I do this: >> > =A0 =A0 =A0 =A0 =A0 =A0 My diffs in a tree structure >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 http://berklix.com/~jhs/src/bs= d/fixes/FreeBSD >> > =A0 =A0 =A0 =A0 =A0 =A0 My application script >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 http://berklix.com/~jhs/bin/.c= sh/customise >> > >> > =A0 =A0 Those trees, FreeBSD could encourage users to keep in a standa= rd >> > =A0 =A0 format (path nomenclature etc) =A0& we should reccomend, >> > =A0 =A0 indexed from a common page on eg wiki.freebsd.org >> > >> > =A0 =A0 It would make a search tool &/or automatic periodic indexing >> > =A0 =A0 for possible diffs so much better than any general purpose >> > =A0 =A0 search engine. >> > >> > =A0 =A0 Index of uncommited patches ready for test, would be ideal >> > =A0 =A0 for those currently stuck, & would assist more motivated >> > =A0 =A0 testers corroborating good patches worth commiting. >> > >> > =A0 =A0 A standard format would increase chances patch kits are found, >> > =A0 =A0 even if patch creator too busy to file send-pr etc. >> > >> > Let's adopt standards to make searches for potential patch trees easie= r: >> > - Adopt a common path root & nomenclature for all our trees of local d= iffs, >> > - Ask users to mirror local uncommited trees of diffs to thir local we= bs >> > =A0 (until if when commited after send-pr, then they delete) >> > - Ask authors of local patch kits to submit a single URL to a new wiki= page, >> > =A0 pointing to top automatically apply-able directory of patches >> > >> > Later we might also list a SOC project for a crawler indexer, >> > - src/ directories could also Optionaly later adopt >> > =A0 .MAINTAINER files (Subject of previous discussions, please dont le= t that >> > =A0 distract from main proposal though) >> > - ports/*/*/Makefile MAINTAINER =3D could also be used by a SOC tool >> >> While this idea is good as a base, doing this with patch-trees is the >> worst possible move. Patchfiles lack comments or 'commit info' and they >> do not easily record the revision and branch they should be applied to. >> >> Stacking multiple patches together with comments on what they do, is >> exactly what revision control systems were made for. And while we cannot >> easily share svn access to random contributors, systems like git or >> mercurial are exactly what we need here. >> >> In other words, we need github for FreeBSD. I'm working on some basics >> for this at repos.freebsd.your.org, but had severe VM instabilities >> during the last weeks. > > I have to admit that this crossed my mind as soon as I read Julian's > e-mail, especially as I've been keeping my local FreeBSD patches in > a version-controlled tree for the past ten years - first CVS, then > Subversion, and recently Git. > > Now, is there a reason we couldn't just use Gitorious? :) > > G'luck, > Peter > > -- > Peter Pentchev =A0roam@space.bg =A0 =A0roam@ringlet.net =A0 =A0roam@FreeB= SD.org > PGP key: =A0 =A0 =A0 =A0http://people.FreeBSD.org/~roam/roam.key.asc > Key fingerprint FDBA FD79 C26F 3C51 C95E =A0DF9E ED18 B68D 1619 4553 > If the meanings of 'true' and 'false' were switched, then this sentence w= ouldn't be false. > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iQIcBAEBCAAGBQJNF6qVAAoJEGUe77AlJ98TeacQAJQEVJJqR/3Lrx/ksypGOgLL > sL8PR8bPEHxur6IBanpmhGuDidxym9xhj/VSsxqAiAEIiOKfX1tsE+fOLcA7Ygtg > pusQTfhJD4KOCS86aFucXGL0r2gAEKkPUyJAwfORiZSaQsDjfVWKClEvZQNuBEOv > wk4DeeZsPcKBJCTDiplF/MJLTLgPTHQT30Xjsq2Ci9/f2atqD630v+GicywRo1Ha > jx9BF4wOF4o/1+XImB/eRWy8ZGAoHAwiFioFQwWSncEVbTPbnrHhiOZGBs2Ad57S > iaD+qEDRrD+Pc/fOcrHEKxE6frk1dskl9TOYUGWXh/XOhrCA4B0I+J8Q7elTCsrB > FFrsO2OTZOzWwTzSF/zLMsx8jAA/Flk7N86Fz5mf566P0GMn4dd/XR6oZXbAMasy > FJOPi7SllmQeVwLERwwQgxKCVIUB2ofYN+76jMxM11qbpXlBD5wipMV/nOAULrSg > vvj2Mi4d4aunUPy8J2AOYfc3RpqPRmDALXGz+g2ExFii2qN6qYxvrrxUa3PZHIu1 > N1d5yZM5Mm0NoiJJtQOlSi51NKydL5wcNvpPKg4XWJXz3H7b+4fNXKY/D4O+lkBr > jhC9OvWV3vzx/ME34GVjJAP3LT7E1ArWz0kyGQYEU5jzthPJd27L7vO+ftLJziVA > yQRepT6Dh6BtD6SdIFCQ > =3Do6Cb > -----END PGP SIGNATURE----- > > From owner-freebsd-arch@FreeBSD.ORG Mon Dec 27 15:02:08 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97823106566C for ; Mon, 27 Dec 2010 15:02:08 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.cksoft.de (mail.cksoft.de [IPv6:2001:4068:10::3]) by mx1.freebsd.org (Postfix) with ESMTP id 6B47E8FC15 for ; Mon, 27 Dec 2010 15:02:07 +0000 (UTC) Received: from localhost (amavis.fra.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 588CE41C7A3 for ; Mon, 27 Dec 2010 15:50:06 +0100 (CET) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([192.168.74.103]) by localhost (amavis.fra.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id kZt+IMJrYmIA for ; Mon, 27 Dec 2010 15:50:05 +0100 (CET) Received: by mail.cksoft.de (Postfix, from userid 66) id B417741C733; Mon, 27 Dec 2010 15:50:05 +0100 (CET) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id BBA2244490B for ; Mon, 27 Dec 2010 14:47:32 +0000 (UTC) Date: Mon, 27 Dec 2010 14:47:32 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: freebsd-arch@freebsd.org In-Reply-To: <20101222123834.GN23098@acme.spoerlein.net> Message-ID: <20101227131140.H6126@maildrop.int.zabbadoz.net> References: <201012220852.oBM8q2Qi039123@lurza.secnetix.de> <20101222123834.GN23098@acme.spoerlein.net> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-2080033646-1293456834=:6126" Content-ID: <20101227133453.P6126@maildrop.int.zabbadoz.net> Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Dec 2010 15:02:08 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-2080033646-1293456834=:6126 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1; FORMAT=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Content-ID: <20101227133453.M6126@maildrop.int.zabbadoz.net> On Wed, 22 Dec 2010, Ulrich Sp=F6rlein wrote: Hi, > I think this is the core "problem". Statistics[1] show, that most > developers run some form of -CURRENT and =2E.. > [1] I just made this statistic up. and I think you are just plain wrong here. Seriously I would bet that >75% of the developers do not run some sort of head for their day-to-day work. They might use it for compile (and boot and maybe sometimes even some more) testing, they might run it in a VM, or a lab machine but not on their servers, not on their notebooks and not on their desktops they work with daily (and neither would I expect most consumers of FreeBSD unfortunately). I am still not convinced that whatever development model people and companies use (and I heard of in here) is better than to just devel on HEAD and if it works there merge it and backport it to your release branch for QA and shipping. This might be a bit more work in the QA cycle but it saves you PYs on updating branches every couple of years and in addition allows you to start to cut a product from any branch {HEAD..your oldest} in theory whenever you want rather than being unable to deliver for half a year or a year. Our naming scheme doesn't matter much in that case. You lay your tags and start your branches yourself. Can you do what you ask us to do? Develop on HEAD and keep your own stuff going all the way down to 8, 7 and 6? In addition if you work on HEAD, you find problems as they occur and not years later when developers have long moved on to other projects and it's a pain for them to task swap back to whatever for a branch that indeed is only barely supported anymore. I know it won't help you with the backporting of new stuff like drivers to kind of unsupported release branches the way you would like to but the last paragraph applies there as well. It might be a lot easier when just done along initially. Having fixed a bug of mine with one of the last 10 commits that happened for RELENG_6, I know the pain. I have also previously agreed with people to not commit changes anymore for the sake that they'd never get the testing they needed - those patches are avail though, in GNATS, ... I know that HEAD is broken once in a while but I think we got a lot better than it sometimes used to be (and similar things apply to stable branches). I have cut images from stable branches in the past; I have given up on that a while ago; I am actually only running HEAD these days (home and production) for the sake of eating that dogfood but also to get all the advantages. I do my own tweaks to it as well, like you would do to your products but I try to keep them to a minimum like I would expect you to as well. I would love to hear from people who previously hit the pitfalls and decided not to go with HEAD again. Why didn't you do it? Do you regret it? Will you change "next time"? We still lack the parts that would tell us something in the last week or last 24 hours caused a regression that made my TCP/NFS/ZFS/UFS/ n% slower. Kris had been doing a good job in the past but as time shows we need more people, different setups, ... It's not only "compiles", "boots", but also the formerly in this thread mentioned "works correctly" and in addition to that the "works well as expected" or "works better than before" - hopefully;). I think this is a community problem as well. We need to have those things show up like quarterly status reports, like nanog gets the weekly CIDR reports, like there used to be "Internet monthly reports" (or do they still exist?), like tinderbox emails come in, ... If you do your daily/weekly/monthly regression tests for your products you can catch that. If you run HEAD, you'll also catch it timely to avoid binary searches of multiple quarters or years. Some of you have the infrastructure and I can understand that you cannot share (most of) it but you could run it on plain FreeBSD as well and show us the reports? Consider to do that regularly (it doesn't have to be daily but maybe (bi-)weekly or monthly). "Budget" for it in terms of infrastructure and employee time. It'll probably save you time (and with that money) in the end and help us to improve the solid foundation you are building your products on. My 0.4cts /bz --=20 Bjoern A. Zeeb You have to have visions! Going to jail sucks -- All my daemons like it! http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html --0-2080033646-1293456834=:6126-- From owner-freebsd-arch@FreeBSD.ORG Tue Dec 28 00:25:19 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E8E81065707 for ; Tue, 28 Dec 2010 00:25:19 +0000 (UTC) (envelope-from sales@nahave.com) Received: from nahave.com (mail.nahave.com [83.160.145.129]) by mx1.freebsd.org (Postfix) with ESMTP id ADC588FC18 for ; Tue, 28 Dec 2010 00:24:45 +0000 (UTC) Received: from ([127.0.0.1]) with MailEnable ESMTP; Tue, 28 Dec 2010 01:10:35 +0100 MIME-Version: 1.0 From: "NAHAVE Machinery" To: freebsd-arch@freebsd.org X-Mailer: SmartSend.2.0.116 Date: Tue, 28 Dec 2010 01:10:33 +0100 Message-ID: <43442161295441415825609@nhv-db01> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Newsletter December 2010 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sales@nahave.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Dec 2010 00:25:19 -0000 De Huufkes 69 D - 5674 TL NUENEN +31 40 737 0075 - +31 84 724 0768 info@nahave.com - www.nahave.com =20 =20 =20 Wheel Loader Doosan 500 DL 2008 80000=E2=82=AC Wheel Loader Caterpillar 918F 1996 22000=E2=82=AC Wheel Loader Caterpillar 966G 2001 79000=E2=82=AC Wheel Loader Caterpillar 972G Series I 1999 66000=E2=82=AC Track dozer Caterpillar D9H 1980 64000=E2=82=AC Wheel excavator M313C Triple boom 2006 =E2=82=AC75000 Wheel Loader Volvo L220F 2008 227500=E2=82=AC Wheel Loader Daewoo Mega 400V 2004 37000=E2=82=AC Wheel Loader Fiat Hitachi W270 1999 27000=E2=82=AC Wheel Loader Komatsu WA470-3H 1999 34000=E2=82=AC Wheel Loader Komatsu WA480-5H 2003 60000=E2=82=AC Wheel Loader Komatsu WA500-6 2007 200000=E2=82=AC Wheel excavator Caterpillar M315C 2006 70000=E2=82=AC Wheel excavator Fuchs MHL360 2003 110000=E2=82=AC Truck Man 12.170 Auto transp. 1986 4300=E2=82=AC Track excavator Case 1188 LC 1996 20000=E2=82=AC Track excavator Hyundai 210LC-3 2001 20000=E2=82=AC Track excavator Caterpillar 325DL 2006 99000=E2=82=AC Track excavator Caterpillar 320N 1995 32500=E2=82=AC Track excavator Caterpillar 324DL 2006 85000=E2=82=AC Track excavator Caterpillar 325CL 2003 60000=E2=82=AC Track excavator Caterpillar 330DL 2007 153000=E2=82=AC Track excavator Caterpillar 365B LME 2001 115000=E2=82=AC Track excavator Caterpillar 375 LME 2001 120000=E2=82=AC Track excavator Case 988 2000 15000=E2=82=AC Track excavator Halla HE280 LCE 1999 22000=E2=82=AC Track excavator Komatsu PC290 LC-6 1998 33000=E2=82=AC Track excavator Komatsu PC290 NLC-7 2002 45000=E2=82=AC Track excavator Komatsu PC340 LC-6 1998 32000=E2=82=AC Track excavator Liebherr R912 LC 1986 9000=E2=82=AC Track excavator Liebherr R934B HD-SL 2005 89000=E2=82=AC Track excavator Hitachi ZX280 LC 2004 74000=E2=82=AC Track excavator Hitachi ZX160 LC-3 2006 68000=E2=82=AC Track excavator Hitachi ZX470 LCH 2007 154000=E2=82=AC Track dozer Caterpillar D4H Series II 1990 34000=E2=82=AC Track dozer Caterpillar D9N 1991 87000=E2=82=AC Track dozer Caterpillar D6N LGP 2003 65000=E2=82=AC Track dozer Caterpillar D6N LGP 2006 138000=E2=82=AC Track dozer Caterpillar D8N LGP 1995 95000=E2=82=AC Track dozer Caterpillar D9L 1982 70000=E2=82=AC Track dozer Klippan Ottosan 513 1980 6000=E2=82=AC Motor grader Caterpillar 14G 1975 50000=E2=82=AC Motor grader Caterpillar 16 1969 30000=E2=82=AC Crawler Loader Atlas 2202E-HD 1985 20000=E2=82=AC Articulated vehicle Iveco 180-26 Rolfo 1992 3800=E2=82=AC Articulated Dump Truck Volvo A25C 6x6 1993 12000=E2=82=AC Articulated Dump Truck Volvo A30E 2008 184000=E2=82=AC Articulated Dump Truck Volvo A40D 2002 80000=E2=82=AC Don't want to receive our newsletter or stocklist anymore=3F Click here. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 28 15:20:14 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DFC2106566B for ; Tue, 28 Dec 2010 15:20:14 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 458228FC1B for ; Tue, 28 Dec 2010 15:20:14 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id A680446B06; Tue, 28 Dec 2010 10:20:13 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id ABD798A009; Tue, 28 Dec 2010 10:20:12 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 28 Dec 2010 10:05:36 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20101102; KDE/4.4.5; amd64; ; ) References: <20101222123834.GN23098@acme.spoerlein.net> <20101227131140.H6126@maildrop.int.zabbadoz.net> In-Reply-To: <20101227131140.H6126@maildrop.int.zabbadoz.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable Message-Id: <201012281005.36372.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 28 Dec 2010 10:20:12 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: "Bjoern A. Zeeb" Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Dec 2010 15:20:14 -0000 On Monday, December 27, 2010 9:47:32 am Bjoern A. Zeeb wrote: > On Wed, 22 Dec 2010, Ulrich Sp=F6rlein wrote: >=20 > Hi, >=20 > > I think this is the core "problem". Statistics[1] show, that most > > developers run some form of -CURRENT and > ... > > [1] I just made this statistic up. >=20 > and I think you are just plain wrong here. Seriously I would bet that > >75% of the developers do not run some sort of head for their > day-to-day work. They might use it for compile (and boot and maybe > sometimes even some more) testing, they might run it in a VM, or a lab > machine but not on their servers, not on their notebooks and not on > their desktops they work with daily (and neither would I expect most > consumers of FreeBSD unfortunately). >=20 >=20 > I am still not convinced that whatever development model people and > companies use (and I heard of in here) is better than to just devel > on HEAD and if it works there merge it and backport it to your release > branch for QA and shipping. Sadly, I need to actually ship the development I do that is work-related, s= o I=20 do it on stable first and then forward port it to HEAD afterward. Other=20 changes that are not work-centric (e.g. PCI hacking) I do on HEAD directly.= =20 However, for tasks specific to what work needs, that model does not work. One key reason it does not work is that many bugs only show up in productio= n. =20 This was true for me at Y! as well as my current job. And we are _not_ goi= ng=20 to run HEAD in production. I simply can't sell that. So that means that w= hen=20 we do encounter various bugs, we have to debug and fix them on the -stable = we=20 currently use and later forward port to HEAD. > In addition if you work on HEAD, you find problems as they occur and > not years later when developers have long moved on to other projects > and it's a pain for them to task swap back to whatever for a branch > that indeed is only barely supported anymore. This does not work for bugs that only show up under production load (which = is=20 most of them). > If you do your daily/weekly/monthly regression tests for your > products you can catch that. If you run HEAD, you'll also catch it > timely to avoid binary searches of multiple quarters or years. It can be a lot of work to maintain support for compiling on multiple=20 platforms that companies may not (I know some of them definitely will not)= =20 want to invest extra time in. > Some of you have the infrastructure and I can understand that you cannot > share (most of) it but you could run it on plain FreeBSD as well and > show us the reports? In many cases if a company has local changes to FreeBSD, their software is= =20 heavily tied to those local modifications and will not run on stock FreeBSD. > Consider to do that regularly (it doesn't have to > be daily but maybe (bi-)weekly or monthly). "Budget" for it in terms of > infrastructure and employee time. It'll probably save you time (and > with that money) in the end and help us to improve the solid > foundation you are building your products on. I think this is an area of give and take though. Some companies already=20 budget time in that they employ committers and allow them to work on FreeBS= D=20 stuff that isn't strictly work-related part-time and merge things upstream = to=20 HEAD when possible. Is it too much to ask that the Project make that path = a=20 bit easier by making stable branches longer? There are some recent examples of companies funding work to go into HEAD fi= rst=20 that they plan to use a backport of (NFSLOCKD, the NFS server GSSAPI suppor= t,=20 SU+J, and infiband), so this model can certainly work. It probably also wo= rks=20 best for upstreamable new features which tend to be some of the largest dif= fs=20 where the features can be tested independently of a company's proprietary=20 software stack. =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Dec 28 19:58:23 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99C0C1065670 for ; Tue, 28 Dec 2010 19:58:23 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5C95D8FC08 for ; Tue, 28 Dec 2010 19:58:23 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id EF11F46B06 for ; Tue, 28 Dec 2010 14:58:22 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E1D208A009 for ; Tue, 28 Dec 2010 14:58:21 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 28 Dec 2010 14:58:21 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20101102; KDE/4.4.5; amd64; ; ) References: <201012101050.45214.jhb@freebsd.org> In-Reply-To: <201012101050.45214.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201012281458.21413.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 28 Dec 2010 14:58:22 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Subject: Re: Realtime thread priorities X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Dec 2010 19:58:23 -0000 On Friday, December 10, 2010 10:50:45 am John Baldwin wrote: > So I finally had a case today where I wanted to use rtprio but it doesn't seem > very useful in its current state. Specifically, I want to be able to tag > certain user processes as being more important than any other user processes > even to the point that if one of my important processes blocks on a mutex, the > owner of that mutex should be more important than sshd being woken up from > sbwait by new data (for example). This doesn't work currently with rtprio due > to the way the priorities are laid out (and I believe I probably argued for > the current layout back when it was proposed). > > The current layout breaks up the global thread priority space (0 - 255) into a > couple of bands: > > 0 - 63 : interrupt threads > 64 - 127 : kernel sleep priorities (PSOCK, etc.) > 128 - 159 : real-time user threads (rtprio) > 160 - 223 : time-sharing user threads > 224 - 255 : idle threads (idprio and kernel idle procs) > > The problem I am running into is that when a time-sharing thread goes to sleep > in the kernel (waiting on select, socket data, tty, etc.) it actually ends up > in the kernel priorities range (64 - 127). This means when it wakes up it > will trump (and preempt) a real-time user thread even though these processes > nominally have a priority down in the 160 - 223 range. We do drop the kernel > sleep priority during userret(), but we don't recheck the scheduler queues to > see if we should preempt the thread during userret(), so it effectively runs > with the kernel sleep priority for the rest of the quantum while it is in > userland. > > My first question is if this behavior is the desired behavior? Originally I > think I preferred the current layout because I thought a thread in the kernel > should always have priority so it can release locks, etc. However, priority > propagation should actually handle the case of some very important thread > needing a lock. In my use case today where I actually want to use rtprio I > think I want different behavior where the rtprio thread is more important than > the thread waking up with PSOCK, etc. > > If we decide to change the behavior I see two possible fixes: > > 1) (easy) just move the real-time priority range above the kernel sleep > priority range I have forward-ported my original patch for 7 to 9 and fixed several other nits I ran into along the way. The updated patch is at http://www.freebsd.org/~jhb/patches/rtpri.patch I think it can probably be broken up into several pieces at least some of which should be non-controversial. :) This patch makes the following changes: - Give the USB kthreads lower priority in the range of software interrupt threads rather than hardware interrupt threads. - Retire some unused ithread priorities: PI_TTYHIGH, PI_TAPE, and PI_DISKLOW. While here, rename PI_TTYLOW to PI_TTY. Also, add a macro PI_SWI() that takes a SWI_* constant as an argument and returns the suitable thread priority. - In sched_yield(), only drop the priority of timeshare threads to PRI_MAX_TIMESHARE. Non-timeshare threads retain whatever priority they currently have. - Only apply a kernel sleep priority from tsleep() to timeshare threads. This is only relevant once realtime threads move to a new priority range to avoid penalizing realtime threads for sleeping. - Explicitly set a sane initial priority (of PVM) for kthreads. Right now new kthreads inherit whatever priority thread0 happens to have when they are created. Since kthreads can be created from threads other than thread0 this priority can be fairly random. In practice, I've seen many kthreads created with an initial priority that is a hardware interrupt thread priority due to thread0 being lent an ithread priority. - Add some helper macros to ULE to define the ranges used for interactive and non-interactive timeshare threads and fix some places that hardcoded assumptions about the location of the realtime priority range. - Add a new option (that should perhaps be on by default) for use in conjunction with moving realtime priorities ULE_INTERACTIVE_TIMESHARE. When this new option is in effect, ULE does not abuse realtime priorites for interactive timeshare threads. Instead, the timeshare range is split into two ranges, one for interactive threads and one for non-interactive threads. The non-interactive range is further divided into three ranges to add bands at the top and bottom for nice levels. Combined with the other changes, the net effect is that interactive threads will have the same priority they have now (i.e. a band of 32 priorities in between kernel sleep priorities and non-interactive timeshare priorities) and that non-interactive threads now have a slightly larger band of priorities (32 priorities in the "middle" instead of 24 with additional bands of 20 above and below for nice values). - Never boost the priority of a thread via tsleep() if the passed in priority is zero. Zero means "don't change the priority", but ULE was still giving a boost in certain cases. In practice I suspect this rarely, if ever, triggered. - Always apply the requested sleep priority to kthreads. Certain kernel processes such as pagedaemon, etc. rely on tsleep() to lower the priority of the kproc so that it is treated as a background task when it is idle. The static_boost code in ULE would never lower the priority due to a sleep, so once a kproc gained a higher priority via sleeping it would never be treated as a background task again. This is especially problematic in the case that a kthread starts off with an ithread priority as noted above. - Retire the PCONFIG kernel sleep priority. We do not need a new priority level for boot time config hooks. When PCONFIG was added, tsleep() did not support leaving the priority alone via 0, but now it does support that so use that instead. - Restore dropping the syncer kthread down to PPAUSE when it is idle. - Drop the flowtable cleaner kthread down to PPAUSE when it is idle. - Move the realtime priority range in between the interrupt thread and kernel sleep priority range. Currently there is a small bit of overlap between SWI_TQ and SWI_TQ_GIANT and 'rtprio 0'. I hope to eliminate this by retiring SWI_TQ_FAST once interrupt filters are in place as then SwI_TQ and SWI_TQ_GIANT can move up a slot. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Dec 28 23:12:27 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 46F58106566C for ; Tue, 28 Dec 2010 23:12:27 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id DFA9F8FC0C for ; Tue, 28 Dec 2010 23:12:26 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3ABDB.dip.t-dialin.net [87.179.171.219]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 7EDAB844010; Wed, 29 Dec 2010 00:04:17 +0100 (CET) Received: from unknown (IO.Leidinger.net [192.168.2.110]) by outgoing.leidinger.net (Postfix) with ESMTP id B394611FF; Wed, 29 Dec 2010 00:04:14 +0100 (CET) Date: Wed, 29 Dec 2010 00:04:12 +0100 From: Alexander Leidinger To: Erik Cederstrand Message-ID: <20101229000412.000049ee@unknown> In-Reply-To: References: <201012211500.16131.jhb@freebsd.org> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 7EDAB844010.A6DB6 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1, required 6, autolearn=disabled, ALL_TRUSTED -1.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1294182258.04075@JG32wfGQRs5JtiFItqkVUw X-EBL-Spam-Status: No Cc: mdf@freebsd.org, Robert Watson , freebsd-arch@freebsd.org Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Dec 2010 23:12:27 -0000 On Wed, 22 Dec 2010 08:21:56 +0100 Erik Cederstrand wrote: > Den 21/12/2010 kl. 23.28 skrev Robert Watson: > > > > > Looking at 7.x, I'm struck by how much it has slowed down. There's > > a significant user community, but not a significant developer > > community. > > Which pretty much sums up a dilemma in the development of FreeBSD, I > think. Developers want users to try out their new shiny stuff, but > users don't want to spend time upgrading. > > I think one of many things that would be great to do is to improve > the usability and coverage of the regression tests. This would take > at least some of the burden off developers who want to MFC their > work. We already have the tinderboxes, Coverity and Clang Static > Analyzer, but apart from pho's stress tests we don't have any > automated runtime testing (as far as I know). Your benchmark suite can not be used for something like this? With the regression tests as the benchmarks? BTW: real benchmarks are also some kind of regression tests, Bye, Alexander. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 28 23:17:27 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32828106566B for ; Tue, 28 Dec 2010 23:17:27 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id DF9C78FC14 for ; Tue, 28 Dec 2010 23:17:26 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3ABDB.dip.t-dialin.net [87.179.171.219]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 99439844010; Wed, 29 Dec 2010 00:01:36 +0100 (CET) Received: from unknown (IO.Leidinger.net [192.168.2.110]) by outgoing.leidinger.net (Postfix) with ESMTP id E69CA11FD; Wed, 29 Dec 2010 00:01:32 +0100 (CET) Date: Wed, 29 Dec 2010 00:01:30 +0100 From: Alexander Leidinger To: "Bjoern A. Zeeb" Message-ID: <20101229000130.000052f4@unknown> In-Reply-To: <20101227131140.H6126@maildrop.int.zabbadoz.net> References: <201012220852.oBM8q2Qi039123@lurza.secnetix.de> <20101222123834.GN23098@acme.spoerlein.net> <20101227131140.H6126@maildrop.int.zabbadoz.net> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 99439844010.A7B93 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1, required 6, autolearn=disabled, ALL_TRUSTED -1.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1294182098.37447@lu/TBM9OqRe7pnVn4mHC8A X-EBL-Spam-Status: No Cc: freebsd-arch@freebsd.org Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Dec 2010 23:17:27 -0000 On Mon, 27 Dec 2010 14:47:32 +0000 (UTC) "Bjoern A. Zeeb" wrote: > On Wed, 22 Dec 2010, Ulrich Sp=F6rlein wrote: >=20 > Hi, >=20 > > I think this is the core "problem". Statistics[1] show, that most > > developers run some form of -CURRENT and > ... > > [1] I just made this statistic up. >=20 > and I think you are just plain wrong here. Seriously I would bet that > >75% of the developers do not run some sort of head for their > day-to-day work. They might use it for compile (and boot and maybe > sometimes even some more) testing, they might run it in a VM, or a lab > machine but not on their servers, not on their notebooks and not on > their desktops they work with daily (and neither would I expect most > consumers of FreeBSD unfortunately). You can count me as one of those which run (more or less) HEAD on his server and (mostly unused) desktop at home. > I am still not convinced that whatever development model people and > companies use (and I heard of in here) is better than to just devel > on HEAD and if it works there merge it and backport it to your release > branch for QA and shipping. It may not be a problem for developers which know enough about FreeBSD, but try to sell this to people which do not know enough about FreeBSD or some management-people (and I'm not talking about the money-argument here). > We still lack the parts that would tell us something in the last week > or last 24 hours caused a regression that made my TCP/NFS/ZFS/UFS/ name it> n% slower. Kris had been doing a good job in the past but as > time shows we need more people, different setups, ... We do not lack the parts, we lack someone to take the parts and get them up and running. See: http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02819.html http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02821.html > It's not only "compiles", "boots", but also the formerly in this > thread mentioned "works correctly" and in addition to that the "works > well as expected" or "works better than before" - hopefully;). Maybe this could also be used to run the regression tests as one of the benchmarks. If yes: As Robert mentioned, we can not go and tell to run them all in one command (ATM), but we could have each of them as a different benchmark. Bye, Alexander. From owner-freebsd-arch@FreeBSD.ORG Wed Dec 29 08:22:24 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E04F106566C; Wed, 29 Dec 2010 08:22:24 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6EDF88FC12; Wed, 29 Dec 2010 08:22:24 +0000 (UTC) Received: from xyf.my.dom (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oBT8MMN3097264; Wed, 29 Dec 2010 08:22:23 GMT (envelope-from davidxu@freebsd.org) Message-ID: <4D1AEFC0.7030400@freebsd.org> Date: Wed, 29 Dec 2010 16:22:24 +0800 From: David Xu User-Agent: Thunderbird 2.0.0.24 (X11/20100630) MIME-Version: 1.0 To: John Baldwin , Julian Elischer , arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: TDF_NEEDRESCHED (was: Realtime thread priorities) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Dec 2010 08:22:24 -0000 Hi all, I think flag TDF_NEEDRESCHED should not be cleared by sched_switch() in ULE or 4BSD, instead it should only be cleared by ast() in subr_trap.c. The reason is that the flag indicates thread should reset its priority and switch context at user boundary because its user mode priority is lowered or there is higher priority thread wants to run. Kernel needs to use this flag to reset its priority to td_user_pri before a thread returns to user mode, in current code, if an interrupt thread preempts a user thread, sched_switch() clears the flag for preempted thread and then switches to preempting thread, this causes preempted thread to forget resetting its current priority to td_user_pri this becauses assemble language code doreti() can not find the flag, and ast() is not called, the thread ends up running user mode code at very high level priority. Fix me, if I am wrong. Regards, David Xu From owner-freebsd-arch@FreeBSD.ORG Wed Dec 29 10:46:29 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A80C31065670; Wed, 29 Dec 2010 10:46:29 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id DD1458FC16; Wed, 29 Dec 2010 10:46:28 +0000 (UTC) Received: by wwf26 with SMTP id 26so9742701wwf.31 for ; Wed, 29 Dec 2010 02:46:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=PByus0SLbpKY9gM+H5U0vAth/G3JMc+zIB2KOZKgde4=; b=ecl6oWQnMVNcDZ4/z5xkxUt6BLfxIcxHmClaeYuqMoTowjXJqnSoylP+5EFxvGE60w xyNOYVdtQFuRi8oqmGtuaNCXzm0duhue6x0zf0x7YMChLd5Y/4d900SlPNL8bnzMh+S3 O9Fz5cIJnYXRwbmiVVwdLilv3g8SQ3e7ukS/k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=OcFysTD5sIGac89oRI4j2sXAvbVC4Srv/s5vULnxu9EGYx3KcEmPemxIAdIlgskowX pewd0fiZr14wnNIvTP+MPBy20SAtWqEEM209K3BhkskATPJRRCx3LxK+OYLa5KKf8tbm PMOha4Dq3qr2RFxeoCRg0eOKWteKDqA6CA/FU= MIME-Version: 1.0 Received: by 10.216.141.75 with SMTP id f53mr2489955wej.16.1293619587398; Wed, 29 Dec 2010 02:46:27 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Wed, 29 Dec 2010 02:46:26 -0800 (PST) In-Reply-To: <20101229000412.000049ee@unknown> References: <201012211500.16131.jhb@freebsd.org> <20101229000412.000049ee@unknown> Date: Wed, 29 Dec 2010 02:46:26 -0800 X-Google-Sender-Auth: erwFATuu74GjpmQk4kb782qbPbw Message-ID: From: Garrett Cooper To: Alexander Leidinger Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: mdf@freebsd.org, Robert Watson , Erik Cederstrand , freebsd-arch@freebsd.org Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Dec 2010 10:46:29 -0000 On Tue, Dec 28, 2010 at 3:04 PM, Alexander Leidinger wrote: > On Wed, 22 Dec 2010 08:21:56 +0100 Erik Cederstrand > wrote: > >> Den 21/12/2010 kl. 23.28 skrev Robert Watson: >> >> > >> > Looking at 7.x, I'm struck by how much it has slowed down. =A0There's >> > a significant user community, but not a significant developer >> > community. >> >> Which pretty much sums up a dilemma in the development of FreeBSD, I >> think. Developers want users to try out their new shiny stuff, but >> users don't want to spend time upgrading. >> >> I think one of many things that would be great to do is to improve >> the usability and coverage of the regression tests. This would take >> at least some of the burden off developers who want to MFC their >> work. We already have the tinderboxes, Coverity and Clang Static >> Analyzer, but apart from pho's stress tests we don't have any >> automated runtime testing (as far as I know). > > Your benchmark suite can not be used for something like this? With the > regression tests as the benchmarks? BTW: real benchmarks are also some > kind of regression tests, Sometimes, but we are lacking in the functional tests department in areas and that's something in coverage that needs to be filled out. Stress tests/benchmarks can exercise a lot or a little code and can create unnecessary results variance depending on how they're written, as I'm sure everyone posting on the list to this topic is aware. Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Wed Dec 29 11:17:43 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A580106566B for ; Wed, 29 Dec 2010 11:17:43 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 8B9198FC0C for ; Wed, 29 Dec 2010 11:17:42 +0000 (UTC) Received: by wwf26 with SMTP id 26so9762238wwf.31 for ; Wed, 29 Dec 2010 03:17:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=sEjyvg8kn5hor2Q3jqvjh2gVlsarROq/otXKXvMP5zA=; b=MC6cpI+Avdkk1WTnye2BSU4qjthEWme0GRC0C4gmjpFqaWrBdtH428caX29bbVKwQl wmqwsj35GGZH2zddLArTBnP+7v2FItdaxKhVgDTLCNZ0Yv9si2GLWiTXeh4Hcbh+MdLH /b1iwmupKoImL9s7rG7vwD/vptN8jIRgnGWc8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=XjiivYVTMRylF/B2lIPjHOTD4WmTA9m/abrF5J8Jrj1TPUGumHi2zPSapoZ2o1JjKA fjG4ytC4vJOAEDOZIGyfS5UUD60/2h0IKhLwNLTRV6BSw1gNBWvEMuPaCCeApkU+K7fn KbubVQAvRNI3GU02yjVwOJbZa++tN3Nlxgo3E= MIME-Version: 1.0 Received: by 10.216.141.37 with SMTP id f37mr765001wej.31.1293621461470; Wed, 29 Dec 2010 03:17:41 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Wed, 29 Dec 2010 03:17:41 -0800 (PST) In-Reply-To: <20101229000130.000052f4@unknown> References: <201012220852.oBM8q2Qi039123@lurza.secnetix.de> <20101222123834.GN23098@acme.spoerlein.net> <20101227131140.H6126@maildrop.int.zabbadoz.net> <20101229000130.000052f4@unknown> Date: Wed, 29 Dec 2010 03:17:41 -0800 X-Google-Sender-Auth: eKPO1s44Y8QlvDTQoBpbOFdzWJc Message-ID: From: Garrett Cooper To: Alexander Leidinger Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "Bjoern A. Zeeb" , freebsd-arch@freebsd.org Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Dec 2010 11:17:43 -0000 On Tue, Dec 28, 2010 at 3:01 PM, Alexander Leidinger wrote: ... >> I am still not convinced that whatever development model people and >> companies use (and I heard of in here) is better than to just devel >> on HEAD and if it works there merge it and backport it to your release >> branch for QA and shipping. > > It may not be a problem for developers which know enough about FreeBSD, > but try to sell this to people which do not know enough about FreeBSD > or some management-people (and I'm not talking about the > money-argument here). Bottom line is always feature set in products and ship date from what I've seen, unless someone understands the implications of licensing and takes a look at the big picture as far as maintainability is concerned. Or maybe that little culture is just Cisco and their desire for filling in pretty checkboxes based on what marketing thinks that users want driven by a program management team and managers *shrug*. >From what I see many people like Linux because it's an established name in free software, there are support companies that `reduce' the need to maintain software because they produce the packages, tools, and infrastructure to make your life `easier' (Android SDK, Montavista, etc). and there are N times many code monkeys in the world that `know' how do hack a Linux distro fifteen different ways to Sunday, compared to developers who work within the opensource communities. Because people see a prototype out the door, and can assemble projects faster with OS X.Y.Z, they usually go with that over something else (sometimes BSD, sometimes not). >> We still lack the parts that would tell us something in the last week >> or last 24 hours caused a regression that made my TCP/NFS/ZFS/UFS/> name it> n% slower. =A0Kris had been doing a good job in the past but as >> time shows we need more people, different setups, ... > > We do not lack the parts, we lack someone to take the parts and get > them up and running. See: > =A0http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02819.h= tml > =A0http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02821.h= tml No: according to Erik in the first link, the parts were never fully integrated and the project as a whole was never completed due to lack of hardware and time. Some bits are also just directions as well as Erik says in his reply to the thread. After seeing enough folks prematurely claim victory within my own company, I know enough that nothing is complete unless it works (to some basic degree, i.e. it can be replicated by someone other than the original author of the infrastructure or code using the directions provided from scratch). >> It's not only "compiles", "boots", but also the formerly in this >> thread mentioned "works correctly" and in addition to that the "works >> well as expected" or "works better than before" - hopefully;). > > Maybe this could also be used to run the regression tests as one of the > benchmarks. If yes: As Robert mentioned, we can not go and tell to run > them all in one command (ATM), but we could have each of them as a > different benchmark. Regression tests should be stable. Regression tests shouldn't crash targets (in general). Benchmarks and stress tests can do that. We need regression tests that can be run on a nightly/weekly/monthly/whatever basis with the needed level of breadth and depth on a dedicated set of cluster machines that we can gather results with in a sane manner otherwise whoever is managing the machines will run into the maintenance nightmare that portmgr was/is running into with the tinderbox machines on a regular/periodic basis (ask Mark Linimon about that). ports infrastructure is partly to blame, yes, but a large part of things as well is infrastructure tied up in pointyhat, etc as far as was explained / conveyed to me over email and IRC. As much as I love Yahoo! and the infrastructure work that Yahoo! has provided, a lot of what I see wrong with how things are done with some of the work that Kris did is that it was never made portable, and documentation is lacking in areas -- thus duplicating infrastructure, results, success, and profit is difficult. IMO we should have something like: make checkworld make checkkernel that would go off and run functional regression tests, and something like: make stressworld make stresskernel that would to similar for benchmarks as well. Having stuff tied together between the sources and somewhere in the tree is a minor infrastructure hurdle to get over, but a must for integration. kyua (because ATF has been abandoned as a project except for maintenance releases and bugfixes are concerned) should be integrated in base as well as a tool. Giorgios has engaged Julio from NetBSD about it in the past couple of weeks and I will follow up again as well, time permitting. Adding it to ports is quite frankly a waste of time. That way once the infrastructure is in place, we can easily port over testcases from NetBSD (technically we could do the same right now using the ATF testcases and that might be a good short-term investment even though it's sort of a wasted effort in one respect), and get the coverage that we need on a regular basis to increase confidence in FreeBSD as far as stability on HEAD and MFCs is concerned, so developers can instead focus on writing more code and tests corresponding to that code, so FreeBSD can better scale longterm as a project. Thanks, -Garrett Warning: the above is based on personal opinion as IANAPGM (not a program manager :P), thank my lucky stars... From owner-freebsd-arch@FreeBSD.ORG Wed Dec 29 11:32:49 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FD72106564A for ; Wed, 29 Dec 2010 11:32:49 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id B14868FC0C for ; Wed, 29 Dec 2010 11:32:48 +0000 (UTC) Received: by wwf26 with SMTP id 26so9771007wwf.31 for ; Wed, 29 Dec 2010 03:32:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=6tEMK/XlpZ27zbHgwDtx+Ekbizel0JWuURAr72x2xMQ=; b=RzmrZTyKRR+oGt6KlMhLTsLlfemKMacXsyyR3dlxgrPRlmaQWiYtk2797vgIQYCHpk LNaM2d2Bt2wFBjFgX/le6fngkFDK1wxVNO5AqmilQEPzlmX1XrmzGWyLsYX/hpwJh6sO FGCYk8r728mf6vzH0e0MXiQlzssQwZBm+TWjc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Em/Qugw155ZYBRTr1ZfV4MwnWwfJMy5E4i4FbrACMvHfyur7hNiFPTH3RX5psFLttU tFciYoVhIwwCgJknqUaJVRV69q9GrsvLYNbig4qC96cYoGy2uX7uK4EKxI6oQ+DPAHfV WmxF6FJG7BcyoR9ktFbNB2l916sML0J4xYIHQ= MIME-Version: 1.0 Received: by 10.216.191.215 with SMTP id g65mr2107986wen.16.1293622367763; Wed, 29 Dec 2010 03:32:47 -0800 (PST) Received: by 10.216.254.226 with HTTP; Wed, 29 Dec 2010 03:32:47 -0800 (PST) In-Reply-To: <20101227131140.H6126@maildrop.int.zabbadoz.net> References: <201012220852.oBM8q2Qi039123@lurza.secnetix.de> <20101222123834.GN23098@acme.spoerlein.net> <20101227131140.H6126@maildrop.int.zabbadoz.net> Date: Wed, 29 Dec 2010 03:32:47 -0800 Message-ID: From: Garrett Cooper To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-arch@freebsd.org" Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Dec 2010 11:32:49 -0000 On Dec 27, 2010, at 6:47 AM, "Bjoern A. Zeeb" wrote: > On Wed, 22 Dec 2010, Ulrich Sp=F6rlein wrote: > > Hi, > >> I think this is the core "problem". Statistics[1] show, that most >> developers run some form of -CURRENT and > ... >> [1] I just made this statistic up. > > and I think you are just plain wrong here. Seriously I would bet that > >75% of the developers do not run some sort of head for their > day-to-day work. They might use it for compile (and boot and maybe > sometimes even some more) testing, they might run it in a VM, or a lab > machine but not on their servers, not on their notebooks and not on > their desktops they work with daily (and neither would I expect most > consumers of FreeBSD unfortunately). Maybe I'm just crazy but I run head on all of my boxes and netboot releng whenever necessary.., > I am still not convinced that whatever development model people and > companies use (and I heard of in here) is better than to just devel > on HEAD and if it works there merge it and backport it to your release > branch for QA and shipping. Yeah, I don't develop that way though except at home with my own personal code; then again I don't hack stuff as much in that respect as other folks do in my group. > This might be a bit more work in the QA cycle but it saves you PYs on > updating branches every couple of years and in addition allows you to > start to cut a product from any branch {HEAD..your oldest} in theory > whenever you want rather than being unable to deliver for half a year > or a year. Our naming scheme doesn't matter much in that case. You > lay your tags and start your branches yourself. Depends. Some folks don't lay your own tags and whatever and try to leverage as much from the project's ports and packaging infrastructure so we don't have to reroll packages (this caveat fails though if the base system is touched, ie bzip2, OpenSSH, OpenSSL, tcpdump, zlib, etc). Some things are worth contributing back to releases and other things may not be. > Can you do what you ask us to do? Develop on HEAD and keep your own > stuff going all the way down to 8, 7 and 6? Parallel development after writing automated tests for non-device driver specific items is the only way to scale here, for all parties :/. > In addition if you work on HEAD, you find problems as they occur and > not years later when developers have long moved on to other projects > and it's a pain for them to task swap back to whatever for a branch > that indeed is only barely supported anymore. This problem isn't going to go away unless the statement above remains true and developers in companies are competent in the areas needing merging and competent in FreeBSD. > I know it won't help you with the backporting of new stuff like > drivers to kind of unsupported release branches the way you would like > to but the last paragraph applies there as well. It might be a lot > easier when just done along initially. Sometimes though (take mfi for example), backporting new driver support completely breaks existing ABI compatibility for the driver -- for example -- because structures change. It would be nice if there was a balance better struck for backwards compatibility vs new hardware support, but often times it seems more difficult in areas than it should be. > Having fixed a bug of mine with one of the last 10 commits that > happened for RELENG_6, I know the pain. I have also previously agreed > with people to not commit changes anymore for the sake that they'd > never get the testing they needed - those patches are avail though, > in GNATS, ... > > > I know that HEAD is broken once in a while but I think we got a lot > better than it sometimes used to be (and similar things apply to stable > branches). I have cut images from stable branches in the past; I have > given up on that a while ago; I am actually only running HEAD these > days (home and production) for the sake of eating that dogfood but > also to get all the advantages. I do my own tweaks to it as well, like > you would do to your products but I try to keep them to a minimum like > I would expect you to as well. > > > I would love to hear from people who previously hit the pitfalls and > decided not to go with HEAD again. Why didn't you do it? Do you > regret it? Will you change "next time"? The only time that I really regretted running HEAD for an extended period of time was when the VM changes were first being done for amd64 this year, when it broke the nvidia driver on my machine to the point that I couldn't work with my desktop machine on real work I needed to complete; I just had to sit back, probe stability periodically and finally I jumped in July when the dust settled, but it was painful for multiple reasons doing that when other bugfixes/enhancements were being made to the system that could have made my life a lot easier. > We still lack the parts that would tell us something in the last week > or last 24 hours caused a regression that made my TCP/NFS/ZFS/UFS/ name it> n% slower. Kris had been doing a good job in the past but as > time shows we need more people, different setups, ... I think you hit a good point. We cannot abandon the wealth of soak testing in a large user base, but in order to ensure that we're doing a good job for the FreeBSD community we need to ensure that we're meeting at least the minimum bar for testing (whatever that might be) to ensure that whatever we're pushing back is reasonably ok. I think everyone on here can agree with that point. > It's not only "compiles", "boots", but also the formerly in this thread > mentioned "works correctly" and in addition to that the "works well as > expected" or "works better than before" - hopefully;). I usually do: 1. Compile. 2. Load. 3. Functional smoke test. 4. More focused probing. Your testing may vary. > I think this is a community problem as well. We need to have those > things show up like quarterly status reports, like nanog gets the > weekly CIDR reports, like there used to be "Internet monthly > reports" (or do they still exist?), like tinderbox emails come in, ... Quarterly is too long (unless you're referring to the performance results). We need tests running a lot more frequently to keep up with the rate of development in the project. > If you do your daily/weekly/monthly regression tests for your > products you can catch that. If you run HEAD, you'll also catch it > timely to avoid binary searches of multiple quarters or years. > Some of you have the infrastructure and I can understand that you cannot > share (most of) it but you could run it on plain FreeBSD as well and > show us the reports? Consider to do that regularly (it doesn't have to > be daily but maybe (bi-)weekly or monthly). "Budget" for it in terms of > infrastructure and employee time. It'll probably save you time (and > with that money) in the end and help us to improve the solid > foundation you are building your products on. Sad part of this statement too is that some companies (like mine) have a separation of development and QA, and due to the way that QA's staffed they often times do manual testing in place of automated testing, which isn't acceptable in the project. Also, I don't know if it's necessarily a good idea to post results as there could be a large degree of fud in them based on what features were added between releases X, Y, and Z, and what modifications were made between those releases, for what little it might be worth. Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Wed Dec 29 14:04:44 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B7F9B1065748; Wed, 29 Dec 2010 14:04:44 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 881018FC21; Wed, 29 Dec 2010 14:04:44 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 1BF5046B03; Wed, 29 Dec 2010 09:04:44 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5B52D8A027; Wed, 29 Dec 2010 09:04:43 -0500 (EST) From: John Baldwin To: David Xu Date: Wed, 29 Dec 2010 08:43:52 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20101102; KDE/4.4.5; amd64; ; ) References: <4D1AEFC0.7030400@freebsd.org> In-Reply-To: <4D1AEFC0.7030400@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201012290843.52677.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Wed, 29 Dec 2010 09:04:43 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: arch@freebsd.org, Julian Elischer Subject: Re: TDF_NEEDRESCHED (was: Realtime thread priorities) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Dec 2010 14:04:44 -0000 On Wednesday, December 29, 2010 3:22:24 am David Xu wrote: > Hi all, > > I think flag TDF_NEEDRESCHED should not be cleared by sched_switch() in > ULE or 4BSD, instead it should only be cleared by ast() in subr_trap.c. > The reason is that the flag indicates thread should reset its priority > and switch context at user boundary because its user mode priority is > lowered or there is higher priority thread wants to run. > Kernel needs to use this flag to reset its priority to td_user_pri > before a thread returns to user mode, in current code, if an interrupt > thread preempts a user thread, sched_switch() clears the flag for > preempted thread and then switches to preempting thread, this causes > preempted thread to forget resetting its current priority to td_user_pri > this becauses assemble language code doreti() can not find the flag, > and ast() is not called, the thread ends up running user mode code > at very high level priority. Fix me, if I am wrong. Hmm, I think you are correct in which case I broke this many years ago. :( I think it might have worked originally because TDF_NEEDRESCHED triggered the preemption itself in ast(). The bug was probably introduced when I moved preemption into setrunqueue() itself (now sched_add() / critical_exit()). I think originally we also set TDF_NEEDRESCHED more often than we needed to, but now we probably don't (though we should check that it is now only set when we need to run that particular bit of code in ast() and nowhere else). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Dec 30 00:36:16 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95EF41065670 for ; Thu, 30 Dec 2010 00:36:16 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id E17F68FC13 for ; Thu, 30 Dec 2010 00:36:15 +0000 (UTC) Received: from park.js.berklix.net (p5B22C67B.dip.t-dialin.net [91.34.198.123]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id oBU0a8rW080032 for ; Thu, 30 Dec 2010 00:36:11 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id oBU0a3Zh010426 for ; Thu, 30 Dec 2010 01:36:05 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id oBU0ZwdB094266 for ; Thu, 30 Dec 2010 01:36:03 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201012300036.oBU0ZwdB094266@fire.js.berklix.net> To: freebsd-arch@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Sun, 26 Dec 2010 22:50:33 +0200." <20101226205033.GA4135@straylight.ringlet.net> Date: Thu, 30 Dec 2010 01:35:58 +0100 Sender: jhs@berklix.com Subject: Re: Let's adopt a standard nomenclature for webs of patch trees etc. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Dec 2010 00:36:16 -0000 Peter Pentchev wrote: > On Sun, Dec 26, 2010 at 09:42:29PM +0100, Ulrich Sp=C3=B6rlein wrote: > > On Sun, 26.12.2010 at 18:28:20 +0100, Julian H. Stacey wrote: > > > Was Subject: Re: Schedule for releases > > > I changed it, as my reply takes it too far off that topic. > > >=20 > > > Erik Cederstrand wrote: > > > > Hi Mike, > > > > Den 22/12/2010 kl. 18.45 skrev Mike Karels: > > > >=20 > > > > > - Those of us doing backports could probably do a better job of > > > > > sharing the results. On the other hand, I'm generally backporting > > > > > to a specific release (currently 6.3 or 7.2) rather than -stable, > > > > > and we're testing our software rather than FreeBSD. > > > >=20 > > > > Thanks for taking the time to write your comments. What strikes me is= > =3D > > > > that we may have lots of non-FreeBSD developers working to backport = > =3D > > > > stuff in their own private trees. Possibly a lot of redundant work is= > =3D > > > > being done. > > > >=20 > > > > Even though you're backporting to a specific release, and even though= > =3D > > > > you're only testing the work via your own software, would it not help= > =3D > > > > others and possibly even yourself if this FreeBSD-specific work lands= > in =3D > > > > the FreeBSD repository instead of your private tree? In my view you'r= > e =3D > > > > just as much a FreeBSD developer as people with commit access that ar= > e =3D > > > > scratching their own itches in CURRENT. > > > >=20 > > > > Erik=3D > > >=20 > > > Good point, Probably loads of fixes from non commiters never get > > > sent back to FreeBSD. Many people will have motivation only to fix loc= > al > > > problems, but no time to send back, especially deterred by clunky send-= > pr. > > >=20 > > > Though I & many others have sent lots of send-pr, =20 > > > contributing even a spelling correction to FreeBSD > > > is much harder than to eg http://wikipedia.org > > > =09 > > > + a beginner has to bend their brain to send-pr=20 > > > =09 > > > + send-pr user should not be burdened exploring tree to find=20 > > > Maintainer to send-pr CC (which should be automaticly > > > extracted from tree on a ports =3DMAINTAINER basis > > > or eg a src/ .MAINTAINER per some sub directories > > > where there is a volunteer or mail list) > > > =09 > > > + send-pr user must spend time composing a > > > diplomatic & attractive subject & body, to catch > > > some gnats@ readers eye, to get them to stop browsing > > > get interested, & commit. > > > =09 > > > Many a potential contributor's attitude will be: I don't > > > have time: Catch the diff or drop it, your loss ! > > >=20 > > > So a lot of potential send-pr won't get filed, but I bet local users > > > don't toss their fixes though, but keep local patch kits, till if > > > ever they or others send-pr & something gets commited, (which might > > > be days or years later). > > >=20 > > > Those diff trees stored localy, users could easily export via > > > rdist/rsync etc to their local webs, eg I do this: > > > My diffs in a tree structure > > > http://berklix.com/~jhs/src/bsd/fixes/FreeBSD > > > My application script > > > http://berklix.com/~jhs/bin/.csh/customise > > >=20 > > > Those trees, FreeBSD could encourage users to keep in a standard > > > format (path nomenclature etc) & we should reccomend, > > > indexed from a common page on eg wiki.freebsd.org > > >=20 > > > It would make a search tool &/or automatic periodic indexing > > > for possible diffs so much better than any general purpose > > > search engine. > > >=20 > > > Index of uncommited patches ready for test, would be ideal > > > for those currently stuck, & would assist more motivated > > > testers corroborating good patches worth commiting. > > >=20 > > > A standard format would increase chances patch kits are found, > > > even if patch creator too busy to file send-pr etc. > > >=20 > > > Let's adopt standards to make searches for potential patch trees easier: > > > - Adopt a common path root & nomenclature for all our trees of local di= > ffs, > > > - Ask users to mirror local uncommited trees of diffs to thir local webs > > > (until if when commited after send-pr, then they delete) > > > - Ask authors of local patch kits to submit a single URL to a new wiki = > page, > > > pointing to top automatically apply-able directory of patches > > >=20 > > > Later we might also list a SOC project for a crawler indexer, > > > - src/ directories could also Optionaly later adopt=20 > > > .MAINTAINER files (Subject of previous discussions, please dont let t= > hat > > > distract from main proposal though) > > > - ports/*/*/Makefile MAINTAINER =3D could also be used by a SOC tool > >=20 > > While this idea is good as a base, doing this with patch-trees is the > > worst possible move. Patchfiles lack comments or 'commit info' and they > > do not easily record the revision and branch they should be applied to. > >=20 > > Stacking multiple patches together with comments on what they do, is > > exactly what revision control systems were made for. And while we cannot > > easily share svn access to random contributors, systems like git or > > mercurial are exactly what we need here. > >=20 > > In other words, we need github for FreeBSD. I'm working on some basics > > for this at repos.freebsd.your.org, but had severe VM instabilities > > during the last weeks. > > I have to admit that this crossed my mind as soon as I read Julian's > e-mail, especially as I've been keeping my local FreeBSD patches in > a version-controlled tree for the past ten years - first CVS, then > Subversion, and recently Git. > > Now, is there a reason we couldn't just use Gitorious? :) I don't know enough to comment re. Git versus others, but there's a comparative feature table of 4 : SVN HG GIT MTN at http://wiki.freebsd.org/VersionControl I take the point that patches would be better in source code control systems, not raw (BTW which revisions my own patches apply to, is currently in (mostly sym-linked) names, I'd convert to whatever if we achieve a standard. ) Taking into account all written, how about this: http://www.berklix.com/~jhs/src/bsd/indexes.html Anything to change Or add to format? Any entries to add ? If agreeable here ? I'd then float it to hackers@ to get more entries, then offer it to be adopted into wiki if possible. Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, or HTML or base 64. Avoid top posting, it cripples itemised cumulative responses. From owner-freebsd-arch@FreeBSD.ORG Thu Dec 30 06:45:19 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB880106564A for ; Thu, 30 Dec 2010 06:45:19 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 4F1D98FC1F for ; Thu, 30 Dec 2010 06:45:19 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3ACFB.dip.t-dialin.net [87.179.172.251]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id F2B69844027; Thu, 30 Dec 2010 07:45:07 +0100 (CET) Received: from unknown (IO.Leidinger.net [192.168.2.110]) by outgoing.leidinger.net (Postfix) with ESMTP id 42D651418; Thu, 30 Dec 2010 07:45:03 +0100 (CET) Date: Thu, 30 Dec 2010 07:45:00 +0100 From: Alexander Leidinger To: Garrett Cooper Message-ID: <20101230074500.000040d5@unknown> In-Reply-To: References: <201012220852.oBM8q2Qi039123@lurza.secnetix.de> <20101222123834.GN23098@acme.spoerlein.net> <20101227131140.H6126@maildrop.int.zabbadoz.net> <20101229000130.000052f4@unknown> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: F2B69844027.A9086 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1, required 6, autolearn=disabled, ALL_TRUSTED -1.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1294296308.57989@b+VosxQmolVXikD0H3pJdg X-EBL-Spam-Status: No Cc: "Bjoern A. Zeeb" , freebsd-arch@freebsd.org Subject: Re: Schedule for releases X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Dec 2010 06:45:19 -0000 On Wed, 29 Dec 2010 03:17:41 -0800 Garrett Cooper wrote: > On Tue, Dec 28, 2010 at 3:01 PM, Alexander Leidinger > wrote: > >> We still lack the parts that would tell us something in the last > >> week or last 24 hours caused a regression that made my > >> TCP/NFS/ZFS/UFS/ n% slower. =A0Kris had been doing a > >> good job in the past but as time shows we need more people, > >> different setups, ... > > > > We do not lack the parts, we lack someone to take the parts and get > > them up and running. See: > > =A0http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02819= .html > > =A0http://www.mail-archive.com/freebsd-performance@freebsd.org/msg02821= .html >=20 > No: according to Erik in the first link, the parts were never fully > integrated and the project as a whole was never completed due to lack > of hardware and time. Some bits are also just directions as well as I understand this link (and my maybe flaky memory when he presented his work the first time) as he didn't had the time and hardware to set up the automatic benchmarking suite to run periodically "forever". And what you call directions looks to me like documentation. As Erik is participating in this thread, I let the last word for him. :) > Erik says in his reply to the thread. After seeing enough folks > prematurely claim victory within my own company, I know enough that > nothing is complete unless it works (to some basic degree, i.e. it can > be replicated by someone other than the original author of the > infrastructure or code using the directions provided from scratch). I agree. Erik can you make your work available for download? If not, I offer webspace (either on my private site or in my area of the FreeBSD owned webspace) to make it possible to donwload it. > >> It's not only "compiles", "boots", but also the formerly in this > >> thread mentioned "works correctly" and in addition to that the > >> "works well as expected" or "works better than before" - > >> hopefully;). > > > > Maybe this could also be used to run the regression tests as one of > > the benchmarks. If yes: As Robert mentioned, we can not go and tell > > to run them all in one command (ATM), but we could have each of > > them as a different benchmark. >=20 > Regression tests should be stable. Regression tests shouldn't crash > targets (in general). Benchmarks and stress tests can do that. We need I was referring to his explanation that there is not central place to kick off all of them. I agree that the should be stable, but in case of a regression a crash may be a possible failure scenario (which can then be fixed by fixing the regression). > IMO we should have something like: >=20 > make checkworld > make checkkernel I agree that this would be nice. I think this is one possible implemenation of what Robert whas referring to when he told that there is no central place to start all the regression tests. > that would go off and run functional regression tests, and something > like: >=20 > make stressworld > make stresskernel >=20 > that would to similar for benchmarks as well. Having stuff tied Depending on the benchmark you need some kind of setup which can not be done by such targets (if we assume that this shall be self-contained and possible just by running those commands instead of doing some setup of 3rd part software like databases or webservers before). Bye, Alexander.