From owner-freebsd-arch@FreeBSD.ORG Sun Dec 18 01:06:48 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15D08106564A for ; Sun, 18 Dec 2011 01:06:48 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 24D058FC0C for ; Sun, 18 Dec 2011 01:06:45 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 16C206E4C; Sun, 18 Dec 2011 01:06:42 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 7E6FB88FB; Sun, 18 Dec 2011 02:06:38 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: "Poul-Henning Kamp" References: <85477.1324155737@critter.freebsd.dk> Date: Sun, 18 Dec 2011 02:06:38 +0100 In-Reply-To: <85477.1324155737@critter.freebsd.dk> (Poul-Henning Kamp's message of "Sat, 17 Dec 2011 21:02:17 +0000") Message-ID: <86ty4y4rj5.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , arch@freebsd.org, threads@freebsd.org, Ed Schouten Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2011 01:06:48 -0000 "Poul-Henning Kamp" writes: > Ohhh, but I know: Lets make a rival to the POSIX threads, we can do it > much better and slightly incompatible, big market there I'm sure. That's not the point. The point is that C until now did not have a concurrency model. The threading API in itself is not important; I'm sure the committee knows perfectly well that nobody is going to use it. What's important is that the standard now defines how C behaves in a concurrent environment. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sun Dec 18 11:55:10 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B76D106566B; Sun, 18 Dec 2011 11:55:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id E4A6E8FC16; Sun, 18 Dec 2011 11:55:09 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBIBrR7V052781 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 18 Dec 2011 13:53:27 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBIBrRBJ063596; Sun, 18 Dec 2011 13:53:27 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBIBrQ7A063595; Sun, 18 Dec 2011 13:53:26 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 18 Dec 2011 13:53:26 +0200 From: Kostik Belousov To: Dag-Erling Sm??rgrav Message-ID: <20111218115326.GD50300@deviant.kiev.zoral.com.ua> References: <85477.1324155737@critter.freebsd.dk> <86ty4y4rj5.fsf@ds4.des.no> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ub6V6mHXHR6pEqUf" Content-Disposition: inline In-Reply-To: <86ty4y4rj5.fsf@ds4.des.no> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Kostik Belousov , arch@freebsd.org, Poul-Henning Kamp , threads@freebsd.org, Ed Schouten Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2011 11:55:10 -0000 --ub6V6mHXHR6pEqUf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Dec 18, 2011 at 02:06:38AM +0100, Dag-Erling Sm??rgrav wrote: > "Poul-Henning Kamp" writes: > > Ohhh, but I know: Lets make a rival to the POSIX threads, we can do it > > much better and slightly incompatible, big market there I'm sure. >=20 > That's not the point. The point is that C until now did not have a > concurrency model. The threading API in itself is not important; I'm > sure the committee knows perfectly well that nobody is going to use it. > What's important is that the standard now defines how C behaves in a > concurrent environment. Well, the reverse was exactly _my_ point. I cannot find the description of how the abstract C machine behaves, in the presence of multiple threads of execution. The atomics chapter covers only some special operations, which are added in the new revision. E.g., there is absolutely no mention of the memory changes visibility, or guarantees of atimicity of the assignments/reads etc. IMO, the threading was slapped nearby, and the standard is not useful as-is. I am sorry if I missed the parts. --ub6V6mHXHR6pEqUf Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7t1DUACgkQC3+MBN1Mb4gUAwCg2c2BM7NGPRcI/wlKmRaZqAJR EtUAoMuh2Gm61rRBSXn5W+VfZsSlM8Ma =3FGP -----END PGP SIGNATURE----- --ub6V6mHXHR6pEqUf-- From owner-freebsd-arch@FreeBSD.ORG Sun Dec 18 13:27:44 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C109106574A; Sun, 18 Dec 2011 13:27:44 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id ECF1C8FC17; Sun, 18 Dec 2011 13:27:43 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id E2B0A3592DF; Sun, 18 Dec 2011 14:27:42 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id B56AB28468; Sun, 18 Dec 2011 14:27:42 +0100 (CET) Date: Sun, 18 Dec 2011 14:27:42 +0100 From: Jilles Tjoelker To: Kostik Belousov Message-ID: <20111218132742.GA52983@stack.nl> References: <20111216214913.GA1771@hoeg.nl> <20111216220914.GW50300@deviant.kiev.zoral.com.ua> <20111216221959.GB1771@hoeg.nl> <20111216223126.GX50300@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111216223126.GX50300@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@freebsd.org, Ed Schouten , threads@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2011 13:27:44 -0000 On Sat, Dec 17, 2011 at 12:31:26AM +0200, Kostik Belousov wrote: > On Fri, Dec 16, 2011 at 11:19:59PM +0100, Ed Schouten wrote: > > * Kostik Belousov , 20111216 23:09: > > > If application that does not use the new interface supposed to be > > > able to implement function with new names, then the not-underscored > > > symbols must be weak. > > For example when an application wants to implement its own functions > > that are named thrd_*(), for example? > Yes. The realistic example is the code written to C99/SUSv4 conformance > that happens to define thrd_. > It might be that easiest solution is to put the functions into > separate library, besides defining them weak. Another idea is to implement the functions as static inline (with the possible exception of thrd_create() and perhaps some more). This pollutes the namespace of C1x programs with pthread_* though. > > > Do you have reference to the draft ? > > Yes, sure: > > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf > BTW, it looks not very useful to add a bunch of threading functions > without at least trying to specify the memory model. I see a discussion of the memory model in 5.1.2.4 Multi-threaded executions and data races. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Sun Dec 18 14:42:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6560E1065670; Sun, 18 Dec 2011 14:42:09 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id CD9578FC21; Sun, 18 Dec 2011 14:42:08 +0000 (UTC) Received: from lstewart1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 923B17E84F; Mon, 19 Dec 2011 01:42:05 +1100 (EST) Message-ID: <4EEDFBBD.9050809@freebsd.org> Date: Mon, 19 Dec 2011 01:42:05 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111016 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <201111210417.pAL4HOdi023556@svn.freebsd.org> <4ED8575D.2010405@freebsd.org> <201112021927.25234.jkim@FreeBSD.org> <201112022002.49618.jkim@FreeBSD.org> <4EDB6814.9040403@freebsd.org> <405E9C58-F999-45C2-BC20-81ED4CA8A3E9@cubinlab.ee.unimelb.edu.au> <4EE9679A.70909@freebsd.org> In-Reply-To: <4EE9679A.70909@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: Julien Ridoux , Benjamin Kaduk , Jung-uk Kim Subject: Re: System clock APIs and feed-forward clock integration with BPF X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2011 14:42:09 -0000 > On 12/15/11 01:30, Julien Ridoux wrote: >> >> On 04/12/2011, at 11:31 PM, Lawrence Stewart wrote: >> >>> On 12/03/11 12:02, Jung-uk Kim wrote: >>>> On Friday 02 December 2011 07:27 pm, Jung-uk Kim wrote: >>>>> On Thursday 01 December 2011 11:43 pm, Lawrence Stewart wrote: >>>>>> On 12/02/11 03:43, Jung-uk Kim wrote: >>>>>>> On Thursday 01 December 2011 10:11 am, Lawrence Stewart wrote: >>>>>>>> On 11/30/11 05:09, Jung-uk Kim wrote: >>>>>>>>> On Tuesday 29 November 2011 11:13 am, Lawrence Stewart wrote: >>> [snip] >>>>>>>>>> Here's the first of the patches: >>>>>>>>>> >>>>>>>>>> http://people.freebsd.org/~lstewart/patches/misc/ffclock_bpf >>>>>>>>>> _i nt act abi_10.x.r228130.patch >>>>>>>>> >>>>>>>>> I only glanced at it but it looks very close to what I wanted >>>>>>>>> to suggest. >>>>>>>> >>>>>>>> Final candidate patch is at: >>>>>>>> >>>>>>>> http://people.freebsd.org/~lstewart/patches/misc/ffclock_bpf_i >>>>>>>> nt act abi_10.x.r228180.patch >>>>>>>> >>>>>>>> Assuming it passes the "make tinderbox" build I'm currently >>>>>>>> running and no further input is received from interested >>>>>>>> parties, I plan to commit it in ~10 hours. >>>>>>>> >>>>>>>> Changes since the r228130 patch I sent previously: >>>>>>>> >>>>>>>> - The new flags in bpf.h are added unconditionally so that >>>>>>>> they can always be referenced at compile time and a decision >>>>>>>> made at runtime as to whether a flag will be set or not. Using >>>>>>>> one of the new flags when the kernel doesn't have FFCLOCK >>>>>>>> compiled in results in the flag being ignored. An app should >>>>>>>> check for the existence of the "ffclock" kernel feature or the >>>>>>>> "kern.sysclock" sysctl tree before attempting to use the >>>>>>>> flags. >>>>>>>> >>>>>>>> - This patch will hopefully be MFCed at some point, so I added >>>>>>>> a CTASSERT to bpf.c to ensure that the ABI of structs >>>>>>>> bpf_hdr32, bpf_hdr and bpf_xhdr remains intact when FFCLOCK is >>>>>>>> enabled and the union of a ffcounter and struct >>>>>>>> timeval32/timeval/bpf_ts is switched in. >>>>>>>> >>>>>>>> - bpf_gettime() more comprehensively covers all the possible >>>>>>>> cases of flag combination and does sensible things for each >>>>>>>> case (even though some cases are rather silly). >>>>>>>> >>>>>>>> - The snippet of code at the beginning of catchpacket() that >>>>>>>> was manipulating the struct bintime derived from bpf_gettime() >>>>>>>> was gross and has been removed in favour of selecting the >>>>>>>> right {get}bin{up}time() function call in bpf_gettime(). >>>>>>> >>>>>>> I did that to reduce branching. Since you are introducing more >>>>>>> branches, it warrants a function pointer now. >>>>> >>>>> There's another reason, BTW. If mbufs are tagged with timestamps >>>>> (where only monotonic timestamps are allowed), they must be >>>>> converted within the bpf.c. I forgot all the details. :-( >>> >>> We should document this knowledge in some code comments. >>> >>>>>> I see, thanks for the explanation. Could you elaborate a bit more >>>>>> about how you would implement the function pointer idea? I'm also >>>>>> curious in the !FFCLOCK case just how much overhead having the >>>>>> 2-layer nested if/else adds. I'm not an very optimisation savvy >>>>>> person, but I'm wondering if it's actually worth micro-optimising >>>>>> this code. >>>>>> >>>>>> My initial thoughts about your function pointer idea lead to >>>>>> adding a function pointer in the bpf_d and setting it to the >>>>>> appropriate function to get the timestamp from at bpf_d creation >>>>>> or ioctl time. Whilst I like this idea, I can't see how it would >>>>>> work given that the various functions involved in time/ffcounter >>>>>> stamp generation all have different signatures. >>>>>> >>>>>> We could have multiple variants of bpf_gettime() which each call >>>>>> the appropriate underlying functions to generate the appropriate >>>>>> stamp. Would add quite a lot of code but would reduce the >>>>>> overhead of calling bpf_gettime() to an indirect function call + >>>>>> the underlying stamp generation function call. This also solves >>>>>> the problem of multiple function signatures. >>>>> >>>>> Please see my patch: >>>>> >>>>> http://people.freebsd.org/~jkim/bpf_ffclock.diff >>>> >>>> I booted up the kernel and found it just crashes because of stupid >>>> typos. Now a new patch is uploaded in place. Sorry. >>>> >>>> Anyway, I see no regression nor ABI breakage. :-) >>> >>> struct bpf_d being part of the ABI was the main thing I was concerned >>> about, so given that it isn't I like your approach a lot. >>> >>> As noted by Julien, this approach does introduce problems with >>> respect to the follow up patch that adds a permanent bh_ffcounter >>> member to the bpf header. I thought the fromclock() API would >>> sufficiently meet the needs of consumers like BPF, but if we were to >>> proceed with something like Jung-uk's proposed patch I don't think >>> that's true anymore. >>> >>> We decided to bite the bullet and devise an API that is more compact >>> and can return all appropriate time information from all underlying >>> sysclocks in an efficient manner - something like a more generic >>> version of ffclock_abstime(). Julien and I spent quite some time >>> today nutting out details, and Julien has done a proof of concept >>> patch against my proposed BPF patch which looks good as a starting >>> point for discussion: >>> >>> http://www.cubinlab.ee.unimelb.edu.au/~jrid/patches/r228254/sysclock_snap.patch >>> >>> >>> It needs a bit more work and should be split into two patches (one >>> introducing the API and the other being the BPF changes). >>> >>> With something like this though, the BPF patch becomes radically >>> simplified in the FFCLOCK case. We don't even need a function pointer >>> cached in the bpf_d anymore, but could cache a struct sysclock_snap >>> there instead if we really wanted to minimise overhead in bpf_gettime(). >>> >>> We'll have a go at refining the patch tomorrow hopefully, but wanted >>> to put this out there for early consideration. >> >> >> Hi Jung-uk (and all), >> >> It took us a bit of time but we have a new patch. There are a lot of >> changes in the patch, and I will start by summarising our motivation >> for the changes. Hopefully I won't forget anything, but if I leave >> something unclear, don't hesitate to ask Lawrence or me. >> >> Starting with the easy bits, we drop the union in the BPF header and >> followed something similar to what Ben and you proposed. >> >> Next, the core motivation was to ensure that the timestamping function >> be called once only per packet captured, while keeping the ability to >> read both clocks and return the time from one of the two to the user. >> This forced us to redesign the timestamping function and one thing >> leading to another a fair bit of the BPF code. >> >> There 3 orthogonal parameters to the timestamping function in the BPF >> context: >> - which clock is used to return the time >> - what is the format of the value returned >> - and a performance issue (nothing to do with timestamping or >> timekeeping): reading the hardware counter or not >> >> We handle the first one with a new sysclock_getsnapshot() which saves >> both clock parameters when timestamping and potentially reads the >> hardware counter. >> The second one has been covered in previous discussion with the >> addition of BPF_T_FFCOUNTER. >> >> The performance issue created problems. In the current code, the order >> with which the BPF devices are open on one interface impacts the >> timestamping function. Let's consider two BPF devices respectively >> open with BPF_TSTAMP_NORMAL and BPF_TSTAMP_FAST quality. If NORMAL is >> open after FAST, it is at the head of the list, the hardware counter >> is read, and FAST benefits from higher quality timestamps. If FAST is >> open after NORMAL, the timestamping function is called once and return >> last kernel tick time, then it is called a second time for NORMAL, >> this time accessing the hardware counter. >> >> None of these cases are satisfactory. This lead to inconsistent >> behaviour based on the order the BPF are open. An application can >> impact the settings of other BPF devices and other applications / >> kernel consumers. The timestamping suffers from higher latency and >> worse, higher variability of the latency, which is bad. >> >> We then took a different approach, which is inspired by your work on >> BPF. We believe the FAST/NORMAL settings should have 2 level of >> granularity: system wide and per interface. The system wide is the >> default behaviour (for example all BPF devices are NORMAL) and the >> interface one super-seeds the system one. This is implemented by the >> use of new sysctl living in the net.bpf tree. >> >> This enables many scenarios such as: >> - system default is FAST to improve performance because the system >> uses a slow timecounter (eg HPET or ACPI). >> - the system has 3 interfaces. Interface 2 is put in NORMAL mode (need >> for accuracy but not much traffic), and the last can do hardware >> timestamping and is put in BPF_TSTAMP_EXTERNAL mode. >> >> This then naturally led to store the performance setting >> (none/fast/normal/external) for each interface instead of each BPF >> descriptor. >> >> Each BPF descriptor still maintains the other 2 settings: the choice >> of clock and the format of the timestamp independently from each other. >> >> We believe this new patch keeps the spirit of the changes you proposed >> and improves it while accomodating the changes required by the >> feed-forward clock and future evolutions with things like IEEE 1588 >> compatible NICs. >> >> We plan to push this patch as soon as possible. We would really >> appreciate if you could comment on this long email and pach very soon. >> >> The patch can be found here: >> http://www.cubinlab.ee.unimelb.edu.au/~jrid/patches/r228429-v4/ffclock_bpf_intactabi.patch Candidate commit patches can be found at: http://people.freebsd.org/~lstewart/patches/misc/sysclockgetsnap_10.x.r228435.patch http://people.freebsd.org/~lstewart/patches/misc/ffclock_bpf_intactabi_10.x.r228435.patch Relative to Julien's patch, I've split his patch into two, updated bpf.4 documentation, added a net.bpf.tscfg.default sysctl to set the default time stamp for all BPF attached interfaces, and added some cleanup and bug fixes. If I don't hear any objections or receive any requests to hold off committing to allow more time for review, I plan to commit these patches in the next few days. Cheers, Lawrence From owner-freebsd-arch@FreeBSD.ORG Sun Dec 18 15:28:53 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4240106566B; Sun, 18 Dec 2011 15:28:53 +0000 (UTC) (envelope-from BATV+6695f58bc8be10e6006b+3038+infradead.org+hch@bombadil.srs.infradead.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2001:4830:2446:ff00:4687:fcff:fea6:5117]) by mx1.freebsd.org (Postfix) with ESMTP id 89A958FC16; Sun, 18 Dec 2011 15:28:53 +0000 (UTC) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RcIfQ-0000eS-Pf; Sun, 18 Dec 2011 15:28:48 +0000 Date: Sun, 18 Dec 2011 10:28:48 -0500 From: Christoph Hellwig To: Poul-Henning Kamp Message-ID: <20111218152848.GA460@infradead.org> References: <20111216223126.GX50300@deviant.kiev.zoral.com.ua> <85477.1324155737@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <85477.1324155737@critter.freebsd.dk> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html Cc: Kostik Belousov , arch@freebsd.org, threads@freebsd.org, Ed Schouten Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2011 15:28:54 -0000 On Sat, Dec 17, 2011 at 09:02:17PM +0000, Poul-Henning Kamp wrote: > A "assert mutex is held" facility ? Or a full FreeBSD / Linux lockdep style lock order validation tool. I still don't understand how your are supposed to write correct userspace code using more than a hand full of synchronization primitives without that. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 18 16:15:05 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A6881065673; Sun, 18 Dec 2011 16:15:05 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from mx0.hoeg.nl (mx0.hoeg.nl [IPv6:2a01:4f8:101:5343::aa]) by mx1.freebsd.org (Postfix) with ESMTP id E590D8FC14; Sun, 18 Dec 2011 16:15:04 +0000 (UTC) Received: by mx0.hoeg.nl (Postfix, from userid 1000) id CEE6A2A28CD0; Sun, 18 Dec 2011 17:15:03 +0100 (CET) Date: Sun, 18 Dec 2011 17:15:03 +0100 From: Ed Schouten To: Jilles Tjoelker Message-ID: <20111218161503.GG1771@hoeg.nl> References: <20111216214913.GA1771@hoeg.nl> <20111216220914.GW50300@deviant.kiev.zoral.com.ua> <20111216221959.GB1771@hoeg.nl> <20111216223126.GX50300@deviant.kiev.zoral.com.ua> <20111218132742.GA52983@stack.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vhOf6eAHdfH9MSjZ" Content-Disposition: inline In-Reply-To: <20111218132742.GA52983@stack.nl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2011 16:15:05 -0000 --vhOf6eAHdfH9MSjZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Jilles, * Jilles Tjoelker , 20111218 14:27: > Another idea is to implement the functions as static inline (with the > possible exception of thrd_create() and perhaps some more). This > pollutes the namespace of C1x programs with pthread_* though. Hmmm... Indeed. This would change the entire C1X threading API simply into a compile-time translation mechanism to pthreads. Unfortunately, it would make things like debugging a bit harder, as placing breakpoints on the functions become a bit harder. Are there any objections to such an approach? --=20 Ed Schouten WWW: http://80386.nl/ --vhOf6eAHdfH9MSjZ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iQIcBAEBAgAGBQJO7hGHAAoJEG5e2P40kaK7XOsP/Rdn46756oot1DgWAvmLDGt8 pC2RhmwqU73w4vo9I3mRErTrbpQ398FUKMd/gouE4oqu4nOqVO/Hws2NjzmynWjv uldqrVQ1dSrS0qNrSuo9SP1D5dAdZkoa1DjS19Fp4eyxk5ipS9ZvG2oaWe8mrWto xiFLYFzAUUFOC3p+V+94vScziL+gyraQQZchfuSUkZHTQfBagmCfkMIxw6mACVpy 6y20xAKtyyPcfc76BesUHts/Tpx3vjPbMrrVP5yAJRtWRLDaDkfyn6+UhwGs3la5 rks+2saHfvtbAK3S6mHo1MReoQgfu7fy0OHzD6QzhIcwhbHyPWJHiX8lvkXVapJG 8dlEEFWQZNIKJB1+dEwVYWRGRNvXllxrxiuOvElnFSX8OONWCn8iSEnFKSP/+EWG L0n0Z5NNujHDewJ4LUpnZcad8Y8ck9w+FTy+WrF3SbXz7b+DC5Kd6ZFRWd+FMGUv epBgyeDo5q9dduvdaY2eNBQ6KjtOFvmGXo97eCh8laZt8IjNG33EoTQw+OF5zMup ZccnP2v8KbaQ286w6N8T4545Mn/qUi/KHtGzBCTIDrWq8ekWH4oWNyUzedaXE31R smm5XPNARFdbf3j6HFkS2JHZ4aU53ipS9BKg+gbHtRRYWHR24/FQ7FDLDazH1rpK kt5legthEuTkBZ3EH7eJ =lmo/ -----END PGP SIGNATURE----- --vhOf6eAHdfH9MSjZ-- From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 10:57:24 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75DF7106566B; Mon, 19 Dec 2011 10:57:24 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 327E98FC0A; Mon, 19 Dec 2011 10:57:23 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id CF1585DC9; Mon, 19 Dec 2011 10:57:21 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBJAvLjX058924; Mon, 19 Dec 2011 10:57:21 GMT (envelope-from phk@phk.freebsd.dk) To: Tijl Coosemans From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 19 Dec 2011 11:52:17 +0100." <201112191152.22907.tijl@coosemans.org> Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 19 Dec 2011 10:57:21 +0000 Message-ID: <58923.1324292241@critter.freebsd.dk> Cc: threads@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 10:57:24 -0000 In message <201112191152.22907.tijl@coosemans.org>, Tijl Coosemans writes: >> Big/Little Endian API ? >> >> Naah, nobody moves binary data between computers. > >Yes, but rather than having the programmer remember when to swap bytes, >it would be better if he could just declare a variable big/little >endian and have the compiler figure it out. You'd think so, wouldn't you ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 11:02:42 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C44321065670; Mon, 19 Dec 2011 11:02:42 +0000 (UTC) (envelope-from tijl@coosemans.org) Received: from mailrelay002.isp.belgacom.be (mailrelay002.isp.belgacom.be [195.238.6.175]) by mx1.freebsd.org (Postfix) with ESMTP id 1F1D28FC13; Mon, 19 Dec 2011 11:02:41 +0000 (UTC) X-Belgacom-Dynamic: yes X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EAGYV705bsVo6/2dsb2JhbABDqROCS4EGgXIBAQVWIxALGC45HgeIDrd5jAQEiAOfKg Received: from 58.90-177-91.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([91.177.90.58]) by relay.skynet.be with ESMTP; 19 Dec 2011 11:52:26 +0100 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.14.5/8.14.5) with ESMTP id pBJAqPv1003189; Mon, 19 Dec 2011 11:52:25 +0100 (CET) (envelope-from tijl@coosemans.org) From: Tijl Coosemans To: freebsd-arch@freebsd.org, threads@freebsd.org Date: Mon, 19 Dec 2011 11:52:17 +0100 User-Agent: KMail/1.13.7 (FreeBSD/9.0-RC2; KDE/4.7.3; i386; ; ) References: <85477.1324155737@critter.freebsd.dk> In-Reply-To: <85477.1324155737@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart6161652.z3pzWeIXpR"; protocol="application/pgp-signature"; micalg=pgp-sha256 Content-Transfer-Encoding: 7bit Message-Id: <201112191152.22907.tijl@coosemans.org> Cc: Poul-Henning Kamp Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 11:02:42 -0000 --nextPart6161652.z3pzWeIXpR Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable On Saturday 17 December 2011 22:02:17 Poul-Henning Kamp wrote: > Big/Little Endian API ? >=20 > Naah, nobody moves binary data between computers. Yes, but rather than having the programmer remember when to swap bytes, it would be better if he could just declare a variable big/little endian and have the compiler figure it out. --nextPart6161652.z3pzWeIXpR Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iF4EABEIAAYFAk7vF2YACgkQfoCS2CCgtiuxZAD5Ac5JwKs79JJnqR+Wn0Juz4DQ T4Q/LMvyWINAgxW+iiYBAIdHumR1292ZBLPl/cyfWPWpkMRjooBBy8epG+lzrzsk =XjtJ -----END PGP SIGNATURE----- --nextPart6161652.z3pzWeIXpR-- From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 15:05:46 2011 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B46D3106566C; Mon, 19 Dec 2011 15:05:46 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from zim.MIT.EDU (ZIM.MIT.EDU [18.95.3.101]) by mx1.freebsd.org (Postfix) with ESMTP id 776898FC0A; Mon, 19 Dec 2011 15:05:46 +0000 (UTC) Received: from zim.MIT.EDU (localhost [127.0.0.1]) by zim.MIT.EDU (8.14.5/8.14.2) with ESMTP id pBJF5hQc031442; Mon, 19 Dec 2011 10:05:43 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by zim.MIT.EDU (8.14.5/8.14.2/Submit) id pBJF5haX031441; Mon, 19 Dec 2011 10:05:43 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Mon, 19 Dec 2011 10:05:43 -0500 From: David Schultz To: Ed Schouten Message-ID: <20111219150543.GA31384@zim.MIT.EDU> Mail-Followup-To: Ed Schouten , Jilles Tjoelker , Kostik Belousov , threads@freebsd.org, arch@freebsd.org References: <20111216214913.GA1771@hoeg.nl> <20111216220914.GW50300@deviant.kiev.zoral.com.ua> <20111216221959.GB1771@hoeg.nl> <20111216223126.GX50300@deviant.kiev.zoral.com.ua> <20111218132742.GA52983@stack.nl> <20111218161503.GG1771@hoeg.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111218161503.GG1771@hoeg.nl> Cc: Kostik Belousov , threads@FreeBSD.ORG, Jilles Tjoelker , arch@FreeBSD.ORG Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 15:05:46 -0000 On Sun, Dec 18, 2011, Ed Schouten wrote: > Hi Jilles, > > * Jilles Tjoelker , 20111218 14:27: > > Another idea is to implement the functions as static inline (with the > > possible exception of thrd_create() and perhaps some more). This > > pollutes the namespace of C1x programs with pthread_* though. > > Hmmm... Indeed. This would change the entire C1X threading API simply > into a compile-time translation mechanism to pthreads. Unfortunately, it > would make things like debugging a bit harder, as placing breakpoints on > the functions become a bit harder. > > Are there any objections to such an approach? Technically there's a requirement that external definitions be provided. Applications that break if you don't are pretty rare, but why not do it right by providing some wrappers or symbol aliases in the library? Nothing precludes you from also defining a non-static inline function in the header file (but note that you may need an ifdef to handle the differences between GNU and C99 inlines.) From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 15:14:49 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86699106566B for ; Mon, 19 Dec 2011 15:14:49 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 417DF8FC15 for ; Mon, 19 Dec 2011 15:14:48 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id pBJFEjS5042732; Mon, 19 Dec 2011 10:14:45 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.6 (mail.netplex.net [204.213.176.10]); Mon, 19 Dec 2011 10:14:46 -0500 (EST) Date: Mon, 19 Dec 2011 10:14:45 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: David Schultz In-Reply-To: <20111219150543.GA31384@zim.MIT.EDU> Message-ID: References: <20111216214913.GA1771@hoeg.nl> <20111216220914.GW50300@deviant.kiev.zoral.com.ua> <20111216221959.GB1771@hoeg.nl> <20111216223126.GX50300@deviant.kiev.zoral.com.ua> <20111218132742.GA52983@stack.nl> <20111218161503.GG1771@hoeg.nl> <20111219150543.GA31384@zim.MIT.EDU> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Kostik Belousov , Ed Schouten , arch@freebsd.org, Jilles Tjoelker , threads@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 15:14:49 -0000 On Mon, 19 Dec 2011, David Schultz wrote: > On Sun, Dec 18, 2011, Ed Schouten wrote: >> Hi Jilles, >> >> * Jilles Tjoelker , 20111218 14:27: >>> Another idea is to implement the functions as static inline (with the >>> possible exception of thrd_create() and perhaps some more). This >>> pollutes the namespace of C1x programs with pthread_* though. >> >> Hmmm... Indeed. This would change the entire C1X threading API simply >> into a compile-time translation mechanism to pthreads. Unfortunately, it >> would make things like debugging a bit harder, as placing breakpoints on >> the functions become a bit harder. >> >> Are there any objections to such an approach? > > Technically there's a requirement that external definitions be > provided. Applications that break if you don't are pretty rare, but > why not do it right by providing some wrappers or symbol aliases in > the library? Nothing precludes you from also defining a non-static > inline function in the header file (but note that you may need an ifdef > to handle the differences between GNU and C99 inlines.) I agree, we probably want external definitions in case one needs to interface to the functions from a different language. Also, please symbol version the library. -- DE From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 17:09:18 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2978106564A for ; Mon, 19 Dec 2011 17:09:18 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 98A5E8FC17 for ; Mon, 19 Dec 2011 17:09:18 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 49E1F46B09; Mon, 19 Dec 2011 12:09:18 -0500 (EST) Received: from John-Baldwins-MacBook-Air.local (c-68-36-150-83.hsd1.nj.comcast.net [68.36.150.83]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C7EC2B959; Mon, 19 Dec 2011 12:09:17 -0500 (EST) Message-ID: <4EEF6FC1.8020306@FreeBSD.org> Date: Mon, 19 Dec 2011 12:09:21 -0500 From: John Baldwin User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: gljennjohn@googlemail.com References: <201112161559.36428.jhb@freebsd.org> <20111217120820.5c2d0ee4@ernst.jennejohn.org> In-Reply-To: <20111217120820.5c2d0ee4@ernst.jennejohn.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 19 Dec 2011 12:09:17 -0500 (EST) Cc: freebsd-arch@freebsd.org Subject: Re: TASK_INITIALIZER() X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 17:09:18 -0000 On 12/17/11 6:08 AM, Gary Jennejohn wrote: > On Fri, 16 Dec 2011 15:59:36 -0500 > John Baldwin wrote: > >> Any objection to adding a macro to make it easy to statically initialize task >> structures (similar to the initializer macros in)? This allows >> global tasks to be statically initalized without requiring a dedicated >> SYSINIT() routine. >> >> Index: share/man/man9/Makefile >> =================================================================== >> --- share/man/man9/Makefile (revision 228534) >> +++ share/man/man9/Makefile (working copy) >> @@ -1250,6 +1250,7 @@ >> sysctl_ctx_init.9 sysctl_ctx_free.9 >> MLINKS+=SYSINIT.9 SYSUNINIT.9 >> MLINKS+=taskqueue.9 TASK_INIT.9 \ >> + taskqueue.9 TASK_INITIALIZER.9 \ >> taskqueue.9 taskqueue_cancel.9 \ >> taskqueue.9 taskqueue_create.9 \ >> taskqueue.9 taskqueue_create_fast.9 \ >> Index: share/man/man9/taskqueue.9 >> =================================================================== >> --- share/man/man9/taskqueue.9 (revision 228534) >> +++ share/man/man9/taskqueue.9 (working copy) >> @@ -80,6 +80,7 @@ >> .Ft void >> .Fn taskqueue_run "struct taskqueue *queue" >> .Fn TASK_INIT "struct task *task" "int priority" "task_fn_t func" "void >> *context" >> +.Fn TASK_INITIALIZER "int priority" "task_fn_t func" "void *context" >> .Fn TASKQUEUE_DECLARE "name" >> .Fn TASKQUEUE_DEFINE "name" "taskqueue_enqueue_fn enqueue" "void *context" >> "init" >> .Fn TASKQUEUE_FAST_DEFINE "name" "taskqueue_enqueue_fn enqueue" "void >> *context" "init" >> @@ -243,9 +244,14 @@ >> is provided to initialise a >> .Va task >> structure. >> +The >> +.Fn TASK_INITIALIZER >> +macro generates an initializer for a task structure. >> A macro >> .Fn TIMEOUT_TASK_INIT "queue" "timeout_task" "priority" "func" "context" >> -initializes the timeout_task structure. >> +initializes the >> +.Va timeout_task >> +structure. >> The values of >> .Va priority , >> .Va func , >> Index: sys/sys/taskqueue.h >> =================================================================== >> --- sys/sys/taskqueue.h (revision 228534) >> +++ sys/sys/taskqueue.h (working copy) >> @@ -77,6 +77,12 @@ >> void taskqueue_unblock(struct taskqueue *queue); >> int taskqueue_member(struct taskqueue *queue, struct thread *td); >> >> +#define TASK_INITIALIZER(priority, func, context) \ >> + { .ta_pending = 0, \ >> + .ta_priority = (priority), \ >> + .ta_func = (func), \ >> + .ta_context = (context) } >> + >> /* >> * Functions for dedicated thread taskqueues >> */ >> > > Seems like an excellent idea. Are you also planning to replace the > existing SYSINIT() entries? I had not planned on doing so, no. I can maybe look at that however. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 17:19:23 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 371171065672; Mon, 19 Dec 2011 17:19:23 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id E7F228FC12; Mon, 19 Dec 2011 17:19:22 +0000 (UTC) Received: from [10.30.101.53] ([209.117.142.2]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBJHA0NJ049289 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Mon, 19 Dec 2011 10:10:02 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <58923.1324292241@critter.freebsd.dk> Date: Mon, 19 Dec 2011 10:09:53 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <58923.1324292241@critter.freebsd.dk> To: "Poul-Henning Kamp" X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Mon, 19 Dec 2011 10:10:03 -0700 (MST) Cc: threads@freebsd.org, Tijl Coosemans , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 17:19:23 -0000 On Dec 19, 2011, at 3:57 AM, Poul-Henning Kamp wrote: > In message <201112191152.22907.tijl@coosemans.org>, Tijl Coosemans = writes: >=20 >>> Big/Little Endian API ? >>>=20 >>> Naah, nobody moves binary data between computers. >>=20 >> Yes, but rather than having the programmer remember when to swap = bytes, >> it would be better if he could just declare a variable big/little >> endian and have the compiler figure it out. >=20 > You'd think so, wouldn't you ? Intel has a compiler that allows one to declare things are big or little = endian and then things work. A certain large router vendor used it to = port its software that was big endian only at a very deep layer to Intel = x86... Linux marks things as beXX or leXX and uses static analysis to prevent = mixing. There's a lot of prior art for the committee to choose from. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 17:30:56 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 613E2106566C; Mon, 19 Dec 2011 17:30:56 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 1EBE18FC12; Mon, 19 Dec 2011 17:30:55 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 8DF6C5DAA; Mon, 19 Dec 2011 17:30:54 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBJHUos4060234; Mon, 19 Dec 2011 17:30:54 GMT (envelope-from phk@phk.freebsd.dk) To: Warner Losh From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 19 Dec 2011 10:09:53 MST." Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 19 Dec 2011 17:30:50 +0000 Message-ID: <60233.1324315850@critter.freebsd.dk> Cc: threads@freebsd.org, Tijl Coosemans , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 17:30:56 -0000 In message , Warner Losh write s: >There's a lot of prior art for the committee to choose from. Same thing for struct packing to match a protocol specification. And as for assert_mutex_held() I belive some of the prior art is only a few hours longer than the word "mutex". I really wonder what they think they are trying to do some times... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 17:47:21 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2ADA1065675; Mon, 19 Dec 2011 17:47:21 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 70CE38FC16; Mon, 19 Dec 2011 17:47:21 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 6E0787300A; Mon, 19 Dec 2011 18:46:24 +0100 (CET) Date: Mon, 19 Dec 2011 18:46:24 +0100 From: Luigi Rizzo To: Warner Losh Message-ID: <20111219174624.GA13576@onelab2.iet.unipi.it> References: <58923.1324292241@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: threads@freebsd.org, Poul-Henning Kamp , Tijl Coosemans , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 17:47:21 -0000 On Mon, Dec 19, 2011 at 10:09:53AM -0700, Warner Losh wrote: > > On Dec 19, 2011, at 3:57 AM, Poul-Henning Kamp wrote: > > > In message <201112191152.22907.tijl@coosemans.org>, Tijl Coosemans writes: > > > >>> Big/Little Endian API ? > >>> > >>> Naah, nobody moves binary data between computers. > >> > >> Yes, but rather than having the programmer remember when to swap bytes, > >> it would be better if he could just declare a variable big/little > >> endian and have the compiler figure it out. > > > > You'd think so, wouldn't you ? > > Intel has a compiler that allows one to declare things are big or little endian and then things work. A certain large router vendor used it to port its software that was big endian only at a very deep layer to Intel x86... > > Linux marks things as beXX or leXX and uses static analysis to prevent mixing. > > There's a lot of prior art for the committee to choose from. and that would be the definition of a blacklist, right ? :) cheers luigi From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 17:56:12 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAE081065670; Mon, 19 Dec 2011 17:56:12 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 711228FC18; Mon, 19 Dec 2011 17:56:12 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1Rch9e-000733-Rp>; Mon, 19 Dec 2011 18:37:38 +0100 Received: from e178006011.adsl.alicedsl.de ([85.178.6.11] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1Rch9e-0007oE-ND>; Mon, 19 Dec 2011 18:37:38 +0100 Message-ID: <4EEF765D.4090300@zedat.fu-berlin.de> Date: Mon, 19 Dec 2011 18:37:33 +0100 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111109 Thunderbird/8.0 MIME-Version: 1.0 To: Warner Losh References: <58923.1324292241@critter.freebsd.dk> In-Reply-To: X-Enigmail-Version: undefined Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigD4679509587AA80EFD2E8B74" X-Originating-IP: 85.178.6.11 Cc: threads@freebsd.org, Poul-Henning Kamp , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 17:56:12 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigD4679509587AA80EFD2E8B74 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 12/19/11 18:09, Warner Losh wrote: >=20 > On Dec 19, 2011, at 3:57 AM, Poul-Henning Kamp wrote: >=20 >> In message <201112191152.22907.tijl@coosemans.org>, Tijl Coosemans wri= tes: >> >>>> Big/Little Endian API ? >>>> >>>> Naah, nobody moves binary data between computers. >>> >>> Yes, but rather than having the programmer remember when to swap byte= s, >>> it would be better if he could just declare a variable big/little >>> endian and have the compiler figure it out. >> >> You'd think so, wouldn't you ? >=20 > Intel has a compiler that allows one to declare things are big or littl= e endian and then things work. A certain large router vendor used it to = port its software that was big endian only at a very deep layer to Intel = x86... >=20 > Linux marks things as beXX or leXX and uses static analysis to prevent = mixing. >=20 > There's a lot of prior art for the committee to choose from. >=20 > Warner >=20 A silly question: How is the other BSD sibbling, NetBSD, dealing with such things? NetBSD is supposed to run on a trmendous variety of hardware, even a mixture of bigendian and littleenddian and I'm quite sure they must have overcome this probleme anyway. Regrads, Oliver --------------enigD4679509587AA80EFD2E8B74 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iQEcBAEBAgAGBQJO73ZiAAoJEOgBcD7A/5N8Yh4IALIrJvxbQWKmE1/SmOqDz84c WyBoUP6O3Rj816VFXM17Mda/YTG3+L11hYIVqIo/tKZO4knyj11IZ2nZUoiNfc9v PaULHHEVBNw7cNOa/fLUiV7H78dUL7galbEKvhMgKrT9Nbzh06UxlUZmOCZlhZGX 3PvG95m9STJb0Ex2UJnUPim/NmoNDvTOxB8mZjwSfjU89A/nk5a23SlOdsRNFJka XCjWejy0B9CMHGOWMSW/eJ46ePrM6gxpoIDA4vGmIaOlg7jGc7umi1e4dunCMZgY Xbl4gOhixbqaybXrvttRkrWENaA+6uqV5APY17hytK8Tk3sa0PgJQ20DNChrNzA= =FZfG -----END PGP SIGNATURE----- --------------enigD4679509587AA80EFD2E8B74-- From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 18:17:21 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0BF3106566C for ; Mon, 19 Dec 2011 18:17:21 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 6428D8FC08 for ; Mon, 19 Dec 2011 18:17:21 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 1367B73027; Mon, 19 Dec 2011 19:14:10 +0100 (CET) Date: Mon, 19 Dec 2011 19:14:10 +0100 From: Luigi Rizzo To: arch@freebsd.org Message-ID: <20111219181410.GA13742@onelab2.iet.unipi.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: Subject: generic pci device_probe routine ? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 18:17:21 -0000 haven't done device drivers for a while, but i just noticed that pretty much all PCI drivers have their own replica of the *_probe code which does the same exact thing -- define an array of vendor,product entries, and lookup the entry in the array. Would it make sense (or, do we have already) to have common struct and routine, similar to what we have in usb_lookup.c ? cheers luigi From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 18:43:56 2011 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF8C2106568D for ; Mon, 19 Dec 2011 18:43:56 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 7C1F78FC0C for ; Mon, 19 Dec 2011 18:43:56 +0000 (UTC) Received: from [10.30.101.53] ([209.117.142.2]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBJIZr2S050349 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Mon, 19 Dec 2011 11:35:54 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <20111219181410.GA13742@onelab2.iet.unipi.it> Date: Mon, 19 Dec 2011 11:35:47 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <19FB5614-B70A-4FDB-A7ED-73A7B5C5970C@bsdimp.com> References: <20111219181410.GA13742@onelab2.iet.unipi.it> To: Luigi Rizzo X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Mon, 19 Dec 2011 11:35:55 -0700 (MST) Cc: arch@FreeBSD.org Subject: Re: generic pci device_probe routine ? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 18:43:56 -0000 On Dec 19, 2011, at 11:14 AM, Luigi Rizzo wrote: > haven't done device drivers for a while, but i just noticed that > pretty much all PCI drivers have their own replica of the *_probe code > which does the same exact thing -- define an array of vendor,product > entries, and lookup the entry in the array. > Would it make sense (or, do we have already) to have common struct > and routine, similar to what we have in usb_lookup.c ? It would make sense. Model it after the PC Card one, however, since = that one also include the size of the elements to allow for piggybacking = data for the driver in the table. It is the biggest stumbling block to allowing automated driver loading = today. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 18:48:58 2011 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 765EB106564A for ; Mon, 19 Dec 2011 18:48:58 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 3B37D8FC08 for ; Mon, 19 Dec 2011 18:48:57 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 67A107300A; Mon, 19 Dec 2011 20:05:24 +0100 (CET) Date: Mon, 19 Dec 2011 20:05:24 +0100 From: Luigi Rizzo To: Warner Losh Message-ID: <20111219190524.GA14261@onelab2.iet.unipi.it> References: <20111219181410.GA13742@onelab2.iet.unipi.it> <19FB5614-B70A-4FDB-A7ED-73A7B5C5970C@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <19FB5614-B70A-4FDB-A7ED-73A7B5C5970C@bsdimp.com> User-Agent: Mutt/1.4.2.3i Cc: arch@FreeBSD.org Subject: Re: generic pci device_probe routine ? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 18:48:58 -0000 On Mon, Dec 19, 2011 at 11:35:47AM -0700, Warner Losh wrote: > > On Dec 19, 2011, at 11:14 AM, Luigi Rizzo wrote: > > haven't done device drivers for a while, but i just noticed that > > pretty much all PCI drivers have their own replica of the *_probe code > > which does the same exact thing -- define an array of vendor,product > > entries, and lookup the entry in the array. > > Would it make sense (or, do we have already) to have common struct > > and routine, similar to what we have in usb_lookup.c ? > > It would make sense. Model it after the PC Card one, however, since that one also include the size of the elements to allow for piggybacking data for the driver in the table. yes, i was thinking of that too. cheers luigi > It is the biggest stumbling block to allowing automated driver loading today. > > Warner > From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 19:36:24 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF6F9106566B; Mon, 19 Dec 2011 19:36:24 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4252A8FC0C; Mon, 19 Dec 2011 19:36:24 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id 6F2E69EE476; Mon, 19 Dec 2011 19:36:23 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324323383; bh=1QF4rBK711EZyM5OzxYp09GQQj9AMrVMdAM6ayIdAUE=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=vxTycPicFNep/rmcti0SNE3xN3r1TEH7youbL2gu8ZVl6CElz/qj0GZPSxMYP4F1A oYE8IDXjxsa/k00JLXMm8rKYsP2RyuBNk0xdqLrxnmS/38EUL8c2OqH9baE2T16RCs FAi3WdB88zKDm5Vx5o8w4JjuG3pHiTwx76N7X63c= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id E2F4C9EE473; Mon, 19 Dec 2011 19:36:21 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324323382; bh=1QF4rBK711EZyM5OzxYp09GQQj9AMrVMdAM6ayIdAUE=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=Ti3NoZobz6rTyqmTioRZQaxd7qRLZdxVEiVLXdxCyfmgnmaSdIgvvwlT1toKPgJdG Hf25zcHG/OHEspb56Q948dfogHaFnwjDA1uNbOznCSsrRMUasKKlKf3j1CEz8XzqLC Kq9LNp4Tyt7RXVbgiVLYBKk4lsOO1QIYKOwe2ruI= From: "Niall Douglas" To: threads@freebsd.org, arch@freebsd.org Date: Mon, 19 Dec 2011 19:36:21 -0000 MIME-Version: 1.0 Message-ID: <4EEF9235.31023.B2519C9A@s_sourceforge.nedprod.com> Priority: normal In-reply-to: <20111216214913.GA1771@hoeg.nl> References: <20111216214913.GA1771@hoeg.nl> X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 19:36:24 -0000 On 16 Dec 2011 at 22:49, Ed Schouten wrote: > In my opinion the ISO folks suffer a bit from the Not Invented Here > syndrome. In an earlier revision of the C1X specification, they even > described a `struct xtime', which had a purpose identical to `struct > timespec'. The same holds for the threading API. It can be 1:1 mapped to > a subset of pthread -- why not simply standardize that subset then? As someone who sits on said committees, I can tell you that the reason why was because at the beginning it was thought that the C1X threading API would diverge significantly from the POSIX API. Indeed, early drafts of the standard had quite a number of changes. However, just recently almost all of those changes have been excised due to pressures from the system vendors and the C++ committee who came in quite late on wanting feature parity between the two, and C++ had chosen a specific subset of POSIX rather than doing anything to try and fix its known problems. Obviously, had we known that from the beginning, things would have been done differently. However, there was - in hindsight - a lack of realisation just how expensive any significant changes would appear to vendors. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 19:36:29 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0AC3E106564A; Mon, 19 Dec 2011 19:36:29 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id B1AB98FC13; Mon, 19 Dec 2011 19:36:28 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id 0968A9EE476; Mon, 19 Dec 2011 19:36:28 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324323388; bh=kAA7hcQIbkEDdfBCWO/gz0tn+ryUvDyiHQL82jX1ZwE=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=sGN3zTM5wFrBsBycpiOOEU57fSC1yChY1WUMgnm0/uxoq+ThQpO5jUQf+aCUkBZCH y22yazHUejmqCgPs5dvvgUJp2ZLrPInKVutVcFxlCbLNMxJ3HbTn5L75dVJxFja/Kc WQdIo4PD/qp8MXUtuAe41aVYpt4f8K39lQP6njzg= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id 86D949EE473; Mon, 19 Dec 2011 19:36:23 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324323387; bh=kAA7hcQIbkEDdfBCWO/gz0tn+ryUvDyiHQL82jX1ZwE=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=AEPh5XOpRcE+712kIFxhqbcav6QC5oO0HhOV52BVc9N9CgvwHknERvNYgSzHGLkK3 olbRU7zhEISxPbSy8xPDXwiMPcdDJIbVocY8uo7zmLB5J1Aztku6TpinFZAykUJ5kx GUwNeGeBtgv4uym0Drkc9PbubqJ3zutIFN9oVMME= From: "Niall Douglas" To: arch@freebsd.org, threads@freebsd.org Date: Mon, 19 Dec 2011 19:36:21 -0000 MIME-Version: 1.0 Message-ID: <4EEF9235.252.B2519BEE@s_sourceforge.nedprod.com> Priority: normal In-reply-to: <86ty4y4rj5.fsf@ds4.des.no> References: <85477.1324155737@critter.freebsd.dk>, <86ty4y4rj5.fsf@ds4.des.no> X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: Quoted-printable Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 19:36:29 -0000 On 18 Dec 2011 at 2:06, Dag-Erling Sm=F8rgrav wrote: > "Poul-Henning Kamp" writes: > > Ohhh, but I know: Lets make a rival to the POSIX threads, we can do i= t > > much better and slightly incompatible, big market there I'm sure. > > That's not the point. The point is that C until now did not have a > concurrency model. The threading API in itself is not important; I'm > sure the committee knows perfectly well that nobody is going to use it. > What's important is that the standard now defines how C behaves in a > concurrent environment. Actually we don't go far enough at all in specifying how C behaves in a concurrent environment. The problem is that in the expected lifetime of the standard (10 years) there are a number of known big changes coming to commodity hardware e.g. memory transactions and non-SMP cache coherency. No one on the committee wants to accidentally break future hardware features, so we have left them unspecified. BTW we entirely expect the C1X threading API to supplant all others, including POSIX whose threading support will be mostly for backwards compatibility and may even become deprecated. A lot of resources and effort has been directed into getting the memory model right and future safe if not future proof. Indeed, the chances are that the next POSIX revision will copy C1X in some areas rather than the other way round. The complaints about API bloating are obvious. Bear in mind that while POSIX based platforms are in a majority, there are significant large implementors of C1X for whom a lot of new support code will have to be written e.g. MS Windows, whereas POSIX platforms have a much easier time of things in supporting C1X. Regarding implementing support for the C1X threading API, I'd suggest a different approach. Windows PE and Apple Mach-O both support symbol aliasing so you can declare an API to be identical to another API. Interestingly, ELF has no such feature. See http://blog.omega-prime.co.uk/?p=3D121. I would suggest the best and easiest way to implement much of the C1X library is to add symbol aliasing to ELF. Then all ELF binary based systems could support C1X features without having to resort to inline headers or thin wrappers. Linux will be needing this too, and it's the easiest way out. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 19:36:30 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 400B2106566C; Mon, 19 Dec 2011 19:36:30 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id EA21A8FC14; Mon, 19 Dec 2011 19:36:29 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id 5DF229EE476; Mon, 19 Dec 2011 19:36:29 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324323389; bh=NVPtnDWhDNEITJK+bcS6IRQnTY8syOaj+spP+Xoudx8=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=vWeRNEQ0Er2Hm5IARfynS2Bui6ZvxFFdHuXzfluOY/5lNrLEuJ+J1yv7cuu2FzGjq elziyDCIJdumvwAghskDTBwpKVECuQoAGOOn/L5PfdFDXM+f+bz3UxkfLFoKhUZVVz RKg6g25UnyK4UHEkK4lk812jo0BggudM43tZ2qiI= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id 7EF569EE473; Mon, 19 Dec 2011 19:36:28 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324323388; bh=NVPtnDWhDNEITJK+bcS6IRQnTY8syOaj+spP+Xoudx8=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=lBfkP+7poaxo+O3wkYbWEkX8pqqeTfTPazmMupvqSFBhJlNY8pMnshNvh1TZybi9D 4UNA/mzuvneMcAwCO98y5m+vdztii+pAkCwwdEimm5OVCnlokD0H7ZKtih8786vNCZ 2ZjCVMr5fpzhu6jPhH2AKghf6ohuExrRqafv+rr8= From: "Niall Douglas" To: arch@freebsd.org, threads@freebsd.org Date: Mon, 19 Dec 2011 19:36:21 -0000 MIME-Version: 1.0 Message-ID: <4EEF9235.24779.B2519D55@s_sourceforge.nedprod.com> Priority: normal In-reply-to: <20111218115326.GD50300@deviant.kiev.zoral.com.ua> References: <85477.1324155737@critter.freebsd.dk>, <86ty4y4rj5.fsf@ds4.des.no>, <20111218115326.GD50300@deviant.kiev.zoral.com.ua> X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 19:36:30 -0000 On 18 Dec 2011 at 13:53, Kostik Belousov wrote: > Well, the reverse was exactly _my_ point. I cannot find the description of > how the abstract C machine behaves, in the presence of multiple threads > of execution. The atomics chapter covers only some special operations, > which are added in the new revision. Indeed. It is deliberately unspecified. > E.g., there is absolutely no mention of the memory changes visibility, > or guarantees of atimicity of the assignments/reads etc. IMO, the threading > was slapped nearby, and the standard is not useful as-is. I am sorry if > I missed the parts. The memory model has been very closely thought through by some of the leading domain experts in the area. What the C1X standard refers to is per-CPU core, so if you do an acquire, acquire, acquire, release, acquire then you get a certain guaranteed minimal ordering of reads and writes on that particular CPU core as seen from other cores. Note I used the condition "minimal". The whole point of acquire/release is that the maximum degree of freedom to reorder is given to both the compiler and the CPU. Note that the CPU may ignore the ordering internally so long as externally the validity of the ordering is maintained. For example, if a cache line is exclusive to a core and no other core has it, the CPU may dispense with any ordering constraints within that cache line. This obviously has no bad consequences. Right now Intel x86/x64 has an extremely tight memory model - all loads acquire and all stores release unless you use special SSE/AVX opcodes to say otherwise. This means that code which works great on Intel can randomly fail on other processors. It's worth bearing this in mind. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 19 22:31:59 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 156911065675; Mon, 19 Dec 2011 22:31:59 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id C7C848FC15; Mon, 19 Dec 2011 22:31:58 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id pBJMVvED021175; Mon, 19 Dec 2011 17:31:57 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.6 (mail.netplex.net [204.213.176.10]); Mon, 19 Dec 2011 17:31:57 -0500 (EST) Date: Mon, 19 Dec 2011 17:31:57 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Niall Douglas In-Reply-To: <4EEF9235.31023.B2519C9A@s_sourceforge.nedprod.com> Message-ID: References: <20111216214913.GA1771@hoeg.nl> <4EEF9235.31023.B2519C9A@s_sourceforge.nedprod.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 22:31:59 -0000 On Mon, 19 Dec 2011, Niall Douglas wrote: > On 16 Dec 2011 at 22:49, Ed Schouten wrote: > >> In my opinion the ISO folks suffer a bit from the Not Invented Here >> syndrome. In an earlier revision of the C1X specification, they even >> described a `struct xtime', which had a purpose identical to `struct >> timespec'. The same holds for the threading API. It can be 1:1 mapped to >> a subset of pthread -- why not simply standardize that subset then? > > As someone who sits on said committees, I can tell you that the > reason why was because at the beginning it was thought that the C1X > threading API would diverge significantly from the POSIX API. Indeed, > early drafts of the standard had quite a number of changes. However, > just recently almost all of those changes have been excised due to > pressures from the system vendors and the C++ committee who came in > quite late on wanting feature parity between the two, and C++ had > chosen a specific subset of POSIX rather than doing anything to try > and fix its known problems. > > Obviously, had we known that from the beginning, things would have > been done differently. However, there was - in hindsight - a lack of > realisation just how expensive any significant changes would appear > to vendors. And why on earth would the thought of having a threading API significantly different from the POSIX API even be on the table? It boggles the mind. -- DE From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 09:48:13 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47F381065676; Tue, 20 Dec 2011 09:48:13 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id B494E8FC0C; Tue, 20 Dec 2011 09:48:12 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id DAE659EE476; Tue, 20 Dec 2011 09:48:11 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324374491; bh=Rs69jaibQ1OZrqifUeiHb8XS3MKtN3hE9gJpzqpcTAs=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=cqlLV1ms0cUCUb7Z55qjpCBgLX49PjznSPpehdzZRZM1IF8aiCfBsUjvJ5d1zn1bF 2x66nnKdCFnDYFFXq7b/T4+dm+aPutTrlSNHP/6gvAmOuC8P5Fc+GaQXmmwUQjv2/7 bnSUYEIMv+PXpXf07UIOUpISvMr+ltD2hPRyYU+8= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id D1BB79EE241; Tue, 20 Dec 2011 09:48:10 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324374491; bh=Rs69jaibQ1OZrqifUeiHb8XS3MKtN3hE9gJpzqpcTAs=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=cqlLV1ms0cUCUb7Z55qjpCBgLX49PjznSPpehdzZRZM1IF8aiCfBsUjvJ5d1zn1bF 2x66nnKdCFnDYFFXq7b/T4+dm+aPutTrlSNHP/6gvAmOuC8P5Fc+GaQXmmwUQjv2/7 bnSUYEIMv+PXpXf07UIOUpISvMr+ltD2hPRyYU+8= From: "Niall Douglas" To: threads@freebsd.org, arch@freebsd.org Date: Tue, 20 Dec 2011 09:48:12 -0000 MIME-Version: 1.0 Message-ID: <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com> Priority: normal In-reply-to: References: <20111216214913.GA1771@hoeg.nl>, <4EEF9235.31023.B2519C9A@s_sourceforge.nedprod.com>, X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 09:48:13 -0000 On 19 Dec 2011 at 17:31, Daniel Eischen wrote: > > Obviously, had we known that from the beginning, things would have > > been done differently. However, there was - in hindsight - a lack of > > realisation just how expensive any significant changes would appear > > to vendors. > > And why on earth would the thought of having a threading API > significantly different from the POSIX API even be on the > table? It boggles the mind. 1. Because POSIX threads is known to have weaknesses in its design e.g. signal handling. These BTW have been fixed in C1X and there are some significant departures from the POSIX API wherever pthreads was just plain broke. Some of these departures may - on some platforms - require rejigging the internal kernel/userspace boundary. Before you ask, no I don't know what these are, it isn't my area of expertise. 2. Because pthreads were designed a long time ago in a world where threading was simpler and likely core count was lower. There are more use models now, and originally it was thought that incorporating some elements of newer threading models would be wise. Bear in mind that C1X - unlike POSIX - has to incorporate EVERY kind of computing system thinkable. C1X needs to run - without major baggage - on everything from tiny, OSless systems right up to thousand CPU core highly proprietary systems. And it must do this while staying as backward compatible with legacy systems as possible. 3. Because pthreads is not the only major threading implementation out there. The NT/OS/2 model is hugely popular if fundamentally anti-scalable in design and pthreads don't map exactly one to one with them. For example, thread cancellation is completely missing from C1X as it would require significant kernel rewriting on NT, so it got chopped completely. It had been hoped we could come up with something which worked around the lack of thread cancellation, but we didn't make it in time. It'll be TR1 before we nail the corner cases. Right now, there are quite a few places where under the C1X API the program will just hang if things go wrong and there is no legal way out at all. That is unfortunate, but as always it's a question of resourcing and time. Most people's employers don't see how investing in standards work improves their bottom line, so a lot of the grunt work is goodwill and hobby time. 4. Because POSIX does evolve over time - indeed, its next release is same year as C1X (i.e. next year). People sit on both ISO committees and are on the Austin Working Group. There is significant cross-pollination. The changes in C1X are highly likely to become normalised in the next iteration of POSIX. So think of this way, the departures from POSIX in C1X were mostly intended as departures by POSIX from POSIX next iteration anyway. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 10:09:25 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E3971065672; Tue, 20 Dec 2011 10:09:25 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 2D02B8FC19; Tue, 20 Dec 2011 10:09:24 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 08D2A5DB3; Tue, 20 Dec 2011 10:09:23 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBKA9N3A003066; Tue, 20 Dec 2011 10:09:23 GMT (envelope-from phk@phk.freebsd.dk) To: "Niall Douglas" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 20 Dec 2011 09:48:12 GMT." <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com> Content-Type: text/plain; charset=ISO-8859-1 Date: Tue, 20 Dec 2011 10:09:23 +0000 Message-ID: <3065.1324375763@critter.freebsd.dk> Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 10:09:25 -0000 In message <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com>, "Niall Douglas" writes: >On 19 Dec 2011 at 17:31, Daniel Eischen wrote: > >> > Obviously, had we known that from the beginning, things would have >> > been done differently. However, there was - in hindsight - a lack of >> > realisation just how expensive any significant changes would appear >> > to vendors. >> >> And why on earth would the thought of having a threading API >> significantly different from the POSIX API even be on the >> table? It boggles the mind. > >1. Because [...] Nice and fine. But can you explain, why the job is done so half-assedly ? Why are fundamentally and necessary programming tools, such as a "assert this mutex is held" missing ? Why are timescale-issues not dealt with ? For instance mtx_timedlock() operates only on the UTC scale. Where is the option to wait on an "elapsed time" timescale to not be hosed by ntpd(8) or root's setting the clock backwards during system startup ? Where did the ability to control a threads stacksize or other attributes go in thrd_create() ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 12:50:49 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3C7D106564A; Tue, 20 Dec 2011 12:50:49 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 2F5018FC0A; Tue, 20 Dec 2011 12:50:48 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id 219519EE70A; Tue, 20 Dec 2011 12:50:48 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324385448; bh=kGB5UvMmPwKneA7dbch3S3ogl6k5qac4BNFrWGHI4G0=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=kKRQ/xbk2XuqkZukKESlKuJvK+gCmf1PkZJ3ZVJ1SyiQK2cM2PF5WWQCW3sae+ypo I2/8at4y4WAZqgdFs0FdRBWJLVEKfD+jsf6yHGEpQtQD/IJiqqrh9Q+PT5dADGysq+ 4qMty26vHXwT2sOyNpoujQKLTeGPPDniUFyQwaB4= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id CB61D9EE49B; Tue, 20 Dec 2011 12:50:46 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324385447; bh=kGB5UvMmPwKneA7dbch3S3ogl6k5qac4BNFrWGHI4G0=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=YfFYTsZdxjAw+WOuoxvdlCvmUgyvAmqM9l5QiwN6xyeZhwXBYnXyYXCX4EkedLanO a4fYXcQz0zEX6x4haBHeWjX23bHM6c9Lq4aYNAuFH8GCoa1Mp4pENdMQqvr4jl2T0b qmN53VD6EfCgC4V2uzkSHhl2FepL+ZQaZ2I+0ZM8= From: "Niall Douglas" To: threads@freebsd.org, arch@freebsd.org Date: Tue, 20 Dec 2011 12:50:48 -0000 MIME-Version: 1.0 Message-ID: <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> Priority: normal In-reply-to: <3065.1324375763@critter.freebsd.dk> References: >, <3065.1324375763@critter.freebsd.dk> X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 12:50:50 -0000 On 20 Dec 2011 at 10:09, Poul-Henning Kamp wrote: > >1. Because [...] > > Nice and fine. > > But can you explain, why the job is done so half-assedly ? The job was NOT done half-arsed. If you had any experience of sitting on these committees you would know how much dedication and effort is put into standards, especially JTC1 SC22 subcommittees. Every single API in there has been studied and pored over at length across multiple years. Everything is the way it is for a good reason. If it doesn't make sense to you that's most likely because you're not half as experienced or clever as you think you are. If you really want to know why something is the way it is, all discussion regarding all points is documented in full. > Why are fundamentally and necessary programming tools, such as a > "assert this mutex is held" missing ? I think that was rejected due to concerns about introducing race conditions if I remember correctly. You'll generally find there is no easy way of checking the state of anything threading related for the same reason. > Why are timescale-issues not dealt with ? > > For instance mtx_timedlock() operates only on the UTC scale. Where > is the option to wait on an "elapsed time" timescale to not be hosed > by ntpd(8) or root's setting the clock backwards during system > startup ? If I remember correctly UTC was seen as the safest of all options available. Annoying to program I agree, but definitely safer than alternatives. There was a long and heated discussion about this over many months, but the committee decided on UTC as the least worst choice. mtx_timedlock() only *tries* to wait for a period up to the time specified. It may return any time before then and indeed if the system clock were changed I would expect it to early out with a thrd_error. None of the timed functions in C11 should ever be used outside a loop which tests and rewaits if necessary. I agree this isn't clear from the spec, but the spec is not a programming manual. Failure to wrap all these calls in double checking loops is bad code which won't be reliable. > Where did the ability to control a threads stacksize or other > attributes go in thrd_create() ? I would assume that they were considered non portable due to vendor objection. In particular, I remember an argument that thread stacksize settings are dangerous and must be omitted. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 13:05:00 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89588106564A; Tue, 20 Dec 2011 13:05:00 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 411A28FC13; Tue, 20 Dec 2011 13:04:59 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id pBKD4wbN046764; Tue, 20 Dec 2011 08:04:58 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.6 (mail.netplex.net [204.213.176.10]); Tue, 20 Dec 2011 08:04:59 -0500 (EST) Date: Tue, 20 Dec 2011 08:04:58 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Niall Douglas In-Reply-To: <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com> Message-ID: References: <20111216214913.GA1771@hoeg.nl>, <4EEF9235.31023.B2519C9A@s_sourceforge.nedprod.com>, <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 13:05:00 -0000 On Tue, 20 Dec 2011, Niall Douglas wrote: > 4. Because POSIX does evolve over time - indeed, its next release is > same year as C1X (i.e. next year). People sit on both ISO committees > and are on the Austin Working Group. There is significant > cross-pollination. The changes in C1X are highly likely to become > normalised in the next iteration of POSIX. So think of this way, the > departures from POSIX in C1X were mostly intended as departures by > POSIX from POSIX next iteration anyway. Think what you want, but monitoring the austin mailing list, it seemed to catch everyone by surprise that C1X was coming up with a threading interface that diverged from POSIX. At least a couple of years ago that was the case, but perhaps that prompted the cross-pollination. -- DE From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 13:40:03 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD7A7106566B; Tue, 20 Dec 2011 13:40:03 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 333008FC08; Tue, 20 Dec 2011 13:40:02 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 6816B5DA8; Tue, 20 Dec 2011 13:40:01 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBKDe0FX062148; Tue, 20 Dec 2011 13:40:00 GMT (envelope-from phk@phk.freebsd.dk) To: "Niall Douglas" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 20 Dec 2011 12:50:48 GMT." <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> Content-Type: text/plain; charset=ISO-8859-1 Date: Tue, 20 Dec 2011 13:40:00 +0000 Message-ID: <62147.1324388400@critter.freebsd.dk> Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 13:40:04 -0000 In message <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com>, "Niall Douglas" writes: >> Why are fundamentally and necessary programming tools, such as a >> "assert this mutex is held" missing ? > >I think that was rejected due to concerns about introducing race >conditions if I remember correctly. You'll generally find there is no >easy way of checking the state of anything threading related for the >same reason. BS: Show me a working implementation of a mutex where you cannot tell if the current thread holds the mutex when it is locked ? (Hint: add a variable to the mutex, protected by the mutex, storing the thread-id if the locking thread ? Nudge, nudge, wink, wink ?) >> For instance mtx_timedlock() operates only on the UTC scale. Where >> is the option to wait on an "elapsed time" timescale to not be hosed >> by ntpd(8) or root's setting the clock backwards during system >> startup ? > >If I remember correctly UTC was seen as the safest of all options >available. Annoying to program I agree, but definitely safer than >alternatives. No, actually UTC is much unsafer than the alternative, and in general much less useful and desirable for the same reasons it is unsafe. UTC as implemented on a computer is not a continuous timescale, it is not even an monotonic timescale if you are unlucky. The far too typical scenario is that you boot your system and then some minutes later NTPD will step your clock forwards if you're lucky, and backwards by a day if you're not. >mtx_timedlock() only *tries* to wait for a period up to the time >specified. It may return any time before then and indeed if the >system clock were changed I would expect it to early out with a >thrd_error. And I would expect that most implementations will not even think about that cornercase, and just do stupid things if the UTC time is stepped backwards. (Case in point: The Oracle databases freezing over the leap-second were caused by code that tried to work around broken standards in this particular space) But if this was a well-thought-out standard, instead of a half-assed job, it wouldn't matter what you and I expect: Then the standard would have said what to expect. >None of the timed functions in C11 should ever be used outside a loop >which tests and rewaits if necessary. A loop doesn't help you if the UTC time is stepped back 1 hour and your sleep was supposed to be only one second. > I agree this isn't clear from >the spec, but the spec is not a programming manual. Which only goes to show that the great stride in education which UNIX brought, has been missed widely by the ISO-C crew. How do you expect actual real-life programmers to know how to use this brittle API's, if you don't even give them an example to look at, or write about their major caveats in the spec ? And maybe, in trying to express that using a real-world example, the standards comittee would realize that UTC was a mistake, and changed the timeout argument to a relative time interval instead, like for instance the poll(2) system-call. >> Where did the ability to control a threads stacksize or other >> attributes go in thrd_create() ? > >I would assume that they were considered non portable due to vendor >objection. In particular, I remember an argument that thread >stacksize settings are dangerous and must be omitted. I would assume that the people who found it dangerous were morons without any actual real-life experience programming threads on computers with finite resources ? The way POSIX did that is far from a piece of beauty, but at least it provides a way to communicate such attributes. You are welcome to think that everybody in the ISO-C group is better and smarter than me. If they are, this standard doesn't show it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 13:52:25 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25B231065673; Tue, 20 Dec 2011 13:52:25 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D71958FC16; Tue, 20 Dec 2011 13:52:24 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 7394D46B06; Tue, 20 Dec 2011 08:52:24 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id F23C1B93E; Tue, 20 Dec 2011 08:52:23 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 20 Dec 2011 08:22:25 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <3065.1324375763@critter.freebsd.dk> <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> In-Reply-To: <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112200822.26369.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 20 Dec 2011 08:52:24 -0500 (EST) Cc: Niall Douglas , freebsd-threads@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 13:52:25 -0000 On Tuesday, December 20, 2011 7:50:48 am Niall Douglas wrote: > On 20 Dec 2011 at 10:09, Poul-Henning Kamp wrote: > > > >1. Because [...] > > > > Nice and fine. > > > > But can you explain, why the job is done so half-assedly ? > > The job was NOT done half-arsed. If you had any experience of sitting > on these committees you would know how much dedication and effort is > put into standards, especially JTC1 SC22 subcommittees. Every single > API in there has been studied and pored over at length across > multiple years. > > Everything is the way it is for a good reason. If it doesn't make > sense to you that's most likely because you're not half as > experienced or clever as you think you are. If you really want to > know why something is the way it is, all discussion regarding all > points is documented in full. Mmmm, you might be well served to check up on the experience of some of the folks you are conversing with. Otherwise you risk reducing your credibility in the present forum. Also, to argue that "everything" in a standard is perfect is, eh, a bit of a stretch. There's a reason for the connotation associated with the phrase "design by committee". > > Why are fundamentally and necessary programming tools, such as a > > "assert this mutex is held" missing ? > > I think that was rejected due to concerns about introducing race > conditions if I remember correctly. You'll generally find there is no > easy way of checking the state of anything threading related for the > same reason. Err, no, there should be no races here. You cannot easily assert that a mutex is held by some other thread; however, if you use a thread-unique identifier as your lock cookie, then you can check to see if the current thread owns a mutex without any races (and this is a _very_ useful debugging technique in real-world applications). The reason I can think of why you might not specify this is if you want to support machines that have very limited support for atomic operations (e.g. only an exchange instruction or a single-bit test-and- set as opposed to a full-world test-and-set such as cmpxchg on x86 or cas on sparc). For those machines, you may be reduced to having a lock cookie of just 0 or 1 and you cannot distinguish the case of the current thread holding the lock versus another thread holding the lock. (It is hard to safely assert read locks for reader/writer locks for much the same reason as common lock- cookie encodings use a simple count for read locks.) However, that is due to a limitation of your abstract machine's atomic ops, not a race. > > Where did the ability to control a threads stacksize or other > > attributes go in thrd_create() ? > > I would assume that they were considered non portable due to vendor > objection. In particular, I remember an argument that thread > stacksize settings are dangerous and must be omitted. I guess you don't want any popular software (e.g. Python) to actually use your API then. :) Sadly, there is some code that wants to create a bazillion threads in a given process and there is other code that wants to create a few threads but use deep call stacks and/or put some large objects on the stack of those threads. There is not a single magical stack size that works equally well for both. I agree that it is very MD and hackish to let an application specify the size directly, but unfortunately the functionality is necessary. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 14:02:23 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50035106566B for ; Tue, 20 Dec 2011 14:02:23 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 038FC8FC18 for ; Tue, 20 Dec 2011 14:02:22 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id DE65D5E50; Tue, 20 Dec 2011 14:02:21 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBKE2LVd073234; Tue, 20 Dec 2011 14:02:21 GMT (envelope-from phk@phk.freebsd.dk) To: John Baldwin From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 20 Dec 2011 08:22:25 EST." <201112200822.26369.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Date: Tue, 20 Dec 2011 14:02:21 +0000 Message-ID: <73233.1324389741@critter.freebsd.dk> Cc: freebsd-threads@freebsd.org, Niall Douglas , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 14:02:23 -0000 In message <201112200822.26369.jhb@freebsd.org>, John Baldwin writes: >The reason I can think of why you might not specify >this is if you want to support machines that have very limited support for >atomic operations (e.g. only an exchange instruction or a single-bit test-and- >set as opposed to a full-world test-and-set such as cmpxchg on x86 or cas on >sparc). There is no way this can be impossible on a platform which can implement a mutex in the first place: mtx_lock(l) { atomic_magic_lock(l->lock_field) l->id = thread_id; } mtx_unlock(l) { assert(l->id == thread_id); l->id = NULL; atomic_magic_unlock(l->lock_field) } mtx_assert_held(l) { assert(l->lock-field != 0); assert(l->id == thread_id); } -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 14:34:56 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A478B1065672; Tue, 20 Dec 2011 14:34:56 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 5A27A8FC17; Tue, 20 Dec 2011 14:34:56 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id BC6D388C001; Tue, 20 Dec 2011 14:34:55 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324391695; bh=CL9mgfJFNZTtTokYarUKRsTn+BkmTtdMN3Bcp6kOBbU=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=xUc/Mi+7nsWCgP/CYpeXhvyZZgXobNo6eegNf/qPmV0NMdCpGa0+mdahPknEAlT8N oOThfOU1EbFWpFHzjBbFsZsjoUfcMpX3BrnkScwc9ws6uf/WywXIurd67JPvLro8MW TQy55CheWSvmkdXe45Mx7Ea7GjJhOyo3JN7i37zw= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id 82D129EE70A; Tue, 20 Dec 2011 14:34:54 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324391695; bh=CL9mgfJFNZTtTokYarUKRsTn+BkmTtdMN3Bcp6kOBbU=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=xUc/Mi+7nsWCgP/CYpeXhvyZZgXobNo6eegNf/qPmV0NMdCpGa0+mdahPknEAlT8N oOThfOU1EbFWpFHzjBbFsZsjoUfcMpX3BrnkScwc9ws6uf/WywXIurd67JPvLro8MW TQy55CheWSvmkdXe45Mx7Ea7GjJhOyo3JN7i37zw= From: "Niall Douglas" To: threads@freebsd.org, arch@freebsd.org Date: Tue, 20 Dec 2011 14:34:54 -0000 MIME-Version: 1.0 Message-ID: <4EF09D0E.10213.B663FC80@s_sourceforge.nedprod.com> Priority: normal In-reply-to: References: <20111216214913.GA1771@hoeg.nl>, <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com>, X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 14:34:56 -0000 On 20 Dec 2011 at 8:04, Daniel Eischen wrote: > > 4. Because POSIX does evolve over time - indeed, its next release is > > same year as C1X (i.e. next year). People sit on both ISO committees > > and are on the Austin Working Group. There is significant > > cross-pollination. The changes in C1X are highly likely to become > > normalised in the next iteration of POSIX. So think of this way, the > > departures from POSIX in C1X were mostly intended as departures by > > POSIX from POSIX next iteration anyway. > > Think what you want, but monitoring the austin mailing list, > it seemed to catch everyone by surprise that C1X was coming > up with a threading interface that diverged from POSIX. > At least a couple of years ago that was the case, but > perhaps that prompted the cross-pollination. You're absolutely correct - it was exactly this divergence which brought a lot more eyeballs to the C11 draft. And indeed it was about two, two and half years ago now. It's actually amazing how fast time has gone. As with any ISO standard, right at the beginning of a draft there's only a few people working on something. You can get some really radical and/or badly thought through stuff coming in at that stage. As the release date nears, more eyeballs come on board and stuff gets much more conservative. Anything unnecessary, especially given vendor objections to doing anything more than necessary, tends to get excised or in particular in C11 made into an optional module. There's a LOT of optional stuff in C11. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 14:38:36 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9799F106566B; Tue, 20 Dec 2011 14:38:36 +0000 (UTC) (envelope-from BATV+4d3c52616b37ff3928b6+3040+infradead.org+hch@bombadil.srs.infradead.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2001:4830:2446:ff00:4687:fcff:fea6:5117]) by mx1.freebsd.org (Postfix) with ESMTP id 42AB58FC0A; Tue, 20 Dec 2011 14:38:36 +0000 (UTC) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1Rd0pu-0000pb-3n; Tue, 20 Dec 2011 14:38:34 +0000 Date: Tue, 20 Dec 2011 09:38:34 -0500 From: Christoph Hellwig To: Niall Douglas Message-ID: <20111220143833.GA31812@infradead.org> References: <20111216214913.GA1771@hoeg.nl> <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com> <4EF09D0E.10213.B663FC80@s_sourceforge.nedprod.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EF09D0E.10213.B663FC80@s_sourceforge.nedprod.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 14:38:36 -0000 On Tue, Dec 20, 2011 at 02:34:54PM -0000, Niall Douglas wrote: > excised or in particular in C11 made into an optional module. There's > a LOT of optional stuff in C11. Which is a very big warning light for a bad standard. Hopefull it's not as bad as the T10 standards. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 14:47:35 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C257106564A; Tue, 20 Dec 2011 14:47:35 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 175EE8FC13; Tue, 20 Dec 2011 14:47:34 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id 641329EE475; Tue, 20 Dec 2011 14:47:33 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324392453; bh=zQ1NjD+skNI/Zx+L69r6YMNznMTLTcZfBobYcklKFJA=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=VQN31HZ4nzeL9ulbV5gZILFeEm+6OzogeOk7R0voF7xKBI2Du2Enp8+QbiVKnG7c9 zmNE3E4muMdGZzng6K3f+UB37fVyufQqpcjUm5MZ1EW17jLYLyH6TKMeju8UgIaXcf AJk99SYsuyc5T9x2EDTVovlTRvPcjdhvUHF7FN2A= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id 67ABC9EE241; Tue, 20 Dec 2011 14:47:24 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324392444; bh=zQ1NjD+skNI/Zx+L69r6YMNznMTLTcZfBobYcklKFJA=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=gVD0lT/h24dsS7RuLQB9jGh4Q2SwoLIeF95sIe3iE+zUZvGuns7lHnXH709GDLr8p 9j3bElOc4V3SYalrRtIRIwp14PmibS/fvGFzaRHpV2wVPis6rPam59G2spHXaFaH0j t5sStiUMfle5MmmcU+6M/fs5i0akmb3Fs7t3sTu0= From: "Niall Douglas" To: threads@freebsd.org, arch@freebsd.org Date: Tue, 20 Dec 2011 14:47:25 -0000 MIME-Version: 1.0 Message-ID: <4EF09FFD.7768.B66F73ED@s_sourceforge.nedprod.com> Priority: normal In-reply-to: <62147.1324388400@critter.freebsd.dk> References: >, <62147.1324388400@critter.freebsd.dk> X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 14:47:35 -0000 On 20 Dec 2011 at 13:40, Poul-Henning Kamp wrote: > >If I remember correctly UTC was seen as the safest of all options > >available. Annoying to program I agree, but definitely safer than > >alternatives. > > No, actually UTC is much unsafer than the alternative, and in general > much less useful and desirable for the same reasons it is unsafe. > > UTC as implemented on a computer is not a continuous timescale, > it is not even an monotonic timescale if you are unlucky. Sure, it even varies slightly between CPU core on some platforms. > And maybe, in trying to express that using a real-world example, > the standards comittee would realize that UTC was a mistake, and > changed the timeout argument to a relative time interval instead, > like for instance the poll(2) system-call. There was some very good argument against relative periods. I honestly can't remember what that was. It was a long time ago. > >> Where did the ability to control a threads stacksize or other > >> attributes go in thrd_create() ? > > > >I would assume that they were considered non portable due to vendor > >objection. In particular, I remember an argument that thread > >stacksize settings are dangerous and must be omitted. > > I would assume that the people who found it dangerous were morons > without any actual real-life experience programming threads on > computers with finite resources ? I think you are out of order in this public comment and you should apologise to those who have served on WG14. If you disagree with the standard, please feel free to submit an erratum to http://open-std.org/jtc1/sc22/wg14/ Or even better, participate and donate your own time to the committee. They are very inclusive and more than happy to give time to all viewpoints. You don't even need to officially join - you can attend or participate as an observer. Otherwise quite frankly I don't care what your background, your rep or your experience is. Feel free to voice an opinion after you have attended a few ISO committee meetings and seen the work done there. Otherwise you don't know what you're talking about. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 14:52:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E4F01065678; Tue, 20 Dec 2011 14:52:51 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 25B6B8FC17; Tue, 20 Dec 2011 14:52:51 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id CBE2A46B32; Tue, 20 Dec 2011 09:52:50 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5D0E6B955; Tue, 20 Dec 2011 09:52:50 -0500 (EST) From: John Baldwin To: "Poul-Henning Kamp" Date: Tue, 20 Dec 2011 09:43:18 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <73233.1324389741@critter.freebsd.dk> In-Reply-To: <73233.1324389741@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112200943.18812.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 20 Dec 2011 09:52:50 -0500 (EST) Cc: freebsd-threads@freebsd.org, Niall Douglas , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 14:52:51 -0000 On Tuesday, December 20, 2011 9:02:21 am Poul-Henning Kamp wrote: > In message <201112200822.26369.jhb@freebsd.org>, John Baldwin writes: > > >The reason I can think of why you might not specify > >this is if you want to support machines that have very limited support for > >atomic operations (e.g. only an exchange instruction or a single-bit test-and- > >set as opposed to a full-world test-and-set such as cmpxchg on x86 or cas on > >sparc). > > There is no way this can be impossible on a platform which can > implement a mutex in the first place: > > > mtx_lock(l) > { > atomic_magic_lock(l->lock_field) > l->id = thread_id; > } > > mtx_unlock(l) > { > assert(l->id == thread_id); > l->id = NULL; > atomic_magic_unlock(l->lock_field) > } > > mtx_assert_held(l) > { > assert(l->lock-field != 0); > assert(l->id == thread_id); > } Yep, having a helper field to track the owner would work fine on such degenerate platforms. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 14:58:24 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A2C41065673; Tue, 20 Dec 2011 14:58:24 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 468D28FC1C; Tue, 20 Dec 2011 14:58:23 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 9C43F5C34; Tue, 20 Dec 2011 14:58:22 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBKEwM4S078108; Tue, 20 Dec 2011 14:58:22 GMT (envelope-from phk@phk.freebsd.dk) To: "Niall Douglas" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 20 Dec 2011 14:34:54 GMT." <4EF09D0E.10213.B663FC80@s_sourceforge.nedprod.com> Content-Type: text/plain; charset=ISO-8859-1 Date: Tue, 20 Dec 2011 14:58:22 +0000 Message-ID: <78107.1324393102@critter.freebsd.dk> Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 14:58:24 -0000 In message <4EF09D0E.10213.B663FC80@s_sourceforge.nedprod.com>, "Niall Douglas" writes: >There's a LOT of optional stuff in C11. And then there are non-optional stuff like , specified so it leaves no doubt, except as to the sanity of the authors. As best I can gather, at one point ISO C decided to use '_' Uppercase Lowercase+digit+'_'* for keywords, and that begat us '_Complex', '_Alignas' (a famous greek poet ?), '_Generic'', and also '_Noreturn'. Then somebody probably pointed out that inline _Noreturn foo(int bar); just looked plain ugly. Then some smart person came up with which shall contain, exactly and only: #define noreturn _Noreturn So that we can write: #include inline noreturn foo(int bar); Now, explain to me, why this was more important, and got included, than being able to say: struct foo_protocol_header { little_endian uint16_t proto_ver @ 0, little_endian uint32_t seq_no @ 4, little_endian uint32_t ack_no @ 10, little_endian uint8_t proto_req @ 16, }; And have the compiler do the error-prone idiot-work for us, no matter which platform or compiler we use ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 15:14:25 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18A9E106566B; Tue, 20 Dec 2011 15:14:25 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 2C35B8FC0A; Tue, 20 Dec 2011 15:14:24 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 2A2DF5DAA; Tue, 20 Dec 2011 15:14:23 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBKFEMcq078247; Tue, 20 Dec 2011 15:14:22 GMT (envelope-from phk@phk.freebsd.dk) To: "Niall Douglas" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 20 Dec 2011 14:47:25 GMT." <4EF09FFD.7768.B66F73ED@s_sourceforge.nedprod.com> Content-Type: text/plain; charset=ISO-8859-1 Date: Tue, 20 Dec 2011 15:14:22 +0000 Message-ID: <78246.1324394062@critter.freebsd.dk> Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 15:14:25 -0000 In message <4EF09FFD.7768.B66F73ED@s_sourceforge.nedprod.com>, "Niall Douglas" writes: >> And maybe, in trying to express that using a real-world example, >> the standards comittee would realize that UTC was a mistake, and >> changed the timeout argument to a relative time interval instead, >> like for instance the poll(2) system-call. > >There was some very good argument against relative periods. I >honestly can't remember what that was. It was a long time ago. There are no good arguments against relative periods, in particular not when the crap they are replaced with requires you to put a loop around the sleep to get the desired behaviour in the first place. The fact that you cant even remeber the argument doesn't in any way make it any more convincing. >> I would assume that the people who found it dangerous were morons >> without any actual real-life experience programming threads on >> computers with finite resources ? > >I think you are out of order in this public comment and you should >apologise to those who have served on WG14. Fuck if will apologize! I will even repeat and clarify my charge to make sure that it is not misunderstood: WG14 are a pile of morons, who have been steadily ruining the C language with utter and useless crap, while only solving an absolute minority of the actual problems and not in any meaningful way developed the language during their time of custody. >Otherwise quite frankly I don't care what your background, your rep >or your experience is. Feel free to voice an opinion after you have >attended a few ISO committee meetings and seen the work done there. >Otherwise you don't know what you're talking about. I run a one man company which does not allow me to fly around the world to sit in meetings with morons who clearly havn't done any serious programming in far too long time. If I could afford it, and did it, I would undoubtedly in a mere matter of months, be a just as useless moron the current crew. And I know exactly what I'm talking about: A programming language I have been using day in and day out for 28 years. I may not now very much about endless meetings churning useless specifications, that have no relationship with what programmers of real-world software actually need from their programming language. It is not a lack of knowledge I see as a problem. WG14 can go to hell, and the C language would be better of if they did. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 20 22:28:38 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 30383106566C; Tue, 20 Dec 2011 22:28:38 +0000 (UTC) (envelope-from lev@serebryakov.spb.ru) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id C005E8FC12; Tue, 20 Dec 2011 22:28:37 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:2831:a229:70d2:ba0b]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id D225B4AC1C; Wed, 21 Dec 2011 02:28:35 +0400 (MSK) Date: Wed, 21 Dec 2011 02:28:32 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <549165194.20111221022832@serebryakov.spb.ru> To: "Niall Douglas" In-Reply-To: <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> References: >, <3065.1324375763@critter.freebsd.dk> <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2011 22:28:38 -0000 Hello, Niall. You wrote 20 =E4=E5=EA=E0=E1=F0=FF 2011 =E3., 16:50:48: > I would assume that they were considered non portable due to vendor > objection. In particular, I remember an argument that thread=20 > stacksize settings are dangerous and must be omitted. Ouch. So, same stack sizes for programs with 10 and 10'000 threads (on one platform)?! OMG. It is completely unusable, according to my experience with massively-parallel programs (think: programs which could completely load SunFire 15K, 144-way, monster with I/O bound load without any significant lock contention!). --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 14:59:07 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B2F41065675; Wed, 21 Dec 2011 14:59:07 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 77F6E8FC1B; Wed, 21 Dec 2011 14:59:05 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 649D86A00; Wed, 21 Dec 2011 14:59:00 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id A104E80C1; Wed, 21 Dec 2011 15:58:57 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: "Poul-Henning Kamp" References: <73233.1324389741@critter.freebsd.dk> Date: Wed, 21 Dec 2011 15:58:57 +0100 In-Reply-To: <73233.1324389741@critter.freebsd.dk> (Poul-Henning Kamp's message of "Tue, 20 Dec 2011 14:02:21 +0000") Message-ID: <86hb0ut1hq.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org, Niall Douglas , freebsd-threads@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 14:59:07 -0000 "Poul-Henning Kamp" writes: > There is no way this can be impossible on a platform which can > implement a mutex in the first place: > > > mtx_lock(l) > { > atomic_magic_lock(l->lock_field) > l->id =3D thread_id; > } OK > mtx_unlock(l) > { > assert(l->id =3D=3D thread_id); > l->id =3D NULL; > atomic_magic_unlock(l->lock_field) > } susceptible to race conditions > mtx_assert_held(l) > { > assert(l->lock-field !=3D 0); > assert(l->id =3D=3D thread_id); > } susceptible to race conditions The canonical solution is to use some low-level lock primitive (worst case, a critical section) to protect the mutex structure, but then you need some way for mtx_lock() to sleep if the mutex is held and some way for mtx_unlock() to wake sleepers. You also need to lock the mutex structure in mtx_assert_held(), unless l->id is atomic. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 15:05:25 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12122106566B; Wed, 21 Dec 2011 15:05:25 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id C90798FC19; Wed, 21 Dec 2011 15:05:24 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 51C8D6A0B; Wed, 21 Dec 2011 15:05:20 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id D6F1580C6; Wed, 21 Dec 2011 16:05:19 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: "Niall Douglas" References: <85477.1324155737@critter.freebsd.dk> <86ty4y4rj5.fsf@ds4.des.no> <4EEF9235.252.B2519BEE@s_sourceforge.nedprod.com> Date: Wed, 21 Dec 2011 16:05:19 +0100 In-Reply-To: <4EEF9235.252.B2519BEE@s_sourceforge.nedprod.com> (Niall Douglas's message of "Mon, 19 Dec 2011 19:36:21 -0000") Message-ID: <86d3bit174.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, threads@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 15:05:25 -0000 "Niall Douglas" writes: > BTW we entirely expect the C1X threading API to supplant all others,=20 > including POSIX whose threading support will be mostly for backwards=20 > compatibility and may even become deprecated. What do they put in the water at WG14 meetings? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 15:17:05 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CE0E106566B; Wed, 21 Dec 2011 15:17:05 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 22B1F8FC0C; Wed, 21 Dec 2011 15:17:05 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id F3E276A17; Wed, 21 Dec 2011 15:16:58 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id B1A6980C9; Wed, 21 Dec 2011 16:16:58 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: "O. Hartmann" References: <58923.1324292241@critter.freebsd.dk> <4EEF765D.4090300@zedat.fu-berlin.de> Date: Wed, 21 Dec 2011 16:16:58 +0100 In-Reply-To: <4EEF765D.4090300@zedat.fu-berlin.de> (O. Hartmann's message of "Mon, 19 Dec 2011 18:37:33 +0100") Message-ID: <868vm6t0np.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: threads@freebsd.org, Poul-Henning Kamp , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 15:17:05 -0000 "O. Hartmann" writes: > How is the other BSD sibbling, NetBSD, dealing with such things? NetBSD > is supposed to run on a trmendous variety of hardware, even a mixture of > bigendian and littleenddian and I'm quite sure they must have overcome > this probleme anyway. The same way FreeBSD does: where ordering matters, use explicit conversions when reading and writing. The conversion functions / macros are defined in such a manner that unnecessary conversions (e.g. host to little-endian on a little-endian system) do not generate any code at all. The only downside is that you can't directly compare variables unless you're certain that they're both in host order. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 15:28:28 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6203F106564A; Wed, 21 Dec 2011 15:28:28 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1EB858FC0A; Wed, 21 Dec 2011 15:28:28 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id AE7A446B09; Wed, 21 Dec 2011 10:28:27 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3513AB93A; Wed, 21 Dec 2011 10:28:27 -0500 (EST) From: John Baldwin To: "Dag-Erling =?utf-8?q?Sm=C3=B8rgrav?=" Date: Wed, 21 Dec 2011 10:28:26 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <73233.1324389741@critter.freebsd.dk> <86hb0ut1hq.fsf@ds4.des.no> In-Reply-To: <86hb0ut1hq.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201112211028.26780.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 21 Dec 2011 10:28:27 -0500 (EST) Cc: Poul-Henning Kamp , freebsd-arch@freebsd.org, Niall Douglas , freebsd-threads@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 15:28:28 -0000 On Wednesday, December 21, 2011 9:58:57 am Dag-Erling Sm=C3=B8rgrav wrote: > "Poul-Henning Kamp" writes: > > There is no way this can be impossible on a platform which can > > implement a mutex in the first place: > > > > > > mtx_lock(l) > > { > > atomic_magic_lock(l->lock_field) > > l->id =3D thread_id; > > } >=20 > OK >=20 > > mtx_unlock(l) > > { > > assert(l->id =3D=3D thread_id); > > l->id =3D NULL; > > atomic_magic_unlock(l->lock_field) > > } >=20 > susceptible to race conditions How so? Assume that atomic_magic_unlock() uses a release barrier on the store to l->lock_field such that the write to l->id will post to all CPUs before the write to l->lock_field. > > mtx_assert_held(l) > > { > > assert(l->lock-field !=3D 0); > > assert(l->id =3D=3D thread_id); > > } >=20 > susceptible to race conditions How so? While the current thread holds the lock, it is always coherent with the lock's state (it can't read "stale" values because it is the last thread to write to the lock, and if the thread migrates to another CPU, the mechanics of migrating to another CPU will use sufficient barriers that the new CPU will see all the writes done by this thread). Given that, if you hold the lock, the assertions will never fail while the current thread holds the lock. Similarly, the current thread will always see at least the newest values it wrote to the lock (it may also possibly see writes by another thread/CPU to the lock since it last wrote to the lock, but these are not guaranteed). However, that is sufficient to ensure that least one of the above assertions will fail if the current thread does not hold the lock. Recall that the last tokens it writes to the lock when it releases the lock is to clear both the lock_field and id fields. After such writes, mtx_assert_held() will fail. If another thread acquires the lock and the CPU only sees the first write to lock_field, then the first assertion will not trip, but the second one will (the thread will never see the old value where it thinks 'l->id' is itself as it is guaranteed to see a value that is at least as new as it's last write (which set it to NULL), and no other thread is going to set 'l->id' to this thread's ID). Keep in mind, all that you have to guarantee for an assertion of this type is that the observed state will match the 'locked by current thread' state when the mutex is locked and will look like something else when the mutex isn't locked by this thread. The observed state can be stale, but the assertion will still work correctly as it does not depend on the specific details of the non-owned state, merely that it is not equal to the locked staet. We use this same practice to use non-atomic ops on the mtx_recursed field in our in-kernel mutex implementation as well as it is altered only=20 while the main lock field is locked by the current thread. > The canonical solution is to use some low-level lock primitive (worst > case, a critical section) to protect the mutex structure, but then you > need some way for mtx_lock() to sleep if the mutex is held and some way > for mtx_unlock() to wake sleepers. You also need to lock the mutex > structure in mtx_assert_held(), unless l->id is atomic. l->id is not required to be atomic I believe, but it is not hard to make that true. On many platforms pointers do fit in a word and thus their loads and stores are atomic. You could also use a cookie for the thread id that is an atomic type if your pointers are not atomic (just assign a unique integer id to each thread on thread creation, etc.). =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 15:37:36 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 629AB1065673; Wed, 21 Dec 2011 15:37:36 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 29E408FC1B; Wed, 21 Dec 2011 15:37:35 +0000 (UTC) Received: by pbcc3 with SMTP id c3so5990134pbc.13 for ; Wed, 21 Dec 2011 07:37:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=On8O59LF/I7JcHFVHQ9ZNJdUybzJch9in9TtjoMED8U=; b=exU3x1+Lm8hqUcqiTT6n1j3uuKBl6+w9NpsFcqoA22rZ+R6Ea1QLdo3m7Q3i+aCwpr P20Wn95PiMfiFCHCEwy7fHzwOVZixLfbYHdfZcYjS69fXvszEUSk6tm5X1gg+ffPAl7W nInm9v4sbzmPT/U1CEI+iYgddZOtI4DVRAtjs= MIME-Version: 1.0 Received: by 10.68.189.97 with SMTP id gh1mr13880346pbc.37.1324481854813; Wed, 21 Dec 2011 07:37:34 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.55.136 with HTTP; Wed, 21 Dec 2011 07:37:34 -0800 (PST) In-Reply-To: <201112211028.26780.jhb@freebsd.org> References: <73233.1324389741@critter.freebsd.dk> <86hb0ut1hq.fsf@ds4.des.no> <201112211028.26780.jhb@freebsd.org> Date: Wed, 21 Dec 2011 07:37:34 -0800 X-Google-Sender-Auth: GJuA8akkMrwJqeD3Q4-3Y3HKrdI Message-ID: From: mdf@FreeBSD.org To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= , Poul-Henning Kamp , freebsd-threads@freebsd.org, Niall Douglas , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 15:37:36 -0000 2011/12/21 John Baldwin : > On Wednesday, December 21, 2011 9:58:57 am Dag-Erling Sm=F8rgrav wrote: >> "Poul-Henning Kamp" writes: >> > There is no way this can be impossible on a platform which can >> > implement a mutex in the first place: >> > >> > >> > =A0 =A0 mtx_lock(l) >> > =A0 =A0 { >> > =A0 =A0 =A0 =A0 =A0 =A0 atomic_magic_lock(l->lock_field) >> > =A0 =A0 =A0 =A0 =A0 =A0 l->id =3D thread_id; >> > =A0 =A0 } >> >> OK >> >> > =A0 =A0 mtx_unlock(l) >> > =A0 =A0 { >> > =A0 =A0 =A0 =A0 =A0 =A0 assert(l->id =3D=3D thread_id); >> > =A0 =A0 =A0 =A0 =A0 =A0 l->id =3D NULL; >> > =A0 =A0 =A0 =A0 =A0 =A0 atomic_magic_unlock(l->lock_field) >> > =A0 =A0 } >> >> susceptible to race conditions > > How so? =A0Assume that atomic_magic_unlock() uses a release barrier on > the store to l->lock_field such that the write to l->id will post to > all CPUs before the write to l->lock_field. ... which it has to do if the mutex is to do anything useful. Having started on PPC assembly, the concept of 'acquire' and 'release' semantics still make no sense to me. On PPC, the mutex code must issue a sync before clearing the lock word, to guarantee all stores performed before the lock is released are visible before any CPU can see the lock is free. I assume that's the same thing as 'release' semantics. If this isn't done then a CPU can read a value 'protected' by the mutex and get wrong information, rendering the mutex non-functional for memory protection but still function to (for some reason) have only one thread executing a given section of code at a time. If this isn't possible on the hardware (i.e. no memory barrier instructions) then the hardware cannot implement a mutex with useful semantics at all. But note that one of PHK's complaints was that the threaded memory model isn't well specified... Cheers, matthew >> > =A0 =A0 mtx_assert_held(l) >> > =A0 =A0 { >> > =A0 =A0 =A0 =A0 =A0 =A0 assert(l->lock-field !=3D 0); >> > =A0 =A0 =A0 =A0 =A0 =A0 assert(l->id =3D=3D thread_id); >> > =A0 =A0 } >> >> susceptible to race conditions > > How so? =A0While the current thread holds the lock, it is always coherent > with the lock's state (it can't read "stale" values because it is the > last thread to write to the lock, and if the thread migrates to another > CPU, the mechanics of migrating to another CPU will use sufficient > barriers that the new CPU will see all the writes done by this thread). > Given that, if you hold the lock, the assertions will never fail while > the current thread holds the lock. =A0Similarly, the current thread will > always see at least the newest values it wrote to the lock (it may also > possibly see writes by another thread/CPU to the lock since it last > wrote to the lock, but these are not guaranteed). =A0However, that is > sufficient to ensure that least one of the above assertions will fail if > the current thread does not hold the lock. =A0Recall that the last tokens > it writes to the lock when it releases the lock is to clear both the > lock_field and id fields. =A0 After such writes, mtx_assert_held() will > fail. =A0If another thread acquires the lock and the CPU only sees the > first write to lock_field, then the first assertion will not trip, but > the second one will (the thread will never see the old value where it > thinks 'l->id' is itself as it is guaranteed to see a value that is at > least as new as it's last write (which set it to NULL), and no other > thread is going to set 'l->id' to this thread's ID). > > Keep in mind, all that you have to guarantee for an assertion of this > type is that the observed state will match the 'locked by current thread' > state when the mutex is locked and will look like something else when the > mutex isn't locked by this thread. =A0The observed state can be stale, bu= t > the assertion will still work correctly as it does not depend on the > specific details of the non-owned state, merely that it is not equal to > the locked staet. > > We use this same practice to use non-atomic ops on the mtx_recursed > field in our in-kernel mutex implementation as well as it is altered only > while the main lock field is locked by the current thread. > >> The canonical solution is to use some low-level lock primitive (worst >> case, a critical section) to protect the mutex structure, but then you >> need some way for mtx_lock() to sleep if the mutex is held and some way >> for mtx_unlock() to wake sleepers. =A0You also need to lock the mutex >> structure in mtx_assert_held(), unless l->id is atomic. > > l->id is not required to be atomic I believe, but it is not hard to make > that true. =A0On many platforms pointers do fit in a word and thus their > loads and stores are atomic. =A0You could also use a cookie for the threa= d > id that is an atomic type if your pointers are not atomic (just assign a > unique integer id to each thread on thread creation, etc.). > > -- > John Baldwin > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 17:27:43 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F08CA106566C; Wed, 21 Dec 2011 17:27:43 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id AC10D8FC13; Wed, 21 Dec 2011 17:27:43 +0000 (UTC) Received: from [10.30.101.53] ([209.117.142.2]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBLHN2ST079514 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Wed, 21 Dec 2011 10:23:05 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <78246.1324394062@critter.freebsd.dk> Date: Wed, 21 Dec 2011 10:22:55 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <6CCA5C04-FC68-4B5F-911A-5F26EBFA296F@bsdimp.com> References: <78246.1324394062@critter.freebsd.dk> To: "Poul-Henning Kamp" X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Wed, 21 Dec 2011 10:23:06 -0700 (MST) Cc: threads@freebsd.org, Niall Douglas , arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 17:27:44 -0000 On Dec 20, 2011, at 8:14 AM, Poul-Henning Kamp wrote: > In message <4EF09FFD.7768.B66F73ED@s_sourceforge.nedprod.com>, "Niall = Douglas"=20 > writes: >=20 >>> And maybe, in trying to express that using a real-world example, >>> the standards comittee would realize that UTC was a mistake, and >>> changed the timeout argument to a relative time interval instead, >>> like for instance the poll(2) system-call. >>=20 >> There was some very good argument against relative periods. I=20 >> honestly can't remember what that was. It was a long time ago. >=20 > There are no good arguments against relative periods, in particular > not when the crap they are replaced with requires you to put a > loop around the sleep to get the desired behaviour in the first > place. >=20 > The fact that you cant even remeber the argument doesn't in any way > make it any more convincing. When time changes in the system, as it is wont to do occasionally, then = absolute time arguments will screw you. There's no way to get them = right that isn't a massively horrible kludge. Let's say you want to = sleep for no more than 1s. You get the time, add 1s to it, get = preempted, ntpd or somebody else notices the clocks are 1 year fast and = adjusts, you get a quatum again and make the call with a timeout now = 1-year in the future. What could possibly outweigh that negative? Warner From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 17:36:44 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B95C106566C; Wed, 21 Dec 2011 17:36:44 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 564EB8FC18; Wed, 21 Dec 2011 17:36:44 +0000 (UTC) Received: from [10.30.101.53] ([209.117.142.2]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBLHXN7D079556 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Wed, 21 Dec 2011 10:33:25 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> Date: Wed, 21 Dec 2011 10:33:16 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: >, <3065.1324375763@critter.freebsd.dk> <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> To: "Niall Douglas" X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Wed, 21 Dec 2011 10:33:26 -0700 (MST) Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 17:36:44 -0000 On Dec 20, 2011, at 5:50 AM, Niall Douglas wrote: > The job was NOT done half-arsed. If you had any experience of sitting=20= > on these committees you would know how much dedication and effort is=20= > put into standards, especially JTC1 SC22 subcommittees. Every single=20= > API in there has been studied and pored over at length across=20 > multiple years. >=20 > Everything is the way it is for a good reason. If it doesn't make=20 > sense to you that's most likely because you're not half as=20 > experienced or clever as you think you are. If you really want to=20 > know why something is the way it is, all discussion regarding all=20 > points is documented in full. Incredible claims require incredible proof. The APIs speak for = themselves: they are half-assed (and the wrong half in some cases). To = assert that they are somehow clever and we're stupid requires that one = walk through the cleverness. The participants in this thread likely = have a combined century of implementation experience with threads. Perhaps you can point us to the archives where all this discussion is = available? Warner From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 17:47:18 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0580106564A; Wed, 21 Dec 2011 17:47:18 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 7B6E58FC12; Wed, 21 Dec 2011 17:47:18 +0000 (UTC) Received: from [10.30.101.53] ([209.117.142.2]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBLHfcO9079629 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Wed, 21 Dec 2011 10:41:40 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: Warner Losh In-Reply-To: <868vm6t0np.fsf@ds4.des.no> Date: Wed, 21 Dec 2011 10:41:33 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <3ECEEFA0-6542-46CE-86D5-8A4D8226C81D@bsdimp.com> References: <58923.1324292241@critter.freebsd.dk> <4EEF765D.4090300@zedat.fu-berlin.de> <868vm6t0np.fsf@ds4.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Wed, 21 Dec 2011 10:41:41 -0700 (MST) Cc: threads@freebsd.org, Poul-Henning Kamp , "O. Hartmann" , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 17:47:18 -0000 On Dec 21, 2011, at 8:16 AM, Dag-Erling Sm=F8rgrav wrote: > "O. Hartmann" writes: >> How is the other BSD sibbling, NetBSD, dealing with such things? = NetBSD >> is supposed to run on a trmendous variety of hardware, even a mixture = of >> bigendian and littleenddian and I'm quite sure they must have = overcome >> this probleme anyway. >=20 > The same way FreeBSD does: where ordering matters, use explicit > conversions when reading and writing. The conversion functions / = macros > are defined in such a manner that unnecessary conversions (e.g. host = to > little-endian on a little-endian system) do not generate any code at > all. The only downside is that you can't directly compare variables > unless you're certain that they're both in host order. And it is difficult for automated tools to help you know if you are = "sure" or not. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 18:07:12 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07129106566C; Wed, 21 Dec 2011 18:07:12 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe03.c2i.net [212.247.154.66]) by mx1.freebsd.org (Postfix) with ESMTP id 2338B8FC13; Wed, 21 Dec 2011 18:07:10 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe03.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 53409308; Wed, 21 Dec 2011 18:57:06 +0100 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Wed, 21 Dec 2011 18:54:40 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> In-Reply-To: X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112211854.40798.hselasky@c2i.net> Cc: threads@freebsd.org, Niall Douglas , arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 18:07:12 -0000 On Wednesday 21 December 2011 18:33:16 Warner Losh wrote: > On Dec 20, 2011, at 5:50 AM, Niall Douglas wrote: > > The job was NOT done half-arsed. If you had any experience of sitting > > on these committees you would know how much dedication and effort is > > put into standards, especially JTC1 SC22 subcommittees. Every single > > API in there has been studied and pored over at length across > > multiple years. > > > > Everything is the way it is for a good reason. If it doesn't make > > sense to you that's most likely because you're not half as > > experienced or clever as you think you are. If you really want to > > know why something is the way it is, all discussion regarding all > > points is documented in full. > > Incredible claims require incredible proof. The APIs speak for themselves: > they are half-assed (and the wrong half in some cases). To assert that > they are somehow clever and we're stupid requires that one walk through > the cleverness. The participants in this thread likely have a combined > century of implementation experience with threads. > > Perhaps you can point us to the archives where all this discussion is > available? > Hi, Absolute timeouts is no good idea! We should stick with kernel-ticks when possible :-) --HPS From owner-freebsd-arch@FreeBSD.ORG Wed Dec 21 18:07:12 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07129106566C; Wed, 21 Dec 2011 18:07:12 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe03.c2i.net [212.247.154.66]) by mx1.freebsd.org (Postfix) with ESMTP id 2338B8FC13; Wed, 21 Dec 2011 18:07:10 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe03.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 53409308; Wed, 21 Dec 2011 18:57:06 +0100 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Wed, 21 Dec 2011 18:54:40 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> In-Reply-To: X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112211854.40798.hselasky@c2i.net> Cc: threads@freebsd.org, Niall Douglas , arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 18:07:12 -0000 On Wednesday 21 December 2011 18:33:16 Warner Losh wrote: > On Dec 20, 2011, at 5:50 AM, Niall Douglas wrote: > > The job was NOT done half-arsed. If you had any experience of sitting > > on these committees you would know how much dedication and effort is > > put into standards, especially JTC1 SC22 subcommittees. Every single > > API in there has been studied and pored over at length across > > multiple years. > > > > Everything is the way it is for a good reason. If it doesn't make > > sense to you that's most likely because you're not half as > > experienced or clever as you think you are. If you really want to > > know why something is the way it is, all discussion regarding all > > points is documented in full. > > Incredible claims require incredible proof. The APIs speak for themselves: > they are half-assed (and the wrong half in some cases). To assert that > they are somehow clever and we're stupid requires that one walk through > the cleverness. The participants in this thread likely have a combined > century of implementation experience with threads. > > Perhaps you can point us to the archives where all this discussion is > available? > Hi, Absolute timeouts is no good idea! We should stick with kernel-ticks when possible :-) --HPS From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 16:05:33 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A68111065679; Thu, 22 Dec 2011 16:05:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5166E8FC1A; Thu, 22 Dec 2011 16:05:33 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id E630C46B09; Thu, 22 Dec 2011 11:05:32 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7DAA6B914; Thu, 22 Dec 2011 11:05:32 -0500 (EST) From: John Baldwin To: arch@freebsd.org Date: Thu, 22 Dec 2011 11:05:31 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201112221105.31746.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Dec 2011 11:05:32 -0500 (EST) Cc: Robert Watson Subject: Post NOTE_ATTRIB EVFILT_VNODE events for extattr changes X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 16:05:33 -0000 A co-worker noticed that there is currently no way to get an EVFILT_VNODE notification if the extended attributes for a given file change. I looked at OS X and found that it posts a NOTE_ATTRIB event (typically used for VOP_SETATTR()) for successful attempts to set or delete an extended attribute. I have a patch to do the same along with a test application for both OS X and FreeBSD. With this patch FreeBSD now has the same behavior as OS X. Index: kern/vnode_if.src =================================================================== --- kern/vnode_if.src (revision 228777) +++ kern/vnode_if.src (working copy) @@ -569,6 +569,7 @@ %% deleteextattr vp E E E +%! deleteextattr post vop_deleteextattr_post vop_deleteextattr { IN struct vnode *vp; @@ -580,6 +581,7 @@ %% setextattr vp E E E +%! setextattr post vop_setextattr_post vop_setextattr { IN struct vnode *vp; Index: kern/vfs_subr.c =================================================================== --- kern/vfs_subr.c (revision 228777) +++ kern/vfs_subr.c (working copy) @@ -4033,6 +4033,15 @@ } void +vop_deleteextattr_post(void *ap, int rc) +{ + struct vop_setattr_args *a = ap; + + if (!rc) + VFS_KNOTE_LOCKED(a->a_vp, NOTE_ATTRIB); +} + +void vop_link_post(void *ap, int rc) { struct vop_link_args *a = ap; @@ -4114,6 +4123,15 @@ } void +vop_setextattr_post(void *ap, int rc) +{ + struct vop_setattr_args *a = ap; + + if (!rc) + VFS_KNOTE_LOCKED(a->a_vp, NOTE_ATTRIB); +} + +void vop_symlink_post(void *ap, int rc) { struct vop_symlink_args *a = ap; Index: sys/vnode.h =================================================================== --- sys/vnode.h (revision 228777) +++ sys/vnode.h (working copy) @@ -705,6 +705,7 @@ /* These are called from within the actual VOPS. */ void vop_create_post(void *a, int rc); +void vop_deleteextattr_post(void *a, int rc); void vop_link_post(void *a, int rc); void vop_lock_pre(void *a); void vop_lock_post(void *a, int rc); @@ -717,6 +718,7 @@ void vop_rename_pre(void *a); void vop_rmdir_post(void *a, int rc); void vop_setattr_post(void *a, int rc); +void vop_setextattr_post(void *a, int rc); void vop_strategy_pre(void *a); void vop_symlink_post(void *a, int rc); void vop_unlock_post(void *a, int rc); /*- * Test to see if attempts to set or remove extended attributes * provoke a NOTE_ATTRIB vnode kevent. */ #ifdef __FreeBSD__ #include #include #endif #ifdef __APPLE__ #include #endif #include #include #include #include #include #include #include #include #include #include #ifdef __FreeBSD__ static ssize_t getea(int fd, const char *name, void *value, size_t size) { return (extattr_get_fd(fd, EXTATTR_NAMESPACE_USER, name, value, size)); } static int setea(int fd, const char *name, const void *value, size_t size) { ssize_t retval; retval = extattr_set_fd(fd, EXTATTR_NAMESPACE_USER, name, value, size); if (retval < 0) return (-1); return (0); } static int delea(int fd, const char *name) { return (extattr_delete_fd(fd, EXTATTR_NAMESPACE_USER, name)); } static int listea(int fd, void *buf, size_t size) { return (extattr_list_fd(fd, EXTATTR_NAMESPACE_USER, buf, size)); } static int compare_ealist(const char *name, void *buf, size_t size) { char *p; int i, len; i = 0; p = buf; if (size == 0) return (1); len = p[0]; p++; if (len != size - 1) return (1); if (strlen(name) != len) return (1); return (strncmp(p, name, len)); } #endif #ifdef __APPLE__ static ssize_t getea(int fd, const char *name, void *value, size_t size) { return (fgetxattr(fd, name, value, size, 0, 0)); } static int setea(int fd, const char *name, const void *value, size_t size) { return (fsetxattr(fd, name, value, size, 0, 0)); } static int delea(int fd, const char *name) { return (fremovexattr(fd, name, 0)); } static int listea(int fd, void *buf, size_t size) { return (flistxattr(fd, buf, size, 0)); } static int compare_ealist(const char *name, void *buf, size_t size) { if (size == 0) return (1); if (strlen(name) != size - 1) return (1); return (strncmp(buf, name, size)); } #endif #define EA_NAME "foo" char template[] = "/tmp/kevent_extattr.XXXXXX"; static int fd, kq; static void check_ea(const char *value, const char *desc) { ssize_t retval; size_t size; char *buf; retval = getea(fd, EA_NAME, NULL, 0); if (retval < 0) { if (errno == ENOATTR) { if (value != NULL) printf("Missing ea: %s\n", desc); return; } err(1, "getea"); } size = (size_t)retval; buf = malloc(size); retval = getea(fd, EA_NAME, buf, size); if (retval < 0) err(1, "getea"); if ((size_t)retval != size) errx(1, "getea: short read"); if (value == NULL) printf("Unexpected ea value %*s: %s\n", (int)size, buf, desc); else if (strncmp(value, buf, size) != 0) printf("Mismatched ea values %*s vs %s: %s\n", (int)size, buf, value, desc); free(buf); } static void check_ealist(const char *name, const char *desc) { ssize_t retval; size_t size; char *buf; retval = listea(fd, NULL, 0); if (retval < 0) err(1, "listea"); size = (size_t)retval; if (size == 0) { if (name != NULL) { printf("Missing ea: %s\n", desc); return; } return; } buf = malloc(size); retval = listea(fd, buf, size); if (retval < 0) err(1, "listea"); if ((size_t)retval != size) errx(1, "listea: short read"); if (name == NULL) printf("Unexpected ea: %s\n", desc); else if (compare_ealist(name, buf, size) != 0) printf("Mismatched ea list: %s\n", desc); free(buf); } static void check_event(bool expected, const char *desc) { struct timespec ts = { 0, 0 }; struct kevent ev; int retval; retval = kevent(kq, NULL, 0, &ev, 1, &ts); if (retval < 0) err(1, "kevent"); if (!expected) { if (retval != 0) printf("Unexpected kevent: %s\n", desc); } else { if (retval == 0) printf("Missing kevent: %s\n", desc); else if (ev.fflags != NOTE_ATTRIB) printf("Wrong flags (%x vs %x): %s\n", ev.fflags, NOTE_ATTRIB, desc); } } int main(int ac, char **av) { struct kevent ev; kq = kqueue(); if (kq < 0) err(1, "kqueue"); fd = mkstemp(template); if (fd < 0) err(1, "mkstemp"); if (unlink(template) < 0) err(1, "unlink"); EV_SET(&ev, fd, EVFILT_VNODE, EV_ADD | EV_CLEAR, NOTE_ATTRIB, 0, 0); if (kevent(kq, &ev, 1, NULL, 0, NULL) < 0) err(1, "kevent(EV_ADD)"); check_event(false, "initial check"); check_ea(NULL, "initial check"); check_ealist(NULL, "initial check"); if (setea(fd, EA_NAME, "bar", sizeof("bar")) < 0) err(1, "setea(%s = \"bar\")", EA_NAME); check_event(true, "first set"); /* * The check_ea() reads the EA and should not have triggered * an event. */ check_ea("bar", "first set"); check_event(false, "getea"); /* * The ea list requests in this check should not trigger an * event either. */ check_ealist(EA_NAME, "first set"); check_event(false, "listea"); if (setea(fd, EA_NAME, "baz", sizeof("baz")) < 0) err(1, "setea(%s = \"baz\")", EA_NAME); check_event(true, "second set"); check_ea("baz", "second set"); check_ealist(EA_NAME, "second set"); if (delea(fd, EA_NAME) < 0) err(1, "delea(%s)", EA_NAME); check_event(true, "remove"); check_ea(NULL, "remove"); check_ealist(NULL, "remove"); close(fd); close(kq); return (0); } -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 16:57:41 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC681106564A; Thu, 22 Dec 2011 16:57:41 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 23EF08FC12; Thu, 22 Dec 2011 16:57:40 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id 1C8489EE476; Thu, 22 Dec 2011 16:57:40 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324573060; bh=rzZdSZC63JQ9PudQzLERKiudK2wP08tJeEQfgXBIB8Y=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=h7KPxFvBAvGI0wH3PBKUaAew/h0DnBfgak6PZrhxq3GfkKmsjOuG8dSYqssF1glfA 6RzBiahLNUb2PZnqL4u5PwopWMWgZ7C/BiJ/t4kNLZkThCBcMCmimXpFOT9sXCVYZd Ccm5E8OYES6sMzviImBRMyBML3ug7/5CtIgFlZzU= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id 004B59EE476; Thu, 22 Dec 2011 16:57:38 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324573059; bh=rzZdSZC63JQ9PudQzLERKiudK2wP08tJeEQfgXBIB8Y=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=yYClqhU/nx5gv6FmNewGcYeIHAJ6yI8tyME4XyOLJRUS62LQClAbwpkWw3ixW2mqa VcKZ1aKHQVfDCq3Ly4orUaozdrpstlfVc+aHjW1tgHs2AAIwgCCrxC2Ppi2lND0tuP FDo4B5EQXWu8Bo9UjLx8ym26a4qkTO2TLo8mfsvI= From: "Niall Douglas" To: threads@freebsd.org, arch@freebsd.org Date: Thu, 22 Dec 2011 16:57:38 -0000 MIME-Version: 1.0 Message-ID: <4EF36182.13367.C1336C10@s_sourceforge.nedprod.com> Priority: normal In-reply-to: References: , <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com>, X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 16:57:41 -0000 On 21 Dec 2011 at 10:33, Warner Losh wrote: > On Dec 20, 2011, at 5:50 AM, Niall Douglas wrote: > > The job was NOT done half-arsed. If you had any experience of sitting > > on these committees you would know how much dedication and effort is > > put into standards, especially JTC1 SC22 subcommittees. Every single > > API in there has been studied and pored over at length across > > multiple years. > > > > Everything is the way it is for a good reason. If it doesn't make > > sense to you that's most likely because you're not half as > > experienced or clever as you think you are. If you really want to > > know why something is the way it is, all discussion regarding all > > points is documented in full. > > Incredible claims require incredible proof. The APIs speak for > themselves: they are half-assed (and the wrong half in some cases). To > assert that they are somehow clever and we're stupid requires that one > walk through the cleverness. The participants in this thread likely > have a combined century of implementation experience with threads. That's not what I said. You're putting words in mouth. Another in this list did the same, I corrected him and here you're doing it again. Quite frankly I've had enough of the rudeness from this list. I saw questions here, I tried to answer them to the best of my ability and all I'm getting is ass whipping from people who don't even bother to read previous posts on the same topic, and quite a few who show an amazing level of emotional and technical ignorance. You can all take a running jump as far as I'm concerned. I'm not being paid to do this. I *was* trying to help. > Perhaps you can point us to the archives where all this discussion is > available? All ISO work is publicly documented from the usual channels, including rationales for all changes to draft standards documents. There's this website called "Google" which will amazingly tell you the answer. I even posted a link to the WG14 site in a previous post. Good luck with your C11 implementation freebsd-threads! I've had enough of this and I'm signing out. Oh, and merry christmas! Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 16:57:41 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE7D91065670; Thu, 22 Dec 2011 16:57:41 +0000 (UTC) (envelope-from s_sourceforge@nedprod.com) Received: from europe2.nedproductions.biz (unknown [IPv6:2a02:748:100:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id A44078FC1B; Thu, 22 Dec 2011 16:57:41 +0000 (UTC) Received: by europe2.nedproductions.biz (Postfix, from userid 1003) id CF5D59EE74C; Thu, 22 Dec 2011 16:57:40 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324573060; bh=cGmdbyUvQ7gpsdJ/RJhi41u4Qt98ItuGuW5XFZg3iJw=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=uKN510rTEAzskUQ6lY+eCqeByyZiePJ5OVwqZ0dNU3D1CAckkkNLoXA4qrONs+wWI qWPlZnIBBNqwHkSUK/BZCwQUU5RObXudyeG1HF7XzUEx8bJ0pG914Rq+l/CwJjnf50 BXLdGWFwsDC0nP6HSYejDkQswEKVN7aotT0gwDls= X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on europe2.nedproductions.biz X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from [192.168.2.8] (dsl-076-041.cust.imagine.ie [87.232.76.41]) by europe2.nedproductions.biz (Postfix) with ESMTPSA id A64A79EE488; Thu, 22 Dec 2011 16:57:39 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nedprod.com; s=mail; t=1324573060; bh=cGmdbyUvQ7gpsdJ/RJhi41u4Qt98ItuGuW5XFZg3iJw=; h=From:To:Date:MIME-Version:Subject:Message-ID:In-reply-to: References:Content-type:Content-transfer-encoding: Content-description; b=uKN510rTEAzskUQ6lY+eCqeByyZiePJ5OVwqZ0dNU3D1CAckkkNLoXA4qrONs+wWI qWPlZnIBBNqwHkSUK/BZCwQUU5RObXudyeG1HF7XzUEx8bJ0pG914Rq+l/CwJjnf50 BXLdGWFwsDC0nP6HSYejDkQswEKVN7aotT0gwDls= From: "Niall Douglas" To: threads@freebsd.org, arch@freebsd.org Date: Thu, 22 Dec 2011 16:57:38 -0000 MIME-Version: 1.0 Message-ID: <4EF36182.29429.C1336CBC@s_sourceforge.nedprod.com> Priority: normal In-reply-to: <6CCA5C04-FC68-4B5F-911A-5F26EBFA296F@bsdimp.com> References: <78246.1324394062@critter.freebsd.dk>, <6CCA5C04-FC68-4B5F-911A-5F26EBFA296F@bsdimp.com> X-mailer: Pegasus Mail for Windows (4.62) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Cc: Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 16:57:42 -0000 On 21 Dec 2011 at 10:22, Warner Losh wrote: > When time changes in the system, as it is wont to do occasionally, then > absolute time arguments will screw you. There's no way to get them > right that isn't a massively horrible kludge. Let's say you want to > sleep for no more than 1s. You get the time, add 1s to it, get > preempted, ntpd or somebody else notices the clocks are 1 year fast and > adjusts, you get a quatum again and make the call with a timeout now > 1-year in the future. > > What could possibly outweigh that negative? As I said earlier, I don't remember what the reason was. And no, that doesn't mean it wasn't important - there is a vast volume of stuff being thrown at you during standards, if you can remember something from even a month ago you're doing well. That's why all decisions are documented in depth so it can get looked up later. Instead of shouting at me - who is trying to help you - go look it up if it matters so much to you. However, if I had to take a guess, I'd say that functional reliability can be higher when you can offer guarantees. Absolute times have higher potential of guarantees than relative, whereas the opposite is not true, because you can draw lines in the sand with absolute times and that's much harder with relative. You can also implement relative using an API taking absolute easily, whereas again the opposite is harder. I can see arguments for very long lived systems where uptime is typically in years that absolute would be a lot more reliable due to relative period drift e.g. if I want to be woken on the hour every hour, you don't want to be using a relative wait. Of course, you can then pull the system time and calculate the delta, but I digress. The point is that more information supplied to the kernel is usually better. Absolute times supply more information than relative. So there you go - and please note this paragraph is my best guess, and may have nothing to do with committee opinion. Instead of people repeatedly whinging about the system clock moving - and yes, people on standards committees are aware that system clocks can move, as they are that different cores can have different system clocks - I would be asking how does your scheduler cope with the clock moving, and how should clock syncs interact with userland? Be engineers instead of children. Ask how to solve the problem instead of complaining about split milk. Be glad you're implementing this on a POSIX system and not a non-POSIX system. Be glad there aren't a million lines of new code being needed to achieve compliance. The standard is now written in stone and is a full ISO/IEC document. Your time for helping shape it is over - we're in errata and bug fixing stage now. You should have been whinging a year or two years ago - it's not like the standard is a surprise to anyone paying attention. Its release date was widely publicised and draft spec documents are freely available over the past 18 months. I might add the C11 spec passed its last meeting with no objection from any committee member. The ISO participants consider it finished. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 16:59:18 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC7931065672; Thu, 22 Dec 2011 16:59:18 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 8D95D8FC0A; Thu, 22 Dec 2011 16:59:18 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 759F96D85; Thu, 22 Dec 2011 16:59:14 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 11D3681A6; Thu, 22 Dec 2011 17:59:13 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: John Baldwin References: <73233.1324389741@critter.freebsd.dk> <86hb0ut1hq.fsf@ds4.des.no> <201112211028.26780.jhb@freebsd.org> Date: Thu, 22 Dec 2011 17:59:13 +0100 In-Reply-To: <201112211028.26780.jhb@freebsd.org> (John Baldwin's message of "Wed, 21 Dec 2011 10:28:26 -0500") Message-ID: <86zkeksftq.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Poul-Henning Kamp , freebsd-threads@freebsd.org, Niall Douglas , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 16:59:18 -0000 John Baldwin writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Poul-Henning Kamp writes: > > > mtx_unlock(l) > > > { > > > assert(l->id =3D=3D thread_id); > > > l->id =3D NULL; > > > atomic_magic_unlock(l->lock_field) > > > } > > susceptible to race conditions > How so? I should have specified "if called from a thread that does not own the mutex" > > > mtx_assert_held(l) > > > { > > > assert(l->lock-field !=3D 0); > > > assert(l->id =3D=3D thread_id); > > > } > > susceptible to race conditions > How so? I was going to point out that the state of the mutex can change between the two asserts, but as you say, at least one of them is guaranteed to fail... *if* you assume that these fields can be read atomically, which was one of my objections. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 17:04:18 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E58EB106566C; Thu, 22 Dec 2011 17:04:18 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id A59AA8FC0A; Thu, 22 Dec 2011 17:04:18 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 02A036D8E; Thu, 22 Dec 2011 17:04:15 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 7C42181A9; Thu, 22 Dec 2011 18:04:14 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Hans Petter Selasky References: <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com> <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> <201112211854.40798.hselasky@c2i.net> Date: Thu, 22 Dec 2011 18:04:14 +0100 In-Reply-To: <201112211854.40798.hselasky@c2i.net> (Hans Petter Selasky's message of "Wed, 21 Dec 2011 18:54:40 +0100") Message-ID: <86vcp8sfld.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, threads@freebsd.org, Niall Douglas , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 17:04:19 -0000 Hans Petter Selasky writes: > Absolute timeouts is no good idea! We should stick with kernel-ticks when= =20 > possible :-) There is no such thing as a kernel in the C standard. All it knows about is the implementation and the program. The best solution would probably have been a timescale that counts the time elapsed since the start of the program. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 17:04:18 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E58EB106566C; Thu, 22 Dec 2011 17:04:18 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id A59AA8FC0A; Thu, 22 Dec 2011 17:04:18 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 02A036D8E; Thu, 22 Dec 2011 17:04:15 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 7C42181A9; Thu, 22 Dec 2011 18:04:14 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Hans Petter Selasky References: <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com> <4EF084A8.32369.B604AD16@s_sourceforge.nedprod.com> <201112211854.40798.hselasky@c2i.net> Date: Thu, 22 Dec 2011 18:04:14 +0100 In-Reply-To: <201112211854.40798.hselasky@c2i.net> (Hans Petter Selasky's message of "Wed, 21 Dec 2011 18:54:40 +0100") Message-ID: <86vcp8sfld.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, threads@freebsd.org, Niall Douglas , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 17:04:19 -0000 Hans Petter Selasky writes: > Absolute timeouts is no good idea! We should stick with kernel-ticks when= =20 > possible :-) There is no such thing as a kernel in the C standard. All it knows about is the implementation and the program. The best solution would probably have been a timescale that counts the time elapsed since the start of the program. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 17:08:53 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D598A106566C; Thu, 22 Dec 2011 17:08:53 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 911BA8FC12; Thu, 22 Dec 2011 17:08:53 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 33FCA5DAA; Thu, 22 Dec 2011 17:08:52 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBMH8p9l082796; Thu, 22 Dec 2011 17:08:51 GMT (envelope-from phk@phk.freebsd.dk) To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 22 Dec 2011 18:04:14 +0100." <86vcp8sfld.fsf@ds4.des.no> Content-Type: text/plain; charset=ISO-8859-1 Date: Thu, 22 Dec 2011 17:08:51 +0000 Message-ID: <82795.1324573731@critter.freebsd.dk> Cc: arch@freebsd.org, freebsd-arch@freebsd.org, threads@freebsd.org, Niall Douglas , Hans Petter Selasky Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 17:08:54 -0000 In message <86vcp8sfld.fsf@ds4.des.no>, =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= wr ites: >There is no such thing as a kernel in the C standard. All it knows >about is the implementation and the program. The best solution would >probably have been a timescale that counts the time elapsed since the >start of the program. That is requiring far more than necessary: All that is needed is a monotonic timescale, such as the, precisely named, CLOCK_MONOTONIC timescale POSIX introduced to solve the exact same kind of issues. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 17:08:53 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D598A106566C; Thu, 22 Dec 2011 17:08:53 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 911BA8FC12; Thu, 22 Dec 2011 17:08:53 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 33FCA5DAA; Thu, 22 Dec 2011 17:08:52 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBMH8p9l082796; Thu, 22 Dec 2011 17:08:51 GMT (envelope-from phk@phk.freebsd.dk) To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 22 Dec 2011 18:04:14 +0100." <86vcp8sfld.fsf@ds4.des.no> Content-Type: text/plain; charset=ISO-8859-1 Date: Thu, 22 Dec 2011 17:08:51 +0000 Message-ID: <82795.1324573731@critter.freebsd.dk> Cc: arch@freebsd.org, freebsd-arch@freebsd.org, threads@freebsd.org, Niall Douglas , Hans Petter Selasky Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 17:08:54 -0000 In message <86vcp8sfld.fsf@ds4.des.no>, =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= wr ites: >There is no such thing as a kernel in the C standard. All it knows >about is the implementation and the program. The best solution would >probably have been a timescale that counts the time elapsed since the >start of the program. That is requiring far more than necessary: All that is needed is a monotonic timescale, such as the, precisely named, CLOCK_MONOTONIC timescale POSIX introduced to solve the exact same kind of issues. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 17:52:20 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E2CA106566B; Thu, 22 Dec 2011 17:52:20 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 48E2C8FC25; Thu, 22 Dec 2011 17:52:20 +0000 (UTC) Received: from [10.0.0.63] (63.imp.bsdimp.com [10.0.0.63]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBMHp7pv090403 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Thu, 22 Dec 2011 10:51:10 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <4EF36182.29429.C1336CBC@s_sourceforge.nedprod.com> Date: Thu, 22 Dec 2011 10:51:07 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <60BA4BA7-70B9-4B4E-9139-8F6DAC303867@bsdimp.com> References: <78246.1324394062@critter.freebsd.dk>, <6CCA5C04-FC68-4B5F-911A-5F26EBFA296F@bsdimp.com> <4EF36182.29429.C1336CBC@s_sourceforge.nedprod.com> To: "Niall Douglas" X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Thu, 22 Dec 2011 10:51:10 -0700 (MST) Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 17:52:20 -0000 On Dec 22, 2011, at 9:57 AM, Niall Douglas wrote: > Instead of people repeatedly whinging about the system clock moving -=20= > and yes, people on standards committees are aware that system clocks=20= > can move, as they are that different cores can have different system=20= > clocks - I would be asking how does your scheduler cope with the=20 > clock moving, and how should clock syncs interact with userland? Be=20 > engineers instead of children. Ask how to solve the problem instead=20 > of complaining about split milk. Be glad you're implementing this on=20= > a POSIX system and not a non-POSIX system. Be glad there aren't a=20 > million lines of new code being needed to achieve compliance. Elapsed time since boot is typically used in cases where you need a = stable, monotonically increasing time base. Changing the system time = then becomes just a change to the boot time, but not the time elapsed = since boot. A pure monotonic time base is safe to specify absolute time = in. One that isn't monotonic, such as UTC or POSIX time_t, is unsafe. = Using it is akin to specifying a mutex system that has known races that = cannot be fixed in it. I'm sorry if my harsh remarks hurt your feelings, but the new APIs do = not learn from the well known mistakes of the past decade, so they = disappoint me. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 18:28:43 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC919106564A; Thu, 22 Dec 2011 18:28:43 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id A0B158FC0A; Thu, 22 Dec 2011 18:28:43 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 5616C46B06; Thu, 22 Dec 2011 13:28:43 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C30FDB962; Thu, 22 Dec 2011 13:28:42 -0500 (EST) From: John Baldwin To: "Dag-Erling =?utf-8?q?Sm=C3=B8rgrav?=" Date: Thu, 22 Dec 2011 13:28:41 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <73233.1324389741@critter.freebsd.dk> <201112211028.26780.jhb@freebsd.org> <86zkeksftq.fsf@ds4.des.no> In-Reply-To: <86zkeksftq.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201112221328.41221.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Dec 2011 13:28:42 -0500 (EST) Cc: Poul-Henning Kamp , freebsd-threads@freebsd.org, Niall Douglas , freebsd-arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 18:28:43 -0000 On Thursday, December 22, 2011 11:59:13 am Dag-Erling Sm=C3=B8rgrav wrote: > John Baldwin writes: > > Dag-Erling Sm=C3=B8rgrav writes: > > > Poul-Henning Kamp writes: > > > > mtx_unlock(l) > > > > { > > > > assert(l->id =3D=3D thread_id); > > > > l->id =3D NULL; > > > > atomic_magic_unlock(l->lock_field) > > > > } > > > susceptible to race conditions > > How so? >=20 > I should have specified "if called from a thread that does not own the > mutex" You can do the same check as mtx_assert_held() internally and fail the unlo= ck=20 request in that case (or abort or what have you). > > > > mtx_assert_held(l) > > > > { > > > > assert(l->lock-field !=3D 0); > > > > assert(l->id =3D=3D thread_id); > > > > } > > > susceptible to race conditions > > How so? >=20 > I was going to point out that the state of the mutex can change between > the two asserts, but as you say, at least one of them is guaranteed to > fail... *if* you assume that these fields can be read atomically, which > was one of my objections. I do think these have to be atomic. I think lock-field must be able to be= =20 atomically read by definition. I think it is not an unreasonable requireme= nt=20 to have the implementation implement a thread_id that fits in an atomic typ= e=20 if it is not able to encode the thread_id into the lock cookie itself. I do think if l->id was not atomic it might be feasible to see a value while the thread is not locked that is equivalent to the current thread's ID based on observing different parts of l->id from different writes. There might b= e=20 ways the implementation could guard against that however. In practice thou= gh=20 I think pointers are atomic with regards to loads and stores on nearly all= =20 machines. =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 18:32:35 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 476C5106566B; Thu, 22 Dec 2011 18:32:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1CC078FC08; Thu, 22 Dec 2011 18:32:35 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id C112646B06; Thu, 22 Dec 2011 13:32:34 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 56ABEB91E; Thu, 22 Dec 2011 13:32:34 -0500 (EST) From: John Baldwin To: freebsd-threads@freebsd.org Date: Thu, 22 Dec 2011 13:31:10 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <4EF059DC.26433.B55D8036@s_sourceforge.nedprod.com> <201112211854.40798.hselasky@c2i.net> <86vcp8sfld.fsf@ds4.des.no> In-Reply-To: <86vcp8sfld.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201112221331.11031.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Dec 2011 13:32:34 -0500 (EST) Cc: arch@freebsd.org, Dag-Erling =?utf-8?q?Sm=C3=B8rgrav?= , threads@freebsd.org, Hans Petter Selasky Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 18:32:35 -0000 On Thursday, December 22, 2011 12:04:14 pm Dag-Erling Sm=C3=B8rgrav wrote: > Hans Petter Selasky writes: > > Absolute timeouts is no good idea! We should stick with kernel-ticks wh= en=20 > > possible :-) >=20 > There is no such thing as a kernel in the C standard. All it knows > about is the implementation and the program. The best solution would > probably have been a timescale that counts the time elapsed since the > start of the program. You could do relative timeouts specified in some absolute timescale like us= or=20 ns or ms. You could even use a 'struct timespec' or some such to do that=20 similar to how the it_interval member of struct itimer is used with=20 setitimer(2). That is what programmers actually want to use, and invariabl= y=20 end up implementing in a wrapper API around APIs that use absolute timeouts. =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Dec 22 23:19:32 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E9F0106566B; Thu, 22 Dec 2011 23:19:32 +0000 (UTC) (envelope-from rwatson@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 138588FC0C; Thu, 22 Dec 2011 23:19:32 +0000 (UTC) Received: from [192.168.2.115] (host86-161-238-124.range86-161.btcentralplus.com [86.161.238.124]) by cyrus.watson.org (Postfix) with ESMTPSA id 9E3D346B0A; Thu, 22 Dec 2011 18:19:30 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <201112221105.31746.jhb@freebsd.org> Date: Thu, 22 Dec 2011 23:19:27 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201112221105.31746.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1084) Cc: arch@freebsd.org Subject: Re: Post NOTE_ATTRIB EVFILT_VNODE events for extattr changes X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 23:19:32 -0000 On 22 Dec 2011, at 16:05, John Baldwin wrote: > A co-worker noticed that there is currently no way to get an = EVFILT_VNODE=20 > notification if the extended attributes for a given file change. I = looked at=20 > OS X and found that it posts a NOTE_ATTRIB event (typically used for=20= > VOP_SETATTR()) for successful attempts to set or delete an extended = attribute. =20 > I have a patch to do the same along with a test application for both = OS X and=20 > FreeBSD. With this patch FreeBSD now has the same behavior as OS X. This seems reasonable to me, although I've not tested the patch you = attached myself. I'm not familiar with the pre/post-vop hook mechanism, but I assume that = this will fire when internal extended attribute I/O, for example = triggered by ACL changes, occurs? Robert >=20 > Index: kern/vnode_if.src > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- kern/vnode_if.src (revision 228777) > +++ kern/vnode_if.src (working copy) > @@ -569,6 +569,7 @@ >=20 >=20 > %% deleteextattr vp E E E > +%! deleteextattr post vop_deleteextattr_post >=20 > vop_deleteextattr { > IN struct vnode *vp; > @@ -580,6 +581,7 @@ >=20 >=20 > %% setextattr vp E E E > +%! setextattr post vop_setextattr_post >=20 > vop_setextattr { > IN struct vnode *vp; > Index: kern/vfs_subr.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- kern/vfs_subr.c (revision 228777) > +++ kern/vfs_subr.c (working copy) > @@ -4033,6 +4033,15 @@ > } >=20 > void > +vop_deleteextattr_post(void *ap, int rc) > +{ > + struct vop_setattr_args *a =3D ap; > + > + if (!rc) > + VFS_KNOTE_LOCKED(a->a_vp, NOTE_ATTRIB); > +} > + > +void > vop_link_post(void *ap, int rc) > { > struct vop_link_args *a =3D ap; > @@ -4114,6 +4123,15 @@ > } >=20 > void > +vop_setextattr_post(void *ap, int rc) > +{ > + struct vop_setattr_args *a =3D ap; > + > + if (!rc) > + VFS_KNOTE_LOCKED(a->a_vp, NOTE_ATTRIB); > +} > + > +void > vop_symlink_post(void *ap, int rc) > { > struct vop_symlink_args *a =3D ap; > Index: sys/vnode.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/vnode.h (revision 228777) > +++ sys/vnode.h (working copy) > @@ -705,6 +705,7 @@ >=20 > /* These are called from within the actual VOPS. */ > void vop_create_post(void *a, int rc); > +void vop_deleteextattr_post(void *a, int rc); > void vop_link_post(void *a, int rc); > void vop_lock_pre(void *a); > void vop_lock_post(void *a, int rc); > @@ -717,6 +718,7 @@ > void vop_rename_pre(void *a); > void vop_rmdir_post(void *a, int rc); > void vop_setattr_post(void *a, int rc); > +void vop_setextattr_post(void *a, int rc); > void vop_strategy_pre(void *a); > void vop_symlink_post(void *a, int rc); > void vop_unlock_post(void *a, int rc); >=20 >=20 > /*- > * Test to see if attempts to set or remove extended attributes > * provoke a NOTE_ATTRIB vnode kevent. > */ >=20 > #ifdef __FreeBSD__ > #include > #include > #endif >=20 > #ifdef __APPLE__ > #include > #endif >=20 > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include >=20 > #ifdef __FreeBSD__ > static ssize_t > getea(int fd, const char *name, void *value, size_t size) > { >=20 > return (extattr_get_fd(fd, EXTATTR_NAMESPACE_USER, name, > value, size)); > } >=20 > static int > setea(int fd, const char *name, const void *value, size_t size) > { > ssize_t retval; >=20 > retval =3D extattr_set_fd(fd, EXTATTR_NAMESPACE_USER, name, > value, size); > if (retval < 0) > return (-1); > return (0); > } >=20 > static int > delea(int fd, const char *name) > { >=20 > return (extattr_delete_fd(fd, EXTATTR_NAMESPACE_USER, name)); > } >=20 > static int > listea(int fd, void *buf, size_t size) > { >=20 > return (extattr_list_fd(fd, EXTATTR_NAMESPACE_USER, buf, size)); > } >=20 > static int > compare_ealist(const char *name, void *buf, size_t size) > { > char *p; > int i, len; >=20 > i =3D 0; > p =3D buf; > if (size =3D=3D 0) > return (1); > len =3D p[0]; > p++; > if (len !=3D size - 1) > return (1); > if (strlen(name) !=3D len) > return (1); > return (strncmp(p, name, len)); > } > #endif >=20 > #ifdef __APPLE__ > static ssize_t > getea(int fd, const char *name, void *value, size_t size) > { >=20 > return (fgetxattr(fd, name, value, size, 0, 0)); > } >=20 > static int > setea(int fd, const char *name, const void *value, size_t size) > { >=20 > return (fsetxattr(fd, name, value, size, 0, 0)); > } >=20 > static int > delea(int fd, const char *name) > { >=20 > return (fremovexattr(fd, name, 0)); > } >=20 > static int > listea(int fd, void *buf, size_t size) > { >=20 > return (flistxattr(fd, buf, size, 0)); > } >=20 > static int > compare_ealist(const char *name, void *buf, size_t size) > { >=20 > if (size =3D=3D 0) > return (1); > if (strlen(name) !=3D size - 1) > return (1); > return (strncmp(buf, name, size)); > } > #endif >=20 > #define EA_NAME "foo" >=20 > char template[] =3D "/tmp/kevent_extattr.XXXXXX"; > static int fd, kq; >=20 > static void > check_ea(const char *value, const char *desc) > { > ssize_t retval; > size_t size; > char *buf; >=20 > retval =3D getea(fd, EA_NAME, NULL, 0); > if (retval < 0) { > if (errno =3D=3D ENOATTR) { > if (value !=3D NULL) > printf("Missing ea: %s\n", desc); > return; > } > err(1, "getea"); > } > size =3D (size_t)retval; > buf =3D malloc(size); > retval =3D getea(fd, EA_NAME, buf, size); > if (retval < 0) > err(1, "getea"); > if ((size_t)retval !=3D size) > errx(1, "getea: short read"); > if (value =3D=3D NULL) > printf("Unexpected ea value %*s: %s\n", (int)size, > buf, desc); > else if (strncmp(value, buf, size) !=3D 0) > printf("Mismatched ea values %*s vs %s: %s\n", = (int)size, > buf, value, desc); > free(buf); > } >=20 > static void > check_ealist(const char *name, const char *desc) > { > ssize_t retval; > size_t size; > char *buf; >=20 > retval =3D listea(fd, NULL, 0); > if (retval < 0) > err(1, "listea"); > size =3D (size_t)retval; > if (size =3D=3D 0) { > if (name !=3D NULL) { > printf("Missing ea: %s\n", desc); > return; > } > return; > } > buf =3D malloc(size); > retval =3D listea(fd, buf, size); > if (retval < 0) > err(1, "listea"); > if ((size_t)retval !=3D size) > errx(1, "listea: short read"); > if (name =3D=3D NULL) > printf("Unexpected ea: %s\n", desc); > else if (compare_ealist(name, buf, size) !=3D 0) > printf("Mismatched ea list: %s\n", desc); > free(buf); > } >=20 > static void > check_event(bool expected, const char *desc) > { > struct timespec ts =3D { 0, 0 }; > struct kevent ev; > int retval; >=20 > retval =3D kevent(kq, NULL, 0, &ev, 1, &ts); > if (retval < 0) > err(1, "kevent"); > if (!expected) { > if (retval !=3D 0) > printf("Unexpected kevent: %s\n", desc); > } else { > if (retval =3D=3D 0) > printf("Missing kevent: %s\n", desc); > else if (ev.fflags !=3D NOTE_ATTRIB) > printf("Wrong flags (%x vs %x): %s\n", > ev.fflags, NOTE_ATTRIB, desc); > } > } >=20 > int > main(int ac, char **av) > { > struct kevent ev; >=20 > kq =3D kqueue(); > if (kq < 0) > err(1, "kqueue"); > fd =3D mkstemp(template); > if (fd < 0) > err(1, "mkstemp"); > if (unlink(template) < 0) > err(1, "unlink"); > EV_SET(&ev, fd, EVFILT_VNODE, EV_ADD | EV_CLEAR, NOTE_ATTRIB, 0, = 0); > if (kevent(kq, &ev, 1, NULL, 0, NULL) < 0) > err(1, "kevent(EV_ADD)"); >=20 > check_event(false, "initial check"); > check_ea(NULL, "initial check"); > check_ealist(NULL, "initial check"); >=20 > if (setea(fd, EA_NAME, "bar", sizeof("bar")) < 0) > err(1, "setea(%s =3D \"bar\")", EA_NAME); > check_event(true, "first set"); >=20 > /* > * The check_ea() reads the EA and should not have triggered > * an event. > */ > check_ea("bar", "first set"); > check_event(false, "getea"); >=20 > /* > * The ea list requests in this check should not trigger an > * event either. > */ > check_ealist(EA_NAME, "first set"); > check_event(false, "listea"); >=20 > if (setea(fd, EA_NAME, "baz", sizeof("baz")) < 0) > err(1, "setea(%s =3D \"baz\")", EA_NAME); > check_event(true, "second set"); > check_ea("baz", "second set"); > check_ealist(EA_NAME, "second set"); >=20 > if (delea(fd, EA_NAME) < 0) > err(1, "delea(%s)", EA_NAME); > check_event(true, "remove"); > check_ea(NULL, "remove"); > check_ealist(NULL, "remove"); >=20 > close(fd); > close(kq); > return (0); > } >=20 > --=20 > John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Dec 23 14:23:18 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04754106566B; Fri, 23 Dec 2011 14:23:18 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id B55398FC19; Fri, 23 Dec 2011 14:23:17 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 6B35846B2A; Fri, 23 Dec 2011 09:23:17 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id F3EB9B955; Fri, 23 Dec 2011 09:23:16 -0500 (EST) From: John Baldwin To: "Robert N. M. Watson" Date: Fri, 23 Dec 2011 09:23:16 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <201112221105.31746.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112230923.16333.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 23 Dec 2011 09:23:17 -0500 (EST) Cc: arch@freebsd.org Subject: Re: Post NOTE_ATTRIB EVFILT_VNODE events for extattr changes X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Dec 2011 14:23:18 -0000 On Thursday, December 22, 2011 6:19:27 pm Robert N. M. Watson wrote: > > On 22 Dec 2011, at 16:05, John Baldwin wrote: > > > A co-worker noticed that there is currently no way to get an EVFILT_VNODE > > notification if the extended attributes for a given file change. I looked at > > OS X and found that it posts a NOTE_ATTRIB event (typically used for > > VOP_SETATTR()) for successful attempts to set or delete an extended attribute. > > I have a patch to do the same along with a test application for both OS X and > > FreeBSD. With this patch FreeBSD now has the same behavior as OS X. > > This seems reasonable to me, although I've not tested the patch you attached myself. > > I'm not familiar with the pre/post-vop hook mechanism, but I assume that this will fire when internal extended attribute I/O, for example triggered by ACL changes, occurs? The hooks fire anytime VOP_SETEXTATTR() or VOP_DELETEEXTATTR() are called. What happens is that calls to the hooks are inserted into the macro functions generated in vnode_if.c, for example: int VOP_SETEXTATTR_APV(struct vop_vector *vop, struct vop_setextattr_args *a) { int rc; VNASSERT(a->a_gen.a_desc == &vop_setextattr_desc, a->a_vp, ("Wrong a_desc in vop_setextattr(%p, %p)", a->a_vp, a)); while(vop != NULL && \ vop->vop_setextattr == NULL && vop->vop_bypass == NULL) vop = vop->vop_default; VNASSERT(vop != NULL, a->a_vp, ("No vop_setextattr(%p, %p)", a->a_vp, a)); SDT_PROBE(vfs, vop, vop_setextattr, entry, a->a_vp, a, 0, 0, 0); ASSERT_VI_UNLOCKED(a->a_vp, "VOP_SETEXTATTR"); ASSERT_VOP_ELOCKED(a->a_vp, "VOP_SETEXTATTR"); if (vop->vop_setextattr != NULL) rc = vop->vop_setextattr(a); else rc = vop->vop_bypass(&a->a_gen); CTR6(KTR_VOP, "VOP_SETEXTATTR(vp 0x%lX, attrnamespace %ld, name 0x%lX, uio 0x%lX, cred 0x%lX, td 0x%lX)", a->a_vp, a->a_attrnamespace, a->a_name, a->a_uio, a->a_cred, a->a_td); SDT_PROBE(vfs, vop, vop_setextattr, return, a->a_vp, a, rc, 0, 0); if (rc == 0) { ASSERT_VI_UNLOCKED(a->a_vp, "VOP_SETEXTATTR"); ASSERT_VOP_ELOCKED(a->a_vp, "VOP_SETEXTATTR"); } else { ASSERT_VI_UNLOCKED(a->a_vp, "VOP_SETEXTATTR"); ASSERT_VOP_ELOCKED(a->a_vp, "VOP_SETEXTATTR"); } vop_setextattr_post(a, rc); return (rc); } Thus, if the ACL code uses VOP_SETEXTATTR() and VOP_DELETEEXTATTR() then it will trigger this, yes. Checking UFS, it's ACL VOPs use vn_extattr_rm() and vn_extattr_set() which are wrappers around the the EA VOP's, so toggling ACL's will trigger NOTE_ATTRIB on UFS. I'm not sure if it is worth it to fire an explicit event for VOP_SETACL() as well. OS X does not appear to have a VOP_SETACL() that I can see (they just assume ACL's are always stored as EA's at the vnode layer), so I can't use that as a model. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Dec 23 20:20:19 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3676B10656B9 for ; Fri, 23 Dec 2011 20:20:01 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 9F4968FC0C for ; Fri, 23 Dec 2011 20:20:01 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 555F346B49 for ; Fri, 23 Dec 2011 15:20:01 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id D8494B93F for ; Fri, 23 Dec 2011 15:20:00 -0500 (EST) To: arch@freebsd.org From: John Baldwin Date: Fri, 23 Dec 2011 15:20:00 -0500 MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112231520.00282.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 23 Dec 2011 15:20:00 -0500 (EST) Cc: Subject: Teach KTR_SCHED to handle changing thread names X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Dec 2011 20:20:19 -0000 When I use the new schedgraph on 8, I find that it commonly calls almost all threads "sh" or "tcsh" because it only uses the name from the initial fork and never notices when a thread changes its name via exec. This makes traces harder to follow. The patch below adds a hook to clear the cached thread name to force it to be regenerated on the next trace when td_name changes. This makes the traces more usable for me at least. Index: kern/sched_ule.c =================================================================== --- kern/sched_ule.c (revision 225431) +++ kern/sched_ule.c (working copy) @@ -2685,6 +2685,17 @@ #endif } +#ifdef KTR +void +sched_clear_tdname(struct thread *td) +{ + struct td_sched *ts; + + ts = td->td_sched; + ts->ts_name[0] = '\0'; +} +#endif + #ifdef SMP /* Index: kern/kern_thr.c =================================================================== --- kern/kern_thr.c (revision 225431) +++ kern/kern_thr.c (working copy) @@ -532,9 +532,12 @@ ttd = td; else ttd = thread_find(p, uap->id); - if (ttd != NULL) + if (ttd != NULL) { strcpy(ttd->td_name, name); - else +#ifdef KTR + sched_clear_tdname(ttd); +#endif + } else error = ESRCH; PROC_UNLOCK(p); return (error); Index: kern/kern_kthread.c =================================================================== --- kern/kern_kthread.c (revision 225431) +++ kern/kern_kthread.c (working copy) @@ -407,6 +407,9 @@ va_start(ap, fmt); vsnprintf(td->td_name, sizeof(td->td_name), fmt, ap); va_end(ap); +#ifdef KTR + sched_clear_tdname(td); +#endif return (0); } va_start(ap, fmt); Index: kern/sched_4bsd.c =================================================================== --- kern/sched_4bsd.c (revision 225431) +++ kern/sched_4bsd.c (working copy) @@ -1611,7 +1611,18 @@ #endif } +#ifdef KTR void +sched_clear_tdname(struct thread *td) +{ + struct td_sched *ts; + + ts = td->td_sched; + ts->ts_name[0] = '\0'; +} +#endif + +void sched_affinity(struct thread *td) { #ifdef SMP Index: kern/kern_exec.c =================================================================== --- kern/kern_exec.c (revision 225431) +++ kern/kern_exec.c (working copy) @@ -54,6 +54,7 @@ #include #include #include +#include #include #include #include @@ -609,6 +610,9 @@ else if (vn_commname(binvp, p->p_comm, sizeof(p->p_comm)) != 0) bcopy(fexecv_proc_title, p->p_comm, sizeof(fexecv_proc_title)); bcopy(p->p_comm, td->td_name, sizeof(td->td_name)); +#ifdef KTR + sched_clear_tdname(td); +#endif /* * mark as execed, wakeup the process that vforked (if any) and tell Index: sys/sched.h =================================================================== --- sys/sched.h (revision 225431) +++ sys/sched.h (working copy) @@ -139,6 +139,9 @@ * functions. */ char *sched_tdname(struct thread *td); +#ifdef KTR +void sched_clear_tdname(struct thread *td); +#endif static __inline void sched_pin(void) -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Dec 23 22:35:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BEBF106564A for ; Fri, 23 Dec 2011 22:35:59 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 070198FC1D for ; Fri, 23 Dec 2011 22:35:58 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so13418550vcb.13 for ; Fri, 23 Dec 2011 14:35:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=5kbRfic3YPik8+lRwL6NAhcKMawSN6H6j7gdZq3XRvg=; b=IqHUQBaIA/6p1Fy4SPlA9yFFBZalyg1GvqS52Q2YNF5SraF28SfPCnumAszWYRWRpJ etGzPdrf4yqN4uS0UhBxBLJ00/QBA7U/RLkrDY23bTx8fzpIrTfgGcIzTUcoEm9LEwG+ 5fyFIuJ5o2FWTmTWdoypH/W0FGj411NAJNpXk= MIME-Version: 1.0 Received: by 10.220.149.193 with SMTP id u1mr10075091vcv.33.1324677950618; Fri, 23 Dec 2011 14:05:50 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.36.5 with HTTP; Fri, 23 Dec 2011 14:05:50 -0800 (PST) Date: Fri, 23 Dec 2011 14:05:50 -0800 X-Google-Sender-Auth: cYRyppF_HtoxpmOLnGLGtV5bZ4s Message-ID: From: Adrian Chadd To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current Subject: [patch] allow local-tools to include local directories X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Dec 2011 22:35:59 -0000 Hi, This patch allows for user-defined extra local tools directories, a la what LOCAL_DIRS does for adding local build directories to the source. This is needed for cross-compiling as some local tools may need to be first built. I've used this successfully to cross-compile my busybox stuff. Thanks, Adrian -- [adrian@pcbsd-macvm] ~/work/freebsd/head/src> svn diff Makefile.inc1 Index: Makefile.inc1 =================================================================== --- Makefile.inc1 (revision 228757) +++ Makefile.inc1 (working copy) @@ -15,6 +15,8 @@ # -DNO_WWWUPDATE do not update www in ${MAKE} update # -DNO_CTF do not run the DTrace CTF conversion tools on built objects # LOCAL_DIRS="list of dirs" to add additional dirs to the SUBDIR list +# LOCAL_TOOL_DIRS="list of dirs" to add additional dirs to the build-tools +# list # TARGET="machine" to crossbuild world for a different machine type # TARGET_ARCH= may be required when a TARGET supports multiple endians @@ -104,6 +106,8 @@ CLEANDIR= cleandir .endif +LOCAL_TOOL_DIRS?= '' + CVS?= cvs CVSFLAGS?= -A -P -d -I! SVN?= svn @@ -1102,6 +1106,7 @@ bin/csh \ bin/sh \ ${_rescue} \ + ${LOCAL_TOOL_DIRS} \ lib/ncurses/ncurses \ lib/ncurses/ncursesw \ ${_share} \ From owner-freebsd-arch@FreeBSD.ORG Fri Dec 23 22:58:03 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F29D1065678; Fri, 23 Dec 2011 22:58:03 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 190678FC16; Fri, 23 Dec 2011 22:58:02 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id pBNMUIQb026087 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 23 Dec 2011 14:30:20 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4EF5011A.70709@freebsd.org> Date: Fri, 23 Dec 2011 14:30:50 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.25) Gecko/20111213 Thunderbird/3.1.17 MIME-Version: 1.0 To: John Baldwin References: <201112231520.00282.jhb@freebsd.org> In-Reply-To: <201112231520.00282.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: Teach KTR_SCHED to handle changing thread names X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Dec 2011 22:58:03 -0000 On 12/23/11 12:20 PM, John Baldwin wrote: > When I use the new schedgraph on 8, I find that it commonly calls almost all > threads "sh" or "tcsh" because it only uses the name from the initial fork and > never notices when a thread changes its name via exec. This makes traces > harder to follow. The patch below adds a hook to clear the cached thread name > to force it to be regenerated on the next trace when td_name changes. This > makes the traces more usable for me at least. > > I have no problems with this.. hard to decipher debugging as as bad as no debugging. From owner-freebsd-arch@FreeBSD.ORG Fri Dec 23 23:32:17 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51505106566B; Fri, 23 Dec 2011 23:32:17 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id EE5F48FC13; Fri, 23 Dec 2011 23:32:16 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so13449987vcb.13 for ; Fri, 23 Dec 2011 15:32:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=C07BhfJKaqM9enRZCyLKg+CZg4m36GhSSvkcXBoikqE=; b=ola8iJjHDLezSRJEFTXhoozbfq59BWN5NBFEBoIAxxc0WwswauI+JKDYONFeQOx1Qj kkaX0CPpOUyLCs0l61jlzXADCYMmTLwSofY7U/hZgEbjbRcX45WtJf1mPuEl426oHoyv aa0tq2YPOgjXdNR9oUon3Ozg127a3YAi0uuAQ= MIME-Version: 1.0 Received: by 10.52.23.44 with SMTP id j12mr2232164vdf.117.1324681274642; Fri, 23 Dec 2011 15:01:14 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.36.5 with HTTP; Fri, 23 Dec 2011 15:01:14 -0800 (PST) In-Reply-To: <4EF5011A.70709@freebsd.org> References: <201112231520.00282.jhb@freebsd.org> <4EF5011A.70709@freebsd.org> Date: Fri, 23 Dec 2011 15:01:14 -0800 X-Google-Sender-Auth: FfEESVJtGJ0BmfdhT5vdiY9cPOM Message-ID: From: Adrian Chadd To: Julian Elischer Content-Type: text/plain; charset=ISO-8859-1 Cc: arch@freebsd.org Subject: Re: Teach KTR_SCHED to handle changing thread names X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Dec 2011 23:32:17 -0000 +1, I hated this when trying to use it for userland. thankfully I only seem to need it for kernel-land.. adrian From owner-freebsd-arch@FreeBSD.ORG Fri Dec 23 23:56:42 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 1368E1065673; Fri, 23 Dec 2011 23:56:42 +0000 (UTC) Date: Fri, 23 Dec 2011 23:56:42 +0000 From: Alexander Best To: freebsd-current@freebsd.org Message-ID: <20111223235642.GA37495@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Cc: freebsd-arch@freebsd.org Subject: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Dec 2011 23:56:42 -0000 hi there, is -mpreferred-stack-boundary=2 really necessary for i386 builds any longer? i built GENERIC (including modules) with and without that flag. the results are: 1654496 bytes with the flag set vs. 1654952 bytes with the flag unset the gcc(1) man page states the following: " This extra alignment does consume extra stack space, and generally increases code size. Code that is sensitive to stack space usage, such as embedded systems and operating system kernels, may want to reduce the preferred alignment to -mpreferred-stack-boundary=2. " the comment in sys/conf/kern.mk however sorta suggests that the default alignment of 4 bytes might improve performance. cheers. alex From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 00:42:08 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 527751065701; Sat, 24 Dec 2011 00:42:08 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id EB65B8FC14; Sat, 24 Dec 2011 00:42:07 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so13484284vcb.13 for ; Fri, 23 Dec 2011 16:42:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=/T0DQIOb3/vFZwbHb4aOFgoDkLgT/Y3X4Zujh79YIX8=; b=aZZ5JMvh6hOQy4jIaRMBUxAiMVKDvZuqZNgJ44syGHSuk4FLhoxL4jRkKuq5R1Liff oEjCWNQEenov441kDDan6aVBHEx00nJ/gRX2HI13mShcIspVw7MfxQPVMmTy+P/5fuRP 7t2p9PV59VqdnyUpo8QlmtdBW2yZ60PCcPzko= MIME-Version: 1.0 Received: by 10.220.149.193 with SMTP id u1mr10261983vcv.33.1324687326815; Fri, 23 Dec 2011 16:42:06 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.36.5 with HTTP; Fri, 23 Dec 2011 16:42:06 -0800 (PST) Date: Fri, 23 Dec 2011 16:42:06 -0800 X-Google-Sender-Auth: 4sxQOnSIjjKDM-JjmY2PSRt8OBs Message-ID: From: Adrian Chadd To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current Subject: [patch] bsdbox changes for base system: add LOCAL_ X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 00:42:08 -0000 Hi, Here are two patches which implement some useful features for crunch building: * Add LOCAL_TOOLS_DIR in src/Makefile.inc1, which adds entries to the 'build-tools' target. This is needed for cross-building bsdbox (and any external directory added by LOCAL_DIRS) * If CRUNCH_SUPPRESS_ALL_LINKS is set to 'yes', don't auto-populate the crunch-gen hard links. I may end up changing the latter to CRUNCH_GENERATE_LINKS and default that to yes, then allow the bsdbox build system to change that to 'no', but the intent is the same. I'd like to commit this tomorrow if possible, so I can then follow it up with the bsdbox import. Thanks, Adrian [adrian@pcbsd-macvm] ~/work/freebsd/head/src> svn diff Makefile.inc1 share/mk Index: Makefile.inc1 =================================================================== --- Makefile.inc1 (revision 228757) +++ Makefile.inc1 (working copy) @@ -15,6 +15,8 @@ # -DNO_WWWUPDATE do not update www in ${MAKE} update # -DNO_CTF do not run the DTrace CTF conversion tools on built objects # LOCAL_DIRS="list of dirs" to add additional dirs to the SUBDIR list +# LOCAL_TOOL_DIRS="list of dirs" to add additional dirs to the build-tools +# list # TARGET="machine" to crossbuild world for a different machine type # TARGET_ARCH= may be required when a TARGET supports multiple endians @@ -104,6 +106,8 @@ CLEANDIR= cleandir .endif +LOCAL_TOOL_DIRS?= '' + CVS?= cvs CVSFLAGS?= -A -P -d -I! SVN?= svn @@ -1102,6 +1106,7 @@ bin/csh \ bin/sh \ ${_rescue} \ + ${LOCAL_TOOL_DIRS} \ lib/ncurses/ncurses \ lib/ncurses/ncursesw \ ${_share} \ Index: share/mk/bsd.crunchgen.mk =================================================================== --- share/mk/bsd.crunchgen.mk (revision 228757) +++ share/mk/bsd.crunchgen.mk (working copy) @@ -22,6 +22,8 @@ # Specific links can be suppressed by setting # CRUNCH_SUPPRESS_LINK_$(NAME) to 1. # +# If CRUNCH_SUPPRESS_ALL_LINKS is set to yes, no links will be generated. +# # $FreeBSD$ @@ -39,6 +41,7 @@ .else CANONICALOBJDIR:= /usr/obj${.CURDIR} .endif +CRUNCH_SUPPRESS_ALL_LINKS?= no CLEANFILES+= $(CONF) *.o *.lo *.c *.mk *.cache *.a *.h @@ -51,6 +54,7 @@ .else $(OUTPUTS): $(.CURDIR)/../../$(D)/$(P)/Makefile .endif +.if ${CRUNCH_SUPPRESS_ALL_LINKS} != "yes" .ifndef CRUNCH_SUPPRESS_LINK_${P} LINKS+= $(BINDIR)/$(PROG) $(BINDIR)/$(P) .endif @@ -59,6 +63,7 @@ LINKS+= $(BINDIR)/$(PROG) $(BINDIR)/$(A) .endif .endfor +.endif .endfor .endfor From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 06:16:44 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C737106564A; Sat, 24 Dec 2011 06:16:44 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by mx1.freebsd.org (Postfix) with ESMTP id DDB1C8FC15; Sat, 24 Dec 2011 06:16:43 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail05.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id pBO6GXLX018222 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 24 Dec 2011 17:16:42 +1100 Date: Sat, 24 Dec 2011 17:16:33 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Alexander Best In-Reply-To: <20111223235642.GA37495@freebsd.org> Message-ID: <20111224160050.T1141@besplex.bde.org> References: <20111223235642.GA37495@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 06:16:44 -0000 On Fri, 23 Dec 2011, Alexander Best wrote: > is -mpreferred-stack-boundary=2 really necessary for i386 builds any longer? > i built GENERIC (including modules) with and without that flag. the results > are: The same as it has always been. It avoids some bloat. > 1654496 bytes with the flag set > vs. > 1654952 bytes with the flag unset I don't believe this. GENERIC is enormously bloated, so it has size more like 16MB than 1.6MB. Even a savings of 4K instead of 456 bytes is hard to believe. I get a savings of 9K (text) in a 5MB kernel. Changing the default target arch from i386 to pentium-undocumented has reduced the text space savings a little, since the default for passing args is now to preallocate stack space for them and store to this, instead of to push them; this preallocation results in more functions needing to allocate some stack space explicitly, and when some is allocated explicitly, the text space cost for this doesn't depend on the size of the allocation. Anyway, the savings are mostly from from avoiding cache misses from sparse allocation on stacks. Also, FreeBSD-i386 hasn't been programmed to support aligned stacks: - KSTACK_PAGES on i386 is 2, while on amd64 it is 4. Using more stack might push something over the edge - not much care is taken to align the initial stack or to keep the stack aligned in calls from asm code. E.g., any alignment for mi_startup() (and thus proc0?) is accidental. This may result in perfect alignment or perfect misalignment. Hopefully, more care is taken with thread startup. For gcc, the alignment is done bogusly in main() in userland, but there is no main() in the kernel. The alignment doesn't matter much (provided the perfect misalignment is still to a multiple of 4), but when it matters, the random misalignment that results from not trying to do it at all is better than perfect misalignment from getting it wrong. With 4-byte alignment, the only cases that it helps are with 64-bit variables. > the gcc(1) man page states the following: > > " > This extra alignment does consume extra stack space, and generally > increases code size. Code that is sensitive to stack space usage, > such as embedded systems and operating system kernels, may want to > reduce the preferred alignment to -mpreferred-stack-boundary=2. > " > > the comment in sys/conf/kern.mk however sorta suggests that the default > alignment of 4 bytes might improve performance. The default stack alignment is 16 bytes, which unimproves performance. clang handles stack alignment correctly (only does it when it is needed) so it doesn't need a -mpreferred-stack-boundary option and doesn't always break without alignment in main(). Well, at least it used to, IIRC. Testing it now shows that it does the necessary andl of the stack pointer for __aligned(32), but for __aligned(16) it now assumes that the stack is aligned by the caller. So it now needs -mpreferred-stack-boundary=2, but doesn't have it. OTOH, clang doesn't do the andl in main() like gcc does (unless you put a dummy __aligned(32) there), but requires crt to pass an aligned stack. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 06:52:53 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9056F106564A; Sat, 24 Dec 2011 06:52:53 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 205CB8FC13; Sat, 24 Dec 2011 06:52:52 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so13649587vcb.13 for ; Fri, 23 Dec 2011 22:52:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=62VFJpXrdVU5weJF6lonc7VFHNOncfUL0dvxMqCZKG0=; b=HMPiZbbqMoVqjeTIzLJR3iOA5LwfveqCCGi9Rl9W/nsB8XnaIrjedjrjujR6xAmbdS 26h+kpPeRzDaTCzFfoJps6/D+8mY72O1OWaIhIbSSecWMFjzit+E/t7QQCR3G2bwrRX+ ICjnbJKmYBqlCIk99vcRDZrlkuMrCN2nEb0d4= MIME-Version: 1.0 Received: by 10.220.149.193 with SMTP id u1mr10717816vcv.33.1324709572425; Fri, 23 Dec 2011 22:52:52 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.36.5 with HTTP; Fri, 23 Dec 2011 22:52:52 -0800 (PST) In-Reply-To: <20111224160050.T1141@besplex.bde.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> Date: Fri, 23 Dec 2011 22:52:52 -0800 X-Google-Sender-Auth: Mc4SE_4-8xBwTCJKGGE0EyfcZv0 Message-ID: From: Adrian Chadd To: Bruce Evans Content-Type: text/plain; charset=ISO-8859-1 Cc: Alexander Best , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 06:52:53 -0000 Well, the whole kernel is bloated at the moment, sorry. I've been trying to build the _bare minimum_ required to bootstrap -HEAD on these embedded boards and I can't get the kernel down below 5 megabytes - ie, one with FFS (with options disabled), MIPS, INET (no INET6), net80211, ath (which admittedly is big, but I need it no matter what, right?) comes in at: -r-xr-xr-x 1 root wheel 5307021 Nov 29 19:14 kernel.LSSR71 And with INET6, on another board (and this includes MSDOS and the relevant geom modules): -r-xr-xr-x 1 root wheel 5916759 Nov 28 12:00 kernel.RSPRO .. honestly, that's what should be addressed. That's honestly a bit ridiculous. 2c, Adrian From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 09:12:50 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id DD28A1065672; Sat, 24 Dec 2011 09:12:50 +0000 (UTC) Date: Sat, 24 Dec 2011 09:12:50 +0000 From: Alexander Best To: Bruce Evans Message-ID: <20111224091250.GA9921@freebsd.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111224160050.T1141@besplex.bde.org> Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 09:12:51 -0000 On Sat Dec 24 11, Bruce Evans wrote: > On Fri, 23 Dec 2011, Alexander Best wrote: > > >is -mpreferred-stack-boundary=2 really necessary for i386 builds any > >longer? > >i built GENERIC (including modules) with and without that flag. the results > >are: > > The same as it has always been. It avoids some bloat. > > >1654496 bytes with the flag set > >vs. > >1654952 bytes with the flag unset > > I don't believe this. GENERIC is enormously bloated, so it has size > more like 16MB than 1.6MB. Even a savings of 4K instead of 456 bytes i'm sorry. i used du(1) to get those numbers, so i believe those numbers represent the ammount of 512-byte blocks. if i'm correct GENERIC is even more bloated than you feared and almost reaches 1GB: 807,859375 megabytes with flag set vs. 808,0820313 megabytes without the flag set > is hard to believe. I get a savings of 9K (text) in a 5MB kernel. > Changing the default target arch from i386 to pentium-undocumented has > reduced the text space savings a little, since the default for passing > args is now to preallocate stack space for them and store to this, > instead of to push them; this preallocation results in more functions > needing to allocate some stack space explicitly, and when some is > allocated explicitly, the text space cost for this doesn't depend on > the size of the allocation. > > Anyway, the savings are mostly from from avoiding cache misses from > sparse allocation on stacks. > > Also, FreeBSD-i386 hasn't been programmed to support aligned stacks: > - KSTACK_PAGES on i386 is 2, while on amd64 it is 4. Using more > stack might push something over the edge > - not much care is taken to align the initial stack or to keep the > stack aligned in calls from asm code. E.g., any alignment for > mi_startup() (and thus proc0?) is accidental. This may result > in perfect alignment or perfect misalignment. Hopefully, more > care is taken with thread startup. For gcc, the alignment is > done bogusly in main() in userland, but there is no main() in > the kernel. The alignment doesn't matter much (provided the > perfect misalignment is still to a multiple of 4), but when it > matters, the random misalignment that results from not trying to > do it at all is better than perfect misalignment from getting it > wrong. With 4-byte alignment, the only cases that it helps are > with 64-bit variables. > > >the gcc(1) man page states the following: > > > >" > >This extra alignment does consume extra stack space, and generally > >increases code size. Code that is sensitive to stack space usage, > >such as embedded systems and operating system kernels, may want to > >reduce the preferred alignment to -mpreferred-stack-boundary=2. > >" > > > >the comment in sys/conf/kern.mk however sorta suggests that the default > >alignment of 4 bytes might improve performance. > > The default stack alignment is 16 bytes, which unimproves performance. > > clang handles stack alignment correctly (only does it when it is needed) > so it doesn't need a -mpreferred-stack-boundary option and doesn't > always break without alignment in main(). Well, at least it used to, > IIRC. Testing it now shows that it does the necessary andl of the > stack pointer for __aligned(32), but for __aligned(16) it now assumes > that the stack is aligned by the caller. So it now needs > -mpreferred-stack-boundary=2, but doesn't have it. OTOH, clang doesn't > do the andl in main() like gcc does (unless you put a dummy __aligned(32) > there), but requires crt to pass an aligned stack. > > Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 09:37:53 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id C08091065670; Sat, 24 Dec 2011 09:37:53 +0000 (UTC) Date: Sat, 24 Dec 2011 09:37:53 +0000 From: Alexander Best To: Bruce Evans Message-ID: <20111224093753.GA12377@freebsd.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Kj7319i9nmIyA2yE" Content-Disposition: inline In-Reply-To: <20111224160050.T1141@besplex.bde.org> Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 09:37:53 -0000 --Kj7319i9nmIyA2yE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sat Dec 24 11, Bruce Evans wrote: > On Fri, 23 Dec 2011, Alexander Best wrote: > > >is -mpreferred-stack-boundary=2 really necessary for i386 builds any > >longer? > >i built GENERIC (including modules) with and without that flag. the results > >are: > > The same as it has always been. It avoids some bloat. > > >1654496 bytes with the flag set > >vs. > >1654952 bytes with the flag unset > > I don't believe this. GENERIC is enormously bloated, so it has size > more like 16MB than 1.6MB. Even a savings of 4K instead of 456 bytes > is hard to believe. I get a savings of 9K (text) in a 5MB kernel. > Changing the default target arch from i386 to pentium-undocumented has > reduced the text space savings a little, since the default for passing > args is now to preallocate stack space for them and store to this, > instead of to push them; this preallocation results in more functions > needing to allocate some stack space explicitly, and when some is > allocated explicitly, the text space cost for this doesn't depend on > the size of the allocation. > > Anyway, the savings are mostly from from avoiding cache misses from > sparse allocation on stacks. > > Also, FreeBSD-i386 hasn't been programmed to support aligned stacks: > - KSTACK_PAGES on i386 is 2, while on amd64 it is 4. Using more > stack might push something over the edge > - not much care is taken to align the initial stack or to keep the > stack aligned in calls from asm code. E.g., any alignment for > mi_startup() (and thus proc0?) is accidental. This may result > in perfect alignment or perfect misalignment. Hopefully, more > care is taken with thread startup. For gcc, the alignment is > done bogusly in main() in userland, but there is no main() in > the kernel. The alignment doesn't matter much (provided the > perfect misalignment is still to a multiple of 4), but when it > matters, the random misalignment that results from not trying to > do it at all is better than perfect misalignment from getting it > wrong. With 4-byte alignment, the only cases that it helps are > with 64-bit variables. > > >the gcc(1) man page states the following: > > > >" > >This extra alignment does consume extra stack space, and generally > >increases code size. Code that is sensitive to stack space usage, > >such as embedded systems and operating system kernels, may want to > >reduce the preferred alignment to -mpreferred-stack-boundary=2. > >" > > > >the comment in sys/conf/kern.mk however sorta suggests that the default > >alignment of 4 bytes might improve performance. > > The default stack alignment is 16 bytes, which unimproves performance. maybe the part of the comment in sys/conf/kern.mk, which mentions that a stack alignment of 16 bytes might improve micro benchmark results should be removed. this would prevent people (like me) from thinking, using a stack alignment of 4 bytes is a compromise between size and efficiently. it isn't! currently a stack alignment of 16 bytes has no advantages towards one with 4 bytes on i386. so specifying -mpreferred-stack-boundary=2 on i386 is absolutely mandatory. please see the attached patch, which also introduduces a line break in order to describe the stack alignment issue in a paragraph of its own. cheers. alex > > clang handles stack alignment correctly (only does it when it is needed) > so it doesn't need a -mpreferred-stack-boundary option and doesn't > always break without alignment in main(). Well, at least it used to, > IIRC. Testing it now shows that it does the necessary andl of the > stack pointer for __aligned(32), but for __aligned(16) it now assumes > that the stack is aligned by the caller. So it now needs > -mpreferred-stack-boundary=2, but doesn't have it. OTOH, clang doesn't > do the andl in main() like gcc does (unless you put a dummy __aligned(32) > there), but requires crt to pass an aligned stack. > > Bruce --Kj7319i9nmIyA2yE Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="kern.mk.diff" Index: /usr/src/sys/conf/kern.mk =================================================================== --- /usr/src/sys/conf/kern.mk (revision 228845) +++ /usr/src/sys/conf/kern.mk (working copy) @@ -30,12 +30,12 @@ # On i386, do not align the stack to 16-byte boundaries. Otherwise GCC 2.95 # and above adds code to the entry and exit point of every function to align the # stack to 16-byte boundaries -- thus wasting approximately 12 bytes of stack -# per function call. While the 16-byte alignment may benefit micro benchmarks, -# it is probably an overall loss as it makes the code bigger (less efficient -# use of code cache tag lines) and uses more stack (less efficient use of data -# cache tag lines). Explicitly prohibit the use of FPU, SSE and other SIMD -# operations inside the kernel itself. These operations are exclusively -# reserved for user applications. +# per function call. This makes the code bigger (less efficient use of code +# cache tag lines) and uses more stack (less efficient use of data cache tag +# lines). +# Explicitly prohibit the use of FPU, SSE and other SIMD operations inside the +# kernel itself. These operations are exclusively reserved for user +# applications. # # gcc: # Setting -mno-mmx implies -mno-3dnow --Kj7319i9nmIyA2yE-- From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 11:06:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C0B11065678; Sat, 24 Dec 2011 11:06:59 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail13.syd.optusnet.com.au (mail13.syd.optusnet.com.au [211.29.132.194]) by mx1.freebsd.org (Postfix) with ESMTP id C4FD98FC14; Sat, 24 Dec 2011 11:06:58 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail13.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id pBOB6sTv019496 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 24 Dec 2011 22:06:56 +1100 Date: Sat, 24 Dec 2011 22:06:54 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Adrian Chadd In-Reply-To: Message-ID: <20111224211903.I2059@besplex.bde.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Alexander Best , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 11:06:59 -0000 On Fri, 23 Dec 2011, Adrian Chadd wrote: > Well, the whole kernel is bloated at the moment, sorry. > > I've been trying to build the _bare minimum_ required to bootstrap > -HEAD on these embedded boards and I can't get the kernel down below 5 > megabytes - ie, one with FFS (with options disabled), MIPS, INET (no > INET6), net80211, ath (which admittedly is big, but I need it no > matter what, right?) comes in at: > > -r-xr-xr-x 1 root wheel 5307021 Nov 29 19:14 kernel.LSSR71 > > And with INET6, on another board (and this includes MSDOS and the > relevant geom modules): > > -r-xr-xr-x 1 root wheel 5916759 Nov 28 12:00 kernel.RSPRO > > .. honestly, that's what should be addressed. That's honestly a bit ridiculous. It's disgusting, but what problems does it cause apart from minor slowness from cache misses? I used to monitor the size of a minimal i386 kernel: % machine i386 % cpu I686_CPU % ident MIN % options SCHED_4BSD In FreeBSD-5-CURRENT between 5.1R and 5.2R, this had size: text data bss dec hex filename 931241 86524 62356 1080121 107b39 /sysc/i386/compile/min/kernel A minimal kernel is not useful, but maybe you can add some i/o to it without bloating it too much. This almost builds in -current too. I had to add the following: - NO_MODULES to de-bloat the compile time - MK_CTF=no to build -current on FreeBSD.9. The kernel .mk files are still broken (depend on nonstandard/new features in sys.mk). - comment out a line in if.c that refers to Vloif. if.c is standard but the loop device is optional. A few more changes to remove non-minimalities that are not defaults made little difference: % machine i386 % cpu I686_CPU % ident MIN % options SCHED_4BSD % % # XXX kill default misconfigurations. % makeoptions NO_MODULES=yes % makeoptions COPTFLAGS="-O -pipe" % % # XXX from here on is to try to kill everything in DEFAULTS. % % # nodevice isa # needed for DELAY... % # nooptions ISAPNP # needed ... % % nodevice npx % % nodevice mem % nodevice io % % nodevice uart_ns8250 % % nooptions GEOM_PART_BSD % nooptions GEOM_PART_EBR % nooptions GEOM_PART_EBR_COMPAT % nooptions GEOM_PART_MBR % % # nooptions NATIVE # needed ... % # nodevice atpic # needed ... % % nooptions NEW_PCIB % % nooptions VFS_ALLOW_NONMPSAFE text data bss dec hex filename 1663902 110632 136892 1911426 1d2a82 kernel (This was about 100K larger with -O2 and all DEFAULTS). The bloat since FreeBSD-5 is only 70%. Here are some sizes for my standard kernel (on i386). The newer versions have about the same number of features since they don't support so many old isa devices or so many NICs: text data bss dec hex filename 1483269 106972 172524 1762765 1ae5cd FreeBSD-3/kernel 1917408 157472 194228 2269108 229fb4 FreeBSD-4/kernel 2604498 198948 237720 3041166 2e678e FreeBSD-5.1.5/kernel 2833842 206856 242936 3283634 321ab2 FreeBSD-5.1.5/kernel-with-acpi 2887573 192456 288696 3368725 336715 FreeBSD-5.1.5/kernel with my changes, -O2 and usb added relative to the above 2582782 195756 298936 3077474 2ef562 previous, with some excessive inlining avoided, and without -O2, and with ipfilter 1998276 159436 137748 2295460 2306a4 kernel.4 a more up to date and less hacked on FreeBSD-4 4365549 262656 209588 4837793 49d1a1 kernel.7 4406155 266496 496532 5169183 4ee01f kernel.7.invariants 3953248 242464 207252 4402964 432f14 kernel.7.noacpi 4418063 268288 240084 4926435 4b2be3 kernel.7.smp various fairly stock FreeBSD-7R kernels 3669544 262848 249712 4182104 3fd058 kernel.c 4174317 258240 540144 4972701 4be09d kernel.c.invariants 3964455 250656 249808 4464919 442117 kernel.c.noacpi 3213928 240160 240596 3694684 38605c kernel.c.noacpi-ule 4285040 268288 286160 4839488 49d840 kernel.c.smp current before FreeBSD-8R not all built at the same time or with the same options. The 20% bloat between kernel.c.noacpi.ule and kernel.c.noacpi is mainly from not killing the default of -O2. 4742714 315008 401692 5459414 534dd6 kernel.8 4816900 319200 1813916 6950016 6a0c80 kernel.8.invariants 4490209 304832 395260 5190301 4f329d kernel.8.noacpi 4795475 323680 475420 5594575 555dcf kernel.8.smp various fairly stock FreeBSD-8R kernels 4979632 287020 488404 5755056 57d0b0 kernel.cur 5062953 289196 1902676 7254825 6eb329 kernel.cur.invariants 5361809 295052 576984 6233845 5f1ef5 kernel.cur.smp amd64 kernels are only bout 4% larger if i386 kernels are built with equivalant CFLAGS (-march=athlon64 for athlon64 CPU), but I usually optimize i386 kernels for space and portability by compiling them with -mtune=old. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 11:12:44 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFAAD106564A; Sat, 24 Dec 2011 11:12:44 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail07.syd.optusnet.com.au (mail07.syd.optusnet.com.au [211.29.132.188]) by mx1.freebsd.org (Postfix) with ESMTP id 6C5B38FC12; Sat, 24 Dec 2011 11:12:44 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail07.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id pBOBCghY028087 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 24 Dec 2011 22:12:42 +1100 Date: Sat, 24 Dec 2011 22:12:42 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Alexander Best In-Reply-To: <20111224091250.GA9921@freebsd.org> Message-ID: <20111224220705.H2059@besplex.bde.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> <20111224091250.GA9921@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 11:12:45 -0000 On Sat, 24 Dec 2011, Alexander Best wrote: > On Sat Dec 24 11, Bruce Evans wrote: >> On Fri, 23 Dec 2011, Alexander Best wrote: >> >>> is -mpreferred-stack-boundary=2 really necessary for i386 builds any >>> longer? >>> i built GENERIC (including modules) with and without that flag. the results >>> are: >> >> The same as it has always been. It avoids some bloat. >> >>> 1654496 bytes with the flag set >>> vs. >>> 1654952 bytes with the flag unset >> >> I don't believe this. GENERIC is enormously bloated, so it has size >> more like 16MB than 1.6MB. Even a savings of 4K instead of 456 bytes > > i'm sorry. i used du(1) to get those numbers, so i believe those numbers > represent the ammount of 512-byte blocks. if i'm correct GENERIC is even > more bloated than you feared and almost reaches 1GB: > > 807,859375 megabytes with flag set > vs. > 808,0820313 megabytes without the flag set That's certainly bloated. It counts all object files and modules, and probably everything is compiled with -g. I only counted kernel text size. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 11:20:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EC85106566C; Sat, 24 Dec 2011 11:20:09 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id EA31C8FC0A; Sat, 24 Dec 2011 11:20:08 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 988FB10E003; Sat, 24 Dec 2011 12:20:07 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20111224211903.I2059@besplex.bde.org> Date: Sat, 24 Dec 2011 12:20:06 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <8883B1C2-738A-4C72-B977-AD610B33982B@lassitu.de> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> <20111224211903.I2059@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.1251.1) Cc: Alexander Best , Adrian Chadd , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 11:20:09 -0000 Am 24.12.2011 um 12:06 schrieb Bruce Evans: > On Fri, 23 Dec 2011, Adrian Chadd wrote: >=20 >> Well, the whole kernel is bloated at the moment, sorry. >>=20 >> I've been trying to build the _bare minimum_ required to bootstrap >> -HEAD on these embedded boards and I can't get the kernel down below = 5 >> megabytes - ie, one with FFS (with options disabled), MIPS, INET (no >> INET6), net80211, ath (which admittedly is big, but I need it no >> matter what, right?) comes in at: >>=20 >> -r-xr-xr-x 1 root wheel 5307021 Nov 29 19:14 kernel.LSSR71 >>=20 >> And with INET6, on another board (and this includes MSDOS and the >> relevant geom modules): >>=20 >> -r-xr-xr-x 1 root wheel 5916759 Nov 28 12:00 kernel.RSPRO >>=20 >> .. honestly, that's what should be addressed. That's honestly a bit = ridiculous. >=20 > It's disgusting, but what problems does it cause apart from minor = slowness > from cache misses? The flash chip on these devices only has 8MB; some of the really cheap = ones only have 4MB (yes MB, not GB). And many have only 32MB RAM. It = would be nice to have space for actual applications :-) Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 11:27:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 1BB4D1065670; Sat, 24 Dec 2011 11:27:59 +0000 (UTC) Date: Sat, 24 Dec 2011 11:27:59 +0000 From: Alexander Best To: Bruce Evans Message-ID: <20111224112759.GA25716@freebsd.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> <20111224091250.GA9921@freebsd.org> <20111224220705.H2059@besplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111224220705.H2059@besplex.bde.org> Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 11:27:59 -0000 On Sat Dec 24 11, Bruce Evans wrote: > On Sat, 24 Dec 2011, Alexander Best wrote: > > >On Sat Dec 24 11, Bruce Evans wrote: > >>On Fri, 23 Dec 2011, Alexander Best wrote: > >> > >>>is -mpreferred-stack-boundary=2 really necessary for i386 builds any > >>>longer? > >>>i built GENERIC (including modules) with and without that flag. the > >>>results > >>>are: > >> > >>The same as it has always been. It avoids some bloat. > >> > >>>1654496 bytes with the flag set > >>>vs. > >>>1654952 bytes with the flag unset > >> > >>I don't believe this. GENERIC is enormously bloated, so it has size > >>more like 16MB than 1.6MB. Even a savings of 4K instead of 456 bytes > > > >i'm sorry. i used du(1) to get those numbers, so i believe those numbers > >represent the ammount of 512-byte blocks. if i'm correct GENERIC is even > >more bloated than you feared and almost reaches 1GB: > > > >807,859375 megabytes with flag set > >vs. > >808,0820313 megabytes without the flag set > > That's certainly bloated. It counts all object files and modules, and > probably everything is compiled with -g. I only counted kernel text > size. yeah, but for demonstrating the different size between the build with -mpreferred-stack-boundary=2 set and -mpreferred-stack-boundary=2 unset, it doesn't really matter how big the directories are and if object files are included. the difference in size is < 1 megabyte. so setting -mpreferred-stack-boundary=2 doesn't aid in reducing the kernel (or modules) size, but merely to improve improve stack performance/efficiency. cheers. alex > > Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 11:44:27 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 499DC106566B; Sat, 24 Dec 2011 11:44:27 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail03.syd.optusnet.com.au (mail03.syd.optusnet.com.au [211.29.132.184]) by mx1.freebsd.org (Postfix) with ESMTP id D24DD8FC0A; Sat, 24 Dec 2011 11:44:26 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail03.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id pBOBiMTM024624 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 24 Dec 2011 22:44:24 +1100 Date: Sat, 24 Dec 2011 22:44:22 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Alexander Best In-Reply-To: <20111224093753.GA12377@freebsd.org> Message-ID: <20111224222040.Q2266@besplex.bde.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> <20111224093753.GA12377@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 11:44:27 -0000 On Sat, 24 Dec 2011, Alexander Best wrote: > On Sat Dec 24 11, Bruce Evans wrote: >> On Fri, 23 Dec 2011, Alexander Best wrote: > ... >>> the gcc(1) man page states the following: >>> >>> " >>> This extra alignment does consume extra stack space, and generally >>> increases code size. Code that is sensitive to stack space usage, >>> such as embedded systems and operating system kernels, may want to >>> reduce the preferred alignment to -mpreferred-stack-boundary=2. >>> " >>> >>> the comment in sys/conf/kern.mk however sorta suggests that the default >>> alignment of 4 bytes might improve performance. >> >> The default stack alignment is 16 bytes, which unimproves performance. > > maybe the part of the comment in sys/conf/kern.mk, which mentions that a stack > alignment of 16 bytes might improve micro benchmark results should be removed. > this would prevent people (like me) from thinking, using a stack alignment of > 4 bytes is a compromise between size and efficiently. it isn't! currently a > stack alignment of 16 bytes has no advantages towards one with 4 bytes on i386. I think the comment is clear enough. It it mentions all the tradeoffs. It is only slightly cryptic in saying that these are tradeoffs and that the configuration is our best guess at the best tradeoff -- it just says "while" for both. It goes without saying that we don't use our worst guess. Anyone wanting to change this should run benchmarks and beware that micro-benchmarks are especially useless. The changed comment is not so good since it no longer mentions micro-bencharmarks or says "while". > so specifying -mpreferred-stack-boundary=2 on i386 is absolutely mandatory. Not mandatory; just an optimization. > > please see the attached patch, which also introduduces a line break in order to > describe the stack alignment issue in a paragraph of its own. There should also be an empty line for a paragraph break. % +# Explicitly prohibit the use of FPU, SSE and other SIMD operations inside the % +# kernel itself. These operations are exclusively reserved for user % +# applications. This part was actually wronger: - these operations are not really reserved, but were just not supported in the kernel - they have been supported in the kernel for some time, although anything wanting to use the compiler to generate them would have to do something to kill the options added here. Kernel code using them must inform the kernel that it is doing so, using fpu_kern*(9undoc), and this is only valid in some contexts (more or less for kernel-only threads) so we still prevent compilers from using them routinely. The makefile is not the right place to describe any of this, Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 11:47:14 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 501841065672; Sat, 24 Dec 2011 11:47:14 +0000 (UTC) Date: Sat, 24 Dec 2011 11:47:14 +0000 From: Alexander Best To: Bruce Evans Message-ID: <20111224114714.GA29325@freebsd.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> <20111224211903.I2059@besplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111224211903.I2059@besplex.bde.org> Cc: Adrian Chadd , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 11:47:14 -0000 On Sat Dec 24 11, Bruce Evans wrote: > On Fri, 23 Dec 2011, Adrian Chadd wrote: > > >Well, the whole kernel is bloated at the moment, sorry. > > > >I've been trying to build the _bare minimum_ required to bootstrap > >-HEAD on these embedded boards and I can't get the kernel down below 5 > >megabytes - ie, one with FFS (with options disabled), MIPS, INET (no > >INET6), net80211, ath (which admittedly is big, but I need it no > >matter what, right?) comes in at: > > > >-r-xr-xr-x 1 root wheel 5307021 Nov 29 19:14 kernel.LSSR71 > > > >And with INET6, on another board (and this includes MSDOS and the > >relevant geom modules): > > > >-r-xr-xr-x 1 root wheel 5916759 Nov 28 12:00 kernel.RSPRO > > > >.. honestly, that's what should be addressed. That's honestly a bit > >ridiculous. > > It's disgusting, but what problems does it cause apart from minor slowness > from cache misses? > > I used to monitor the size of a minimal i386 kernel: > > % machine i386 > % cpu I686_CPU > % ident MIN > % options SCHED_4BSD > > In FreeBSD-5-CURRENT between 5.1R and 5.2R, this had size: > > text data bss dec hex filename > 931241 86524 62356 1080121 107b39 /sysc/i386/compile/min/kernel > > A minimal kernel is not useful, but maybe you can add some i/o to it > without bloating it too much. > > This almost builds in -current too. I had to add the following: > - NO_MODULES to de-bloat the compile time > - MK_CTF=no to build -current on FreeBSD.9. The kernel .mk files are > still broken (depend on nonstandard/new features in sys.mk). strange. the build(7) man page claims that: " WITH_CTF If defined, the build process will run the DTrace CTF conversion tools on built objects. Please note that this WITH_ option is handled differently than all other WITH_ options (there is no WITHOUT_CTF, or correspond- ing MK_CTF in the build system). " ... so setting MK_CTF to anything shouldn't have (according to the man page). cheers. alex > - comment out a line in if.c that refers to Vloif. if.c is standard > but the loop device is optional. > > A few more changes to remove non-minimalities that are not defaults > made little difference: > > % machine i386 > % cpu I686_CPU > % ident MIN > % options SCHED_4BSD > % > % # XXX kill default misconfigurations. > % makeoptions NO_MODULES=yes > % makeoptions COPTFLAGS="-O -pipe" > % > % # XXX from here on is to try to kill everything in DEFAULTS. > % > % # nodevice isa # needed for DELAY... > % # nooptions ISAPNP # needed ... > % > % nodevice npx > % > % nodevice mem > % nodevice io > % > % nodevice uart_ns8250 > % > % nooptions GEOM_PART_BSD > % nooptions GEOM_PART_EBR > % nooptions GEOM_PART_EBR_COMPAT > % nooptions GEOM_PART_MBR > % > % # nooptions NATIVE # needed ... > % # nodevice atpic # needed ... > % > % nooptions NEW_PCIB > % > % nooptions VFS_ALLOW_NONMPSAFE > > text data bss dec hex filename > 1663902 110632 136892 1911426 1d2a82 kernel > > (This was about 100K larger with -O2 and all DEFAULTS). The bloat since > FreeBSD-5 is only 70%. > > Here are some sizes for my standard kernel (on i386). The newer > versions have about the same number of features since they don't support > so many old isa devices or so many NICs: > > text data bss dec hex filename > 1483269 106972 172524 1762765 1ae5cd FreeBSD-3/kernel > 1917408 157472 194228 2269108 229fb4 FreeBSD-4/kernel > 2604498 198948 237720 3041166 2e678e FreeBSD-5.1.5/kernel > 2833842 206856 242936 3283634 321ab2 > FreeBSD-5.1.5/kernel-with-acpi > 2887573 192456 288696 3368725 336715 FreeBSD-5.1.5/kernel > with my changes, -O2 and usb > added relative to the above > 2582782 195756 298936 3077474 2ef562 previous, with some excessive > inlining avoided, and without -O2, > and with ipfilter > 1998276 159436 137748 2295460 2306a4 kernel.4 > a more up to date and less hacked on > FreeBSD-4 > 4365549 262656 209588 4837793 49d1a1 kernel.7 > 4406155 266496 496532 5169183 4ee01f kernel.7.invariants > 3953248 242464 207252 4402964 432f14 kernel.7.noacpi > 4418063 268288 240084 4926435 4b2be3 kernel.7.smp > various fairly stock FreeBSD-7R > kernels > 3669544 262848 249712 4182104 3fd058 kernel.c > 4174317 258240 540144 4972701 4be09d kernel.c.invariants > 3964455 250656 249808 4464919 442117 kernel.c.noacpi > 3213928 240160 240596 3694684 38605c kernel.c.noacpi-ule > 4285040 268288 286160 4839488 49d840 kernel.c.smp > current before FreeBSD-8R > not all built at the same time or > with the same options. The 20% > bloat between kernel.c.noacpi.ule > and kernel.c.noacpi is mainly > from not killing the default of > -O2. > 4742714 315008 401692 5459414 534dd6 kernel.8 > 4816900 319200 1813916 6950016 6a0c80 kernel.8.invariants > 4490209 304832 395260 5190301 4f329d kernel.8.noacpi > 4795475 323680 475420 5594575 555dcf kernel.8.smp > various fairly stock FreeBSD-8R > kernels > 4979632 287020 488404 5755056 57d0b0 kernel.cur > 5062953 289196 1902676 7254825 6eb329 kernel.cur.invariants > 5361809 295052 576984 6233845 5f1ef5 kernel.cur.smp > > amd64 kernels are only bout 4% larger if i386 kernels are built with > equivalant CFLAGS (-march=athlon64 for athlon64 CPU), but I usually > optimize i386 kernels for space and portability by compiling them > with -mtune=old. > > Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 12:14:25 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: by hub.freebsd.org (Postfix, from userid 1233) id B6854106566B; Sat, 24 Dec 2011 12:14:25 +0000 (UTC) Date: Sat, 24 Dec 2011 12:14:25 +0000 From: Alexander Best To: Bruce Evans Message-ID: <20111224121425.GA31084@freebsd.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> <20111224093753.GA12377@freebsd.org> <20111224222040.Q2266@besplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111224222040.Q2266@besplex.bde.org> Cc: freebsd-current@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 12:14:26 -0000 On Sat Dec 24 11, Bruce Evans wrote: > On Sat, 24 Dec 2011, Alexander Best wrote: > > >On Sat Dec 24 11, Bruce Evans wrote: > >>On Fri, 23 Dec 2011, Alexander Best wrote: > >... > >>>the gcc(1) man page states the following: > >>> > >>>" > >>>This extra alignment does consume extra stack space, and generally > >>>increases code size. Code that is sensitive to stack space usage, > >>>such as embedded systems and operating system kernels, may want to > >>>reduce the preferred alignment to -mpreferred-stack-boundary=2. > >>>" > >>> > >>>the comment in sys/conf/kern.mk however sorta suggests that the default > >>>alignment of 4 bytes might improve performance. > >> > >>The default stack alignment is 16 bytes, which unimproves performance. > > > >maybe the part of the comment in sys/conf/kern.mk, which mentions that a > >stack > >alignment of 16 bytes might improve micro benchmark results should be > >removed. > >this would prevent people (like me) from thinking, using a stack alignment > >of > >4 bytes is a compromise between size and efficiently. it isn't! currently a > >stack alignment of 16 bytes has no advantages towards one with 4 bytes on > >i386. > > I think the comment is clear enough. It it mentions all the tradeoffs. > It is only slightly cryptic in saying that these are tradeoffs and that > the configuration is our best guess at the best tradeoff -- it just says > "while" for both. It goes without saying that we don't use our worst > guess. Anyone wanting to change this should run benchmarks and beware > that micro-benchmarks are especially useless. The changed comment is not > so good since it no longer mentions micro-bencharmarks or says "while". if micro benchmark results aren't of any use, why should the claim that the default stack alignment of 16 bytes might produce better outcome stay? it doesn't seem as if anybody has micro benchmarked 16 bytes vs. 4 bytes stack alignment, until now. so the micro benchmark statement in the comment seems to be pure speculation. even worse...it indicates that by removing the -mpreferred-stack-boundary=2 flag, one can gain a performance boost by sacrifying a few more bytes of kernel (and module) size. this suggests that the behavior -mpreferred-stack-boundary=2 vs. not specyfing it, losely equals the semantics of -Os vs. -O2. i don't see how a 4 byte stack alignment for the kernel has any tradeoffs against the default 16 byte alignment. so if there are no tradeoffs, the comment shouldn't imply that there are. cheers. alex > > >so specifying -mpreferred-stack-boundary=2 on i386 is absolutely mandatory. > > Not mandatory; just an optimization. > > > > >please see the attached patch, which also introduduces a line break in > >order to > >describe the stack alignment issue in a paragraph of its own. > > There should also be an empty line for a paragraph break. > > % +# Explicitly prohibit the use of FPU, SSE and other SIMD operations > inside the > % +# kernel itself. These operations are exclusively reserved for user > % +# applications. > > This part was actually wronger: > - these operations are not really reserved, but were just not supported > in the kernel > - they have been supported in the kernel for some time, although anything > wanting to use the compiler to generate them would have to do something > to kill the options added here. Kernel code using them must inform the > kernel that it is doing so, using fpu_kern*(9undoc), and this is > only valid in some contexts (more or less for kernel-only threads) > so we still prevent compilers from using them routinely. The makefile > is not the right place to describe any of this, > > Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 12:40:14 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC6D6106564A; Sat, 24 Dec 2011 12:40:13 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail02.syd.optusnet.com.au (mail02.syd.optusnet.com.au [211.29.132.183]) by mx1.freebsd.org (Postfix) with ESMTP id 6CF2B8FC0C; Sat, 24 Dec 2011 12:40:13 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail02.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id pBOCe9TG027147 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 24 Dec 2011 23:40:11 +1100 Date: Sat, 24 Dec 2011 23:40:09 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Alexander Best In-Reply-To: <20111224114714.GA29325@freebsd.org> Message-ID: <20111224225717.P2417@besplex.bde.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> <20111224211903.I2059@besplex.bde.org> <20111224114714.GA29325@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Adrian Chadd , freebsd-current@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 12:40:14 -0000 On Sat, 24 Dec 2011, Alexander Best wrote: > On Sat Dec 24 11, Bruce Evans wrote: >> This almost builds in -current too. I had to add the following: >> - NO_MODULES to de-bloat the compile time >> - MK_CTF=no to build -current on FreeBSD.9. The kernel .mk files are >> still broken (depend on nonstandard/new features in sys.mk). > > strange. the build(7) man page claims that: > > " > WITH_CTF If defined, the build process will run the DTrace CTF > conversion tools on built objects. Please note that > this WITH_ option is handled differently than all other > WITH_ options (there is no WITHOUT_CTF, or correspond- > ing MK_CTF in the build system). > " > > ... so setting MK_CTF to anything shouldn't have (according to the man page). MK_CTF is an implementation detail. It is normally set in bsd.own.mk (not in sys.mk line I said -- this gives another, much larger bug (*)). But when usr/share/mk is old, it doesn't know anything about MK_CTF. (For example, in FreeBSD-9, sys.mk sets NO_CTF to 1 if WITH_CTF is not defined. This corresponds to bsd.own.mk in -current setting MK_CTF to "no" if WITH_CTF is not defined. Go back to an older version of FreeBSD and /usr/share/mk/* won't know anything about any CTF variable.) So when you try to build a current kernel under an old version of FreeBSD, MK_CTF is used uninitialized and the build fails. (Of course, "you" build kernels normally and don't use the bloated buildkernel method.) The bug is in the following files: kern.post.mk:.if ${MK_CTF} != "no" kern.pre.mk:.if ${MK_CTF} != "no" kmod.mk:.if defined(MK_CTF) && ${MK_CTF} != "no" except for the last one where it has been fixed. (*) Well, not completely broken, but just annoyingly unportabile. Consider the following makefile: %%% foo: foo.c %%% Invoking this under FreeBSD-9 gives: %%% cc -O2 -pipe foo.c -o foo [ -z "ctfconvert" -o -n "1" ] || (echo ctfconvert -L VERSION foo && ctfconvert -L VERSION foo) %%% This is the old ctf method. It is ugly but is fairly portable. Invoking this under FreeBSD-9 but with -m gives %%% cc -O2 -pipe foo.c -o foo ${CTFCONVERT_CMD} expands to empty string %%% This is because: - the rule in sys.mk says ${CTFCONVERT_CMD} - CTFCONVERT_CMD is normally defined in bsd.own.mk. But bsd.own.mk is only included by BSD makefiles. It is never included by portable makefiles. So ${CTFCONVERT_CMD} is used uninitialized. - for some reason, using variables uninitialized is not fatal in this context, although it is for the comparisons of ${MK_CTF} above. - ${CTFCONVERT_CMD} is replaced by the empty string. Old versions of make warn about the use of an empty string as a shell command. - the code that is supposed to prevent the previous warning is in bsd.own.mk, where it is not reached for portable makefiles. It is: % .if ${MK_CTF} != "no" % CTFCONVERT_CMD= ${CTFCONVERT} ${CTFFLAGS} ${.TARGET} This uses the full ctfconvert if WITH_CTF. % .elif ${MAKE_VERSION} >= 5201111300 % CTFCONVERT_CMD= make(1) has been modified to not complain about the empty string. The version test detects which versions of make don't complain. % .else % CTFCONVERT_CMD= @: The default is to generate this non-empty string and an extra shell command to execute it, for old versions of make. % .endif But none of this works for portable makefiles, since it is not reached. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 16:26:56 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 494A3106566B for ; Sat, 24 Dec 2011 16:26:56 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) by mx1.freebsd.org (Postfix) with ESMTP id 948398FC13 for ; Sat, 24 Dec 2011 16:26:52 +0000 (UTC) Received: from secured.by.ipfw.ru ([81.200.11.182] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1ReUQt-000AIF-VZ; Sat, 24 Dec 2011 20:26:52 +0400 Message-ID: <4EF5FD0E.9050500@FreeBSD.org> Date: Sat, 24 Dec 2011 20:25:50 +0400 From: "Alexander V. Chernikov" User-Agent: Thunderbird 2.0.0.24 (X11/20100515) MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org, glebuis@FreeBSD.org X-Enigmail-Version: 0.96.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig91B66FE36B84030B8F648289" Cc: Subject: Use of RCU (read-copy-update) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 16:26:56 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig91B66FE36B84030B8F648289 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello list! Questions related to RCU happens to be asked every several years, last on was on 2010, nearly 2 years have passed, so I'm asking again (to keep tradition). What is RCU? RCU stands for read-copy-update locking technology that permits users not to lock readers at all. It works by using delayed free which is triggered after all CPUs have flushed its cachelines. It is now heavily used in various places in Linux kernel significantly raising performance. More technical information: http://lwn.net/Articles/262464/ (general explanation from authors) http://www.rdrop.com/users/paulmck/RCU/ (RCU "Homepage") http://en.wikipedia.org/wiki/Read-copy-update Previous discussions: http://lists.freebsd.org/pipermail/freebsd-arch/2004-March/001889.html http://lists.freebsd.org/pipermail/freebsd-arch/2006-November/005762.html= http://lists.freebsd.org/pipermail/freebsd-current/2010-October/020320.ht= ml Main problem: Idea is patended: General worlds: http://www.groklaw.net/articlebasic.php?story=3D20061028211523142 Patents in PDF: http://www.groklaw.net/pdf/IBM-835-Exhibit_522.pdf http://www.groklaw.net/pdf/IBM-835-Exhibit_523.pdf http://www.groklaw.net/pdf/IBM-835-Exhibit_524.pdf RCU was also one of discussed topics in SCO-Linux lawsuit ( see http://lwn.net/Articles/36164/ or http://en.wikipedia.org/wiki/SCO_v._IBM#Increased_damages_claims.2C_and_r= ead-copy-update_claims ) Currently IBM holds all RCU-related patents. Generally it is spoken that IBM permits using RCU for opensource or GPL projects, however I can't find any official link with explanation / conditions. However, userland RCU project exists under LGPL: http://lttng.org/urcu There is also (theoretially) an alternative approach implemented in dfBSD, lwkt: see lwkt_serialize.c and lwkt_token.c in kern/ subdirectory: http://fxr.watson.org/fxr/source/kern/?v=3DDFBSD I can't unfortunately find any finished explanation/documentation about how it works (on SMP) and if this works at all. It seems we need some kind of RCU to be implemented since performance abyss (at least in networking) between us and Linux grows. What can we do about this? 1) Do nothing. 2) Check if there a way to write and implementation non-covered by those patents 3) Determine exact conditions under which IBM permits using RCU? (maybe ask IBM directly?) 4) Consider the possibility of buying license from IBM (if we can redistribute code under BSD license after that) --------------enig91B66FE36B84030B8F648289 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk71/RgACgkQwcJ4iSZ1q2nKpQCeOiycWWvrHjiWJMtsKpe/zZEt RFYAn3M3c6wvFZIaHxseBTbruBGaEyK7 =1kLc -----END PGP SIGNATURE----- --------------enig91B66FE36B84030B8F648289-- From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 16:30:09 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31F5D106564A; Sat, 24 Dec 2011 16:30:09 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) by mx1.freebsd.org (Postfix) with ESMTP id D9CA88FC16; Sat, 24 Dec 2011 16:30:08 +0000 (UTC) Received: from secured.by.ipfw.ru ([81.200.11.182] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1ReUU4-000AJY-9b; Sat, 24 Dec 2011 20:30:08 +0400 Message-ID: <4EF5FDDC.504@FreeBSD.org> Date: Sat, 24 Dec 2011 20:29:16 +0400 From: "Alexander V. Chernikov" User-Agent: Thunderbird 2.0.0.24 (X11/20100515) MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org, Gleb Smirnoff References: <4EF5FD0E.9050500@FreeBSD.org> In-Reply-To: <4EF5FD0E.9050500@FreeBSD.org> X-Enigmail-Version: 0.96.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB45F0E590685030A7994A17C" Cc: Subject: Re: Use of RCU (read-copy-update) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 16:30:09 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB45F0E590685030A7994A17C Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Alexander V. Chernikov wrote: > Hello list! =2E. Sorry, glebius@ address was incorrect. >=20 > Questions related to RCU happens to be asked every several years, last > on was on 2010, nearly 2 years have passed, so I'm asking again (to kee= p > tradition). >=20 > What is RCU? RCU stands for read-copy-update locking technology that > permits users not to lock readers at all. It works by using delayed fre= e > which is triggered after all CPUs have flushed its cachelines. >=20 > It is now heavily used in various places in Linux kernel significantly > raising performance. >=20 > More technical information: > http://lwn.net/Articles/262464/ (general explanation from authors) > http://www.rdrop.com/users/paulmck/RCU/ (RCU "Homepage") > http://en.wikipedia.org/wiki/Read-copy-update >=20 > Previous discussions: > http://lists.freebsd.org/pipermail/freebsd-arch/2004-March/001889.html > http://lists.freebsd.org/pipermail/freebsd-arch/2006-November/005762.ht= ml > http://lists.freebsd.org/pipermail/freebsd-current/2010-October/020320.= html >=20 >=20 > Main problem: Idea is patended: > General worlds: > http://www.groklaw.net/articlebasic.php?story=3D20061028211523142 > Patents in PDF: > http://www.groklaw.net/pdf/IBM-835-Exhibit_522.pdf > http://www.groklaw.net/pdf/IBM-835-Exhibit_523.pdf > http://www.groklaw.net/pdf/IBM-835-Exhibit_524.pdf >=20 >=20 > RCU was also one of discussed topics in SCO-Linux lawsuit ( see > http://lwn.net/Articles/36164/ or > http://en.wikipedia.org/wiki/SCO_v._IBM#Increased_damages_claims.2C_and= _read-copy-update_claims > ) > Currently IBM holds all RCU-related patents. > Generally it is spoken that IBM permits using RCU for opensource or GPL= > projects, however I can't find any official link with explanation / > conditions. >=20 > However, userland RCU project exists under LGPL: http://lttng.org/urcu >=20 > There is also (theoretially) an alternative approach implemented in > dfBSD, lwkt: see lwkt_serialize.c and lwkt_token.c in kern/ > subdirectory: http://fxr.watson.org/fxr/source/kern/?v=3DDFBSD >=20 > I can't unfortunately find any finished explanation/documentation about= > how it works (on SMP) and if this works at all. >=20 > It seems we need some kind of RCU to be implemented since performance > abyss (at least in networking) between us and Linux grows. >=20 >=20 > What can we do about this? > 1) Do nothing. >=20 > 2) Check if there a way to write and implementation non-covered by thos= e > patents >=20 > 3) Determine exact conditions under which IBM permits using RCU? (maybe= > ask IBM directly?) >=20 > 4) Consider the possibility of buying license from IBM (if we can > redistribute code under BSD license after that) >=20 >=20 >=20 >=20 >=20 >=20 --------------enigB45F0E590685030A7994A17C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk71/dwACgkQwcJ4iSZ1q2noGACaAnQjDKTiKAJ/ehvTxuUs0rTR Pw4An2GsoaYsZEjQHntSUn+P+jRCvhMM =lEOu -----END PGP SIGNATURE----- --------------enigB45F0E590685030A7994A17C-- From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 16:54:57 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 654B51065670; Sat, 24 Dec 2011 16:54:57 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id EEACC8FC12; Sat, 24 Dec 2011 16:54:56 +0000 (UTC) Received: by vbbfr13 with SMTP id fr13so14022259vbb.13 for ; Sat, 24 Dec 2011 08:54:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=kQKwhY8zkRbgTY4s65iTdqpAN2W6Gy+TjQ1vZCYiblw=; b=grTE/rBcS2pTuDaq+ztpslYRgHqO2Q9tMi86XTqFu0zScUVqEpG3MqaLtdfFxokBi8 u9kCHYEln69ANIE1nhSapruYNLS7nRP6pGIngTeemHiEY1vP3VuxVNBYueV11ytCpvLb u8MiYrVznppk5m0PSBFig/rYlv6i+tXvfijuE= MIME-Version: 1.0 Received: by 10.52.29.16 with SMTP id f16mr9608023vdh.45.1324745696270; Sat, 24 Dec 2011 08:54:56 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.36.5 with HTTP; Sat, 24 Dec 2011 08:54:56 -0800 (PST) In-Reply-To: <4EF5FDDC.504@FreeBSD.org> References: <4EF5FD0E.9050500@FreeBSD.org> <4EF5FDDC.504@FreeBSD.org> Date: Sat, 24 Dec 2011 08:54:56 -0800 X-Google-Sender-Auth: YEgfnYlB9fgrOKZCpOBCt8qJAmk Message-ID: From: Adrian Chadd To: "Alexander V. Chernikov" Content-Type: text/plain; charset=ISO-8859-1 Cc: Gleb Smirnoff , freebsd-arch@freebsd.org Subject: Re: Use of RCU (read-copy-update) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 16:54:57 -0000 Please just keep in mind that BSD code _is_ used commercially and I wonder if using RCU (and thus being potentially patent-encumbered) would make FreeBSD less attractive. Adrian From owner-freebsd-arch@FreeBSD.ORG Sat Dec 24 21:28:38 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CF9D106564A for ; Sat, 24 Dec 2011 21:28:38 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from zim.MIT.EDU (ZIM.MIT.EDU [18.95.3.101]) by mx1.freebsd.org (Postfix) with ESMTP id AD1568FC0A for ; Sat, 24 Dec 2011 21:28:31 +0000 (UTC) Received: from zim.MIT.EDU (localhost [127.0.0.1]) by zim.MIT.EDU (8.14.5/8.14.2) with ESMTP id pBOLSURf058831; Sat, 24 Dec 2011 16:28:30 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by zim.MIT.EDU (8.14.5/8.14.2/Submit) id pBOLSU3b058830; Sat, 24 Dec 2011 16:28:30 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Sat, 24 Dec 2011 16:28:30 -0500 From: David Schultz To: "Alexander V. Chernikov" Message-ID: <20111224212830.GA58693@zim.MIT.EDU> Mail-Followup-To: "Alexander V. Chernikov" , freebsd-arch@freebsd.org, glebuis@freebsd.org References: <4EF5FD0E.9050500@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EF5FD0E.9050500@FreeBSD.org> Cc: glebuis@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: Use of RCU (read-copy-update) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2011 21:28:38 -0000 On Sat, Dec 24, 2011, Alexander V. Chernikov wrote: > Main problem: Idea is patended: > General worlds: > http://www.groklaw.net/articlebasic.php?story=20061028211523142 > Patents in PDF: > http://www.groklaw.net/pdf/IBM-835-Exhibit_522.pdf > http://www.groklaw.net/pdf/IBM-835-Exhibit_523.pdf > http://www.groklaw.net/pdf/IBM-835-Exhibit_524.pdf [...] > What can we do about this? > 1) Do nothing. > > 2) Check if there a way to write and implementation non-covered by those > patents > > 3) Determine exact conditions under which IBM permits using RCU? (maybe > ask IBM directly?) > > 4) Consider the possibility of buying license from IBM (if we can > redistribute code under BSD license after that) Negotiating or fighting patents is a messy business, so a technical solution may be easier. The ideas behind RCU are very old, so one way out is to implement one of the older out-of-patent ideas directly. In an expired patent, IBM called the idea "passive serialization." You can also find a lot of (hopefully unencumbered) research on "software transactional memory", which is a nice abstraction for a similar mechanism. (Most of the papers focus on how to do atomic read-write transactions; read-only transactions are much easier.) But be careful: If you independently invent an improvement that isn't covered by the older patent but is covered by RCU, that idea may still infringe.