From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 07:03:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 78DBC733 for ; Tue, 8 Apr 2014 07:03:37 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CA991BEB for ; Tue, 8 Apr 2014 07:03:36 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 8995B73FDF; Tue, 8 Apr 2014 00:03:35 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 69944-07; Tue, 8 Apr 2014 00:03:35 -0700 (PDT) Received: from [10.8.0.18] (unknown [10.8.0.18]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 5515273FD9; Tue, 8 Apr 2014 00:03:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1396940615; bh=vhi29iPZoQ6K5RaYBl0csmQAHA3vTaP/f86XNqZPCIM=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=EnV5v+vlHHsoHknMmdq6ROLTfYg4me3CDRb/42WX+hRFPxpEd/CZz5T+pBqSNKIiK dd+6dc8+TcuElkWDsL8Nx3jhAbxzA7qzCuRm/MbQ69dIQOlkL5gPZq2lOCZgK1z5ma O0jjUnCkkgebDv6zxlcjAV4qUHBOaikqVgpcOiOk= Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Compiler toolchain roadmap From: Jordan Hubbard In-Reply-To: Date: Tue, 8 Apr 2014 12:03:31 +0500 Content-Transfer-Encoding: quoted-printable Message-Id: <482357EF-5596-45FD-8D2C-3742BB4872E8@ixsystems.com> References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> <9E11A6D4-9D18-422D-9514-4714AADDAEF4@gmail.com> <8E22F8FA-CF71-4A47-BDE8-F3CE6158E1C9@ixsystems.com> To: Warner Losh X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 07:03:37 -0000 On Apr 7, 2014, at 8:03 PM, Warner Losh wrote: > This is a desire, not a hard requirement. As such, there may be = missing bits if you chose to go this route. At least that=92s been the = notion in the past when using llvm features has come up. The notion for = this path has always been =91It is possible, but only a subset of the = functionality may be available.=94 >=20 > So if one were to add blocks, it would need a knob WITHOUT_BLOCKS that = would disable all functionality tied to them. For many applications, = this is a reasonable subset (based on my guesses at how intrusive this = would be to the system). And it might even be automatically selected = based on compiler support, but that=92s another can of worms. I=92m glad to hear that building with foreign compilers is more of a = desire than a hard constraint. I=92m still not sure what the scenario = would look like where libdispatch has been built WITHOUT_BLOCKS and is = now essentially API incompatible with the libdispatch compiled = WITH_BLOCKS (since you wouldn=92t get functions like dispatch_async() = but would get the dispatch_async_f() function which takes a function = pointer/context variable instead), but I guess that=92s a problem for = the hypothetical wacky-compiler user to figure out? You seemed to imply that when you said FreeBSD was already using a = number of custom compiler extensions, I just wasn=92t sure that this = extended as far as =93you can build FreeBSD, for the most part, with = your weird-ass compiler but you can also expect it to be a different = FreeBSD than what=92s in base for x86_64 (for example) because various = things will be switched off, API incompatible, or simply not compiled at = all." If that=92s the case, then woohoo, when can I import libdispatch into = base? :) [I=92m being semi-facetious here, I realize there are a few = more steps to go]. > So to take this a step further=85 There=92s many levels of integration = here=85 First, there=92s the kernel, which is most often the bit of = code people want/need the special compiler for. Next, there=92s having = the feature available in user land. Finally, there=92d be a wide-scale = integration of this feature. I see very few programs in base benefiting = from libdispatch, honestly, but that doesn=92t mean the set is empty. Do = you have a longer write up on what you=92d like to do here? Sure, I can write something up as part of my =93any objections to doing = this?=94 step, but just to summarize briefly for this conversation: 1. Libdispatch does not need to touch the kernel at all, nor does the = kernel require blocks support. The pthread_workqueue() stuff that = Stacey Son did all those years ago would be a nice optimization (this = lets libdispatch balance its thread pool loads across multiple = applications, not just within a single application) but it=92s not = strictly necessary, and it can also be added later without any consumer = of libdispatch (henceforth referred to by its other name of GCD, because = that=92s easier to type!) needing to know or care. 2. Once we have GCD in base, I can then start looking at adding = libnotify and its associated notifyd daemon, which is a generic system = notification mechanism. 3. Once libnotify is in base, I=92ll then add some basic notifications = to Libc, like timezone changes, hostname changes, and so on, = invalidating some of its internal caches in the face of said changes = (there are some fairly expensive stat() calls you can eliminate once you = have the far lighter-weight shared memory flag checking that notifyd = provides, which also helps save power by not walking around the = filesystem so much). The notifyd service can also translate filesystem = notifications into system notifications, which means services like samba = and netatalk can detect (cheaply) when something has changed out from = under them. Beyond that, we probably need to talk about another daemon, which on OS = X is called configd, which deals with larger scope configuration changes = like network devices coming and going, default routes changing, and so = on. This does require some kernel up-call mechanisms, but it will = =93publish=94 its results in terms of notify(3) notifications so = applications can subscribe to those events the same way. If we can hack = usbd into doing the same for USB events, that=92s great, otherwise I = would imagine configd dealing with that sort of hardware notification = service as well. > I see a continuum of answer here: If you want to modify the kernel = extensive to use blocks, then that=92s going to be a much bigger problem = than having a few daemons and a library in the tree that require them = which is a bigger problem than having a few daemons using it, but = optionally, and a library which is a bigger problem than a library in = the tree which is a bigger problem than the status quo. I.. think=85 I follow that sentence enough to hope I=92ve answered it. = No blocks in the kernel. Nothing more than a few daemons and a library = and, at some point, a way of figuring out when the kernel has made = certain changes to the system configuration, if and where said changes = can=92t be intercepted and dealt with at the libc layer or otherwise = noticed by a user-land daemon which can post the notifications for any = subscribers of them. As to the subscribers, they won=92t have to use blocks unless they = choose to consume one or more of notify(3)=92s block APIs. The notify = mechanism was written with both old-style and new-style clients in mind. = If your daemon wants a signal when the notification it=92s subscribing = to changes, fine. If it wants some data on a file descriptor which it=92s= poll(3)/kqueue(3)ing, fine. See = https://developer.apple.com/library/mac/documentation/Darwin/Reference/Man= Pages/man3/notify.3.html and, of course, ignore the mach-specific = delivery methods as N/A. > And 8.1 for libdispatch? Is it really so portable it would work with = our old, crappy gcc pre blocks update? Sorry, misunderstanding. I was referring to the original slashdot / = phoronix article which stated that GCD would be coming to FreeBSD in = 8.1. Apparently, according to FreeBSD=92s own wiki = (https://wiki.freebsd.org/GCD) it works, too. - Jordan