From owner-freebsd-current@FreeBSD.ORG Wed Oct 30 05:07:22 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5EF512F5; Wed, 30 Oct 2013 05:07:22 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 28A5C23A5; Wed, 30 Oct 2013 05:07:21 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.31]) by ltcfislmsgpa02.fnfis.com (8.14.5/8.14.5) with ESMTP id r9U53qYB012571 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Wed, 30 Oct 2013 00:07:20 -0500 Received: from LTCFISWMSGMB21.FNFIS.com ([169.254.1.205]) by LTCFISWMSGHT03.FNFIS.com ([10.132.206.31]) with mapi id 14.03.0146.000; Wed, 30 Oct 2013 00:06:39 -0500 From: "Teske, Devin" To: Nathan Whitehorn Subject: Re: [RFC] libdispatch (aka Grand Central Dispatch) in base Thread-Topic: [RFC] libdispatch (aka Grand Central Dispatch) in base Thread-Index: AQHO1RRnBr/4vusiu0ivD2Ie4//LzA== Date: Wed, 30 Oct 2013 05:06:38 +0000 Message-ID: <7A331B54-8D7C-4A31-9168-47F96216FCB5@fisglobal.com> References: <527084AC.2020701@freebsd.org> In-Reply-To: <527084AC.2020701@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.132.253.120] Content-Type: text/plain; charset="us-ascii" Content-ID: <028D8D079A0BED4C8A2196B47BC24E75@fisglobal.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794, 1.0.431, 0.0.0000 definitions=2013-10-29_08:2013-10-30,2013-10-29,1970-01-01 signatures=0 Cc: Devin Teske , FreeBSD-Current Current , "Teske, Devin" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Devin Teske List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 05:07:22 -0000 On Oct 29, 2013, at 9:01 PM, Nathan Whitehorn wrote: > On 10/29/13 21:04, Teske, Devin wrote: >> Hi all, >>=20 >> I'd like to bring up the discussion for topic.. >>=20 >> Importing libdispatch (aka Apple's Grand Central Dispatch) into base (co= ntrib?). >>=20 >> Specifically into HEAD then MFC'd only as far back as stable/10. >>=20 >> Here's the reason why: >> http://devinteske.com/freebsd-installer-enhancements >>=20 >> Summary: >> For the purpose of providing a concurrency model better than pthreads fo= r the >> expressed desire to bring about concurrent data processing (applicable d= irectly >> to distributions, packages, signing and more). >>=20 >> Multiple people have confirmed with me with respect to the above blog ar= ticle >> that the concurrency model would be most efficient with libdispatch. >>=20 >> Since the tool mentioned in the blog is >> a. Compiling with clang >> b. Requires newest dialog(3) that is only in stable/10 or higher >>=20 >> I'd say that it looks like a match made in heaven. >>=20 >> But of course, there's that one hang-up... dispatch is not available in = base yet. >>=20 >> Is anyone working on getting dispatch into base? >=20 > I have no opinion on GCD in base -- probably a good idea -- but I was hop= ing you could explain further what you are trying to do here. Parallelism i= n these steps is usually of very limited utility. For checksum evaluation, = it could speed things up on multicore systems. For fetch and extract, howev= er, it either has no effect (if you are bandwidth limited during fetch) or = slows things down significantly by causing extra seeks. This problem is esp= ecially bad for CD installs. For extract, it can actually cause lasting pro= blems on the installed system by increasing disk fragmentation. The parallel concurrency was actually an after-thought. First... I started with this... http://pastebin.com/LvDtJNGh A straight-forward attempt to see if I could add X11 support by finding a feature that dialog(1) and Xdialog(1) agree upon. They agree that when they are reading pipe data for the gauge widget, they will update their prompt text when the input looks like: XXX New prompt text XXX NB: Literally "XXX" on a line by itself, sandwiching the new prompt. I then quickly rewrote that into C and added dozens of features to get where the fdpv prototype is at now. I'm glad we're having this discussion, because I acknowledge that I very well can push ahead without concurrency (and we may want to do that, because the utility brings other benefits that are foremost on my list behind parallelism). Trying to achieve: 1. X11 support 2. Get rid of the need for temporary files (without sacrificing signing) 3. Internationalization So would you recommend just moving forward without the parallelism so we could add that as a value-add down the line? That would actually speed up the development of fdpv as a new base utility that can take either stdin or a list of fifo's with size and signat= ure data, and verify the data before forking off a tar command, or pkg etc. --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you.