From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 19 22:14:43 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54E97106566B for ; Wed, 19 Mar 2008 22:14:43 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: from hyperion.scode.org (hyperion.scode.org [85.17.42.115]) by mx1.freebsd.org (Postfix) with ESMTP id D1B8E8FC1C for ; Wed, 19 Mar 2008 22:14:42 +0000 (UTC) (envelope-from scode@hyperion.scode.org) Received: by hyperion.scode.org (Postfix, from userid 1001) id 8A38D23C49B; Wed, 19 Mar 2008 23:14:41 +0100 (CET) Date: Wed, 19 Mar 2008 23:14:41 +0100 From: Peter Schuller To: Julian Elischer Message-ID: <20080319221441.GA22580@hyperion.scode.org> References: <20080318211453.GA37126@hyperion.scode.org> <47E183CC.6050309@elischer.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="VbJkn9YxBvnuCH5J" Content-Disposition: inline In-Reply-To: <47E183CC.6050309@elischer.org> User-Agent: Mutt/1.5.17 (2007-11-01) Cc: freebsd-hackers@freebsd.org Subject: Re: Asynchronous syscalls for massive concurrency X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2008 22:14:43 -0000 --VbJkn9YxBvnuCH5J Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > KSE's asynchronous system calls could be used to do what you suggest > however it turns out that if you are going to do this you probably want t= o=20 > do it in a way where the program is aware of what is going on, > so the ability to make all syscalls "TRANSPARENTLY ASYNCHRONOUS" is > not really a requirement. I did not mean to imply it had to be transparent; in fact that was not my goal. But I see it as a problem that, right now, if a process wishes to make n concurrent system calls (concurrent in the sense of the time period between request to response, not actually concurrent in terms of execution in the kernel or in user space), it *must* have n threads (or n co-operating processes). In other words, there is an enforcement that logical concurrency be matched with actual machine/kernel concurrency. This makes userland threading non-transparent (from the perspective of application code written in a language or framework that attempts to provide light-weight concurrency) because you can suddenly not count on all operations playing well with it. Basically, disregarding tricks that do things like offloading certain operations to separate processes (which still does not remove the n process/n thread requirement), any light-weight concurrency implementation is not generally applicable. That is, you are forced to trade trade light weight against a restricted feature set. > The async syscall part of KSE could be kept/reintroduced pretty > seperatly from the thread support part, but it really requires > someone to decide how to use it. The intended use would be to augment exiting code to take advantage of it. If it caught on among operating systems, it would be a big step forward in my opinion, from the perspective of application development. If a single or a few operating systems did it, they would have an advantage to the extent that someone actually implements the necessary support in $language_or_framework. In particular various higher level languages where it would make a huge difference to be able to support this in the interpreter/vm/core library. It would remove, or lessen, the need to differentiate between "not quite threads" and "threads" (witness stackless python tasks or ruby fibers, vs. the actual native threads in Python and Ruby 1.9; once again trading light-weight vs. generality). As far as I can see, it can be applied to the C runtime library as well, but the advantage would be less since you won't get around the need to maintain a per-thread native stack. This latter point is why I am mostly interested in it from the higher level language perspective, where higher level primitives (e.g., continuations, interpreter level stacks on the heap, etc) can be used for thread context instead of the native stack, thus removing the need to take an kbyte hit per thread just for the stack alone. It is not even mutually exclusive to native code compilation. Oh and btw, regarding my comment on paging: While as far as I can tell it holds relative to native threading, it is still not worse (better in fact) than the single-threaded state machine model where a high-latency paging operation blocks the entire process. --=20 / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller ' Key retrieval: Send an E-Mail to getpgpkey@scode.org E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org --VbJkn9YxBvnuCH5J Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.8 (FreeBSD) iEYEARECAAYFAkfhkFEACgkQDNor2+l1i30DagCfb2S8OF/XRsFuSVL/Vmv9746u BdcAnA/ZLM8QkWmI601gkiPZIzrMX7zv =qfbe -----END PGP SIGNATURE----- --VbJkn9YxBvnuCH5J--