From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 7 19:38:24 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92298F9; Sat, 7 Jun 2014 19:38:24 +0000 (UTC) Received: from mail-qg0-x235.google.com (mail-qg0-x235.google.com [IPv6:2607:f8b0:400d:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C62A2B0E; Sat, 7 Jun 2014 19:38:24 +0000 (UTC) Received: by mail-qg0-f53.google.com with SMTP id f51so6978235qge.26 for ; Sat, 07 Jun 2014 12:38:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=QqwSkqrlOuZEyHmHg2XnMkN0j2ybvZ2OjcvuPazu6Jk=; b=XhakTGB47TzfohaifmcXLhcbqMGgLU0PuQWUlnBZQpaR5CMOYhZddl7x+fT6mBDvpl ZdjQM7/OFeyMk+u4hbvb4AsxGSa+PIHeHzcwpB0mqNaiRKGP0sYhc/vE1YDENckh6XT1 TOosM/kC8Y5R90J796c1qORDYgdyzpAHVG+izgt1NoglPdcPRpfabz7+y457I5iBiy6K wFfOngtBlkDNXz2frl6ECftRpaJvyfOBs7lwTBxZ1KV6v4zCn+dWqVCXB6/mT7kIeC9u 9vhAHuQ4AnOsAQUztVJBte8VYRTik5JrTzu/Z6EFGdgfIhXtbT0T4z/Bvcg3egF03e+j hnhQ== MIME-Version: 1.0 X-Received: by 10.224.43.148 with SMTP id w20mr20685071qae.26.1402169903418; Sat, 07 Jun 2014 12:38:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Sat, 7 Jun 2014 12:38:23 -0700 (PDT) In-Reply-To: References: <1402159374.20883.160.camel@revolution.hippie.lan> Date: Sat, 7 Jun 2014 15:38:23 -0400 X-Google-Sender-Auth: JHcX5-Dc0ByIPS9xtD8o7FHQhps Message-ID: Subject: Re: Best practice for accepting TCP connections on multicore? From: Adrian Chadd To: Igor Mozolevsky Content-Type: text/plain; charset=UTF-8 Cc: Hackers freeBSD , Daniel Janzon , Dirk Engling , Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 19:38:24 -0000 On 7 June 2014 15:26, Igor Mozolevsky wrote: > I did say it was in Java, but so what? By 2008 Sun were going out of their > way to make Java both scalable and zero-cost. The point I was making was > that we set off with the assumption that kevent-based sockets are faster, > therefore server (as a whole) was assumed to have a higher performance; and > never bothered to actually measure it. On coding complexity front, from > slide 36: > > The story of Rob Van Behren > (as remembered by me from a lunch with Rob) > > Set out to write a high-performance asynchronous server system > > Found that when switching between clients, the code for saving and restoring > values/state was difficult > > Took a step back and wrote a finely-tuned, organized system for saving and > restoring state between clients > > When he was done, he sat back and realized he had written the foundation for > a threading package Right, and he implemented a _userland_ threading package. He tried writing a generic framework for doing this stuff, likely with coroutines, and yes he ended up with a userland threading library. This doesn't surprise me. We can do better. We can batch syscalls. We can batch event updates. We can batch IO. We can look at doing zero-copy IO. We can schedule things to keep them hot on CPU cores. He didn't talk about any of that. As I said, there's much more to this picture than just "threaded or not threaded." Take a look at the recentish Yahoo! paper talking about batched syscalls, for example. I'm jut tackling a small corner of it. Once it's done I'll likely look at some VM-light way of doing zero-copy IO. (The VM locking and refcouting overhead for doing all those 4k operations just sucks.) But this is all in my spare time, so it'll happen when it happens. I'm just fed up with it not being done. -a