From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 06:41:52 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4439B16A417; Mon, 21 Jan 2008 06:41:52 +0000 (UTC) (envelope-from jhealy@swin.edu.au) Received: from swin.edu.au (gpo5.cc.swin.edu.au [136.186.1.225]) by mx1.freebsd.org (Postfix) with ESMTP id AA50213C455; Mon, 21 Jan 2008 06:41:51 +0000 (UTC) (envelope-from jhealy@swin.edu.au) Received: from groupwise.swin.edu.au (hwncl3-inet.cc.swin.edu.au [136.186.3.197]) by swin.edu.au (8.13.6.20060614/8.13.1) with ESMTP id m0L5v5ih014184; Mon, 21 Jan 2008 16:57:05 +1100 Received: from [136.186.229.102] (jhealy.caia.swin.edu.au [136.186.229.102]) by groupwise.swin.edu.au with ESMTP; Mon, 21 Jan 2008 16:57:03 +1100 Message-ID: <4794342E.8080602@swin.edu.au> Date: Mon, 21 Jan 2008 16:57:02 +1100 From: James Healy User-Agent: Thunderbird 2.0.0.6 (X11/20071113) MIME-Version: 1.0 To: Andre Oppermann References: <20071219123305.Y95322@fledge.watson.org> <47693DBD.6050104@swin.edu.au> <476A45D6.6030305@freebsd.org> <47858D35.6060006@swin.edu.au> <4786D23A.1080509@freebsd.org> In-Reply-To: <4786D23A.1080509@freebsd.org> X-Enigmail-Version: 0.95.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.1.9 X-Spam-Checker-Version: SpamAssassin 3.1.9 (2007-02-13) on gpo5.cc.swin.edu.au Cc: Lawrence Stewart , net@freebsd.org, Robert Watson , grenville armitage , arch@freebsd.org Subject: Re: Coordinating TCP projects X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2008 06:41:52 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andre Oppermann wrote: > The first cut is now at //depot/projects/tcp_reass/ however I made > a mistake when creating the branch and now the code is in the same > changeset as the branching itself. Doesn't make it easy to do a > diff. Have to see how I can fix that. Anyway, have a look and I'm > going to finish/fix the code tomorrow evening. Our initial thoughts on dynamically resizing the TCP reassembly queue in step with the receive buffer size to avoid segments being dropped unnecessarily have been superseded by the tcp_reass branch. Amongst a number of number of optimisations, this dynamic cap is now implemented. Hopefully we'll find some spare cycles to test the branch out soon. James Healy http://caia.swin.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHlDQu4oawkrbYo/kRAvZdAJkB+vBb8Eu/a4cCqF/6v0q221/dNwCfbqOU Qvm2dFjGlJ2AQmOTFEeOZZ8= =053h -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 09:51:22 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2ACFB16A479 for ; Mon, 21 Jan 2008 09:51:22 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5CAC113C4EA for ; Mon, 21 Jan 2008 09:51:21 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 38142 invoked from network); 21 Jan 2008 09:13:10 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 21 Jan 2008 09:13:10 -0000 Message-ID: <47946B1B.7070009@freebsd.org> Date: Mon, 21 Jan 2008 10:51:23 +0100 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: James Healy References: <20071219123305.Y95322@fledge.watson.org> <47693DBD.6050104@swin.edu.au> <476A45D6.6030305@freebsd.org> <47858D35.6060006@swin.edu.au> <4786D23A.1080509@freebsd.org> <4794342E.8080602@swin.edu.au> In-Reply-To: <4794342E.8080602@swin.edu.au> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Lawrence Stewart , net@freebsd.org, Robert Watson , grenville armitage , arch@freebsd.org Subject: Re: Coordinating TCP projects X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2008 09:51:22 -0000 James Healy wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Andre Oppermann wrote: >> The first cut is now at //depot/projects/tcp_reass/ however I made >> a mistake when creating the branch and now the code is in the same >> changeset as the branching itself. Doesn't make it easy to do a >> diff. Have to see how I can fix that. Anyway, have a look and I'm >> going to finish/fix the code tomorrow evening. > > Our initial thoughts on dynamically resizing the TCP reassembly queue in > step with the receive buffer size to avoid segments being dropped > unnecessarily have been superseded by the tcp_reass branch. > > Amongst a number of number of optimisations, this dynamic cap is now > implemented. Hopefully we'll find some spare cycles to test the branch > out soon. It's not yet finished. I expect to be done later this week. I'll send an announcement when the code is ready for wider testing. Any exercising and testing of the code you can do is very appreciated. -- Andre From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 16:52:18 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20C0016A473; Mon, 21 Jan 2008 16:52:18 +0000 (UTC) (envelope-from jkim@FreeBSD.org) Received: from anuket.mj.niksun.com (gwnew.niksun.com [65.115.46.162]) by mx1.freebsd.org (Postfix) with ESMTP id D967413C447; Mon, 21 Jan 2008 16:52:16 +0000 (UTC) (envelope-from jkim@FreeBSD.org) Received: from niksun.com (anuket [10.70.0.5]) by anuket.mj.niksun.com (8.13.6/8.13.6) with ESMTP id m0LGqFkq003969; Mon, 21 Jan 2008 11:52:15 -0500 (EST) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: David Schultz Date: Mon, 21 Jan 2008 11:52:00 -0500 User-Agent: KMail/1.6.2 References: <200801181217.52788.jkim@FreeBSD.org> <20080119121316.GA21917@VARK.MIT.EDU> In-Reply-To: <20080119121316.GA21917@VARK.MIT.EDU> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200801211152.07680.jkim@FreeBSD.org> X-Virus-Scanned: ClamAV 0.92/5507/Mon Jan 21 09:34:07 2008 on anuket.mj.niksun.com X-Virus-Status: Clean Cc: freebsd-current@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: [RFC] SysV SHM on 64-bit platforms X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2008 16:52:18 -0000 On Saturday 19 January 2008 07:13 am, David Schultz wrote: > On Fri, Jan 18, 2008, Jung-uk Kim wrote: > > Because fixing this PR breaks ABI, I had to make compat shims. > > But the question is which branch to break? Since we are very > > close to 7.0-RELEASE, the question became more complicated. A > > lot of people are asking me to fix this PR because it is much > > needed fix for large database installations, so I have to make a > > quick decision. :-( > > It's far too late to fix it for 7.0. RC1 is out, the packages have > already been built, and this could potentially break things. In > particular, you have no way of guaranteeing that an "old" shared > library that uses semctl(2) linked with a "new" binary will work > properly, or vise versa. I know the compat shim is not bullet-proof and I perfectly understand your concerns. That's why I had to give four options and three sets of patches. :-( > For 8-CURRENT, I'd say go ahead and fix it and add the syscall > compat goop. Then I guess backport it to 7-STABLE after a while if > nothing important breaks and nobody has any objections to that > plan. Well, since the policy is 'no seat belt for -CURRENT', I think that is aceptable, too. > It would also be cool if struct ipcperm got fixed at the same time > so we only have to do this once. I think that only entails adding > compat crud to semctl and maybe one or two other syscalls. Those > changes aren't urgent enough to make it worthwhile to backport to > 7.X IMO, though. So, you are in between option 1 and 4, i.e., bigger fix goes to HEAD and partial MFC to RELENG_7, right? > Two questions/comments about the patches: > - Why the #if !defined(__i386__)? There are better ways to test > whether you're compiling for a 32-bit architecture, if that's > what you meant. I reused oshmid_ds, which was only for __i386__ && (COMPAT_FREEBSD4 || COMPAT_43). Basically I didn't want to add struct oh_my_god_we_did_it_again_shmid_ds. ;-) Actually, the arch check is done like this: if (sizeof(size_t) != sizeof(int) && td->td_proc->p_osrel <= WHATEVER) I am hoping the compiler is smart enough to optimize away 'sizeof(size_t) != sizeof(int)'. > - The 8.X version needs to take the p_osrel for the equivalent > change in 7.X into account. I thought it did, didn't it? Jung-uk Kim From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 21:01:49 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32B2A16A419; Mon, 21 Jan 2008 21:01:49 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from VARK.MIT.EDU (VARK.MIT.EDU [18.95.3.179]) by mx1.freebsd.org (Postfix) with ESMTP id DF6F013C457; Mon, 21 Jan 2008 21:01:48 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from VARK.MIT.EDU (localhost [127.0.0.1]) by VARK.MIT.EDU (8.14.2/8.14.1) with ESMTP id m0LL1jQd009419; Mon, 21 Jan 2008 16:01:45 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.MIT.EDU (8.14.2/8.14.1/Submit) id m0LL1jBP009418; Mon, 21 Jan 2008 16:01:45 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Mon, 21 Jan 2008 16:01:45 -0500 From: David Schultz To: Jung-uk Kim Message-ID: <20080121210145.GA9107@VARK.MIT.EDU> Mail-Followup-To: Jung-uk Kim , freebsd-current@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG References: <200801181217.52788.jkim@FreeBSD.org> <20080119121316.GA21917@VARK.MIT.EDU> <200801211152.07680.jkim@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200801211152.07680.jkim@FreeBSD.org> Cc: freebsd-current@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: [RFC] SysV SHM on 64-bit platforms X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2008 21:01:49 -0000 On Mon, Jan 21, 2008, Jung-uk Kim wrote: > On Saturday 19 January 2008 07:13 am, David Schultz wrote: > > For 8-CURRENT, I'd say go ahead and fix it and add the syscall > > compat goop. Then I guess backport it to 7-STABLE after a while if > > nothing important breaks and nobody has any objections to that > > plan. > > Well, since the policy is 'no seat belt for -CURRENT', I think that is > aceptable, too. Right, but you need the compat shims to avoid breaking the ABI when 8.0 is released anyway. > > It would also be cool if struct ipcperm got fixed at the same time > > so we only have to do this once. I think that only entails adding > > compat crud to semctl and maybe one or two other syscalls. Those > > changes aren't urgent enough to make it worthwhile to backport to > > 7.X IMO, though. > > So, you are in between option 1 and 4, i.e., bigger fix goes to HEAD > and partial MFC to RELENG_7, right? I'd say try to fix everything once and for all in HEAD, including struct ipcperm, but only MFC the shmid_ds changes to minimize the chance that anything breaks between 7.X releases. Others may have different opinions... > Actually, the arch check is > done like this: > > if (sizeof(size_t) != sizeof(int) && td->td_proc->p_osrel <= WHATEVER) > > I am hoping the compiler is smart enough to optimize away > 'sizeof(size_t) != sizeof(int)'. Yes, gcc does basic dead code elimination like this at all optimization levels, including -O0. Of course the code within the if block still has to be syntactically valid on all architectures; if it isn't, then you need ifdefs. > > - The 8.X version needs to take the p_osrel for the equivalent > > change in 7.X into account. > > I thought it did, didn't it? Oops, maybe you did and I was looking at one of the other versions of the patch. From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 21:22:52 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C74E016A52D; Mon, 21 Jan 2008 21:22:52 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 8D9BD13C4D3; Mon, 21 Jan 2008 21:22:52 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.2/8.14.2/NETPLEX) with ESMTP id m0LLA2Mm022834; Mon, 21 Jan 2008 16:10:02 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Mon, 21 Jan 2008 16:10:02 -0500 (EST) Date: Mon, 21 Jan 2008 16:10:02 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: David Schultz In-Reply-To: <20080121210145.GA9107@VARK.MIT.EDU> Message-ID: References: <200801181217.52788.jkim@FreeBSD.org> <20080119121316.GA21917@VARK.MIT.EDU> <200801211152.07680.jkim@FreeBSD.org> <20080121210145.GA9107@VARK.MIT.EDU> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org, Jung-uk Kim , freebsd-arch@freebsd.org Subject: Re: [RFC] SysV SHM on 64-bit platforms X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2008 21:22:52 -0000 On Mon, 21 Jan 2008, David Schultz wrote: > On Mon, Jan 21, 2008, Jung-uk Kim wrote: >> On Saturday 19 January 2008 07:13 am, David Schultz wrote: >>> For 8-CURRENT, I'd say go ahead and fix it and add the syscall >>> compat goop. Then I guess backport it to 7-STABLE after a while if >>> nothing important breaks and nobody has any objections to that >>> plan. >> >> Well, since the policy is 'no seat belt for -CURRENT', I think that is >> aceptable, too. > > Right, but you need the compat shims to avoid breaking the ABI > when 8.0 is released anyway. The compat shims should be committed at the same time as the actual change so that applications built on [4-7].x will continue to work. -- DE From owner-freebsd-arch@FreeBSD.ORG Tue Jan 22 17:47:05 2008 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D517816A418; Tue, 22 Jan 2008 17:47:05 +0000 (UTC) (envelope-from jkim@FreeBSD.org) Received: from anuket.mj.niksun.com (gwnew.niksun.com [65.115.46.162]) by mx1.freebsd.org (Postfix) with ESMTP id 88F0113C4DD; Tue, 22 Jan 2008 17:47:05 +0000 (UTC) (envelope-from jkim@FreeBSD.org) Received: from niksun.com (anuket [10.70.0.5]) by anuket.mj.niksun.com (8.13.6/8.13.6) with ESMTP id m0MHl4WK090477; Tue, 22 Jan 2008 12:47:04 -0500 (EST) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-current@FreeBSD.org Date: Tue, 22 Jan 2008 12:46:45 -0500 User-Agent: KMail/1.6.2 References: <200801181217.52788.jkim@FreeBSD.org> In-Reply-To: <200801181217.52788.jkim@FreeBSD.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200801221247.02342.jkim@FreeBSD.org> X-Virus-Scanned: ClamAV 0.92/5521/Tue Jan 22 09:55:56 2008 on anuket.mj.niksun.com X-Virus-Status: Clean Cc: freebsd-arch@FreeBSD.org Subject: Re: [RFC] SysV SHM on 64-bit platforms X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2008 17:47:06 -0000 On Friday 18 January 2008 12:17 pm, Jung-uk Kim wrote: > While I was working on kern/113218, I realized it is a bigger > problem than I originally thought: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=113218 > > Because fixing this PR breaks ABI, I had to make compat shims. But > the question is which branch to break? Since we are very close to > 7.0-RELEASE, the question became more complicated. A lot of people > are asking me to fix this PR because it is much needed fix for > large database installations, so I have to make a quick decision. > :-( FYI, re@ said "it's simply too late" for RELENG_7_0. Let's just shut up and fix ipc_perm and friends once and for all. ;-) Thank you all! Jung-uk Kim