From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 02:11:53 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBD6916A417; Sun, 2 Dec 2007 02:11:53 +0000 (UTC) (envelope-from obrien@NUXI.org) Received: from dragon.nuxi.org (trang.nuxi.org [74.95.12.85]) by mx1.freebsd.org (Postfix) with ESMTP id D018F13C4D1; Sun, 2 Dec 2007 02:11:53 +0000 (UTC) (envelope-from obrien@NUXI.org) Received: from dragon.nuxi.org (obrien@localhost [127.0.0.1]) by dragon.nuxi.org (8.14.1/8.14.1) with ESMTP id lB22BrUE055112; Sat, 1 Dec 2007 18:11:53 -0800 (PST) (envelope-from obrien@dragon.nuxi.org) Received: (from obrien@localhost) by dragon.nuxi.org (8.14.1/8.14.1/Submit) id lB22Bqmh055111; Sat, 1 Dec 2007 18:11:52 -0800 (PST) (envelope-from obrien) Date: Sat, 1 Dec 2007 18:11:52 -0800 From: "David O'Brien" To: Peter Jeremy Message-ID: <20071202021152.GA54329@dragon.NUXI.org> Mail-Followup-To: obrien@freebsd.org, Peter Jeremy , Daniel Eischen , freebsd-arch@freebsd.org References: <20071128211022.GA74762@lor.one-eyed-alien.net> <20071128213947.Q7555@fledge.watson.org> <20071129105133.GS50167@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071129105133.GS50167@server.vk2pj.dyndns.org> X-Operating-System: FreeBSD 7.0-CURRENT User-Agent: Mutt/1.5.16 (2007-06-09) Cc: Daniel Eischen , freebsd-arch@freebsd.org Subject: Re: RFC: libkse*.a in 7.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: obrien@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 02:11:54 -0000 On Thu, Nov 29, 2007 at 09:51:33PM +1100, Peter Jeremy wrote: > >I think we should remove libthr.a, libkse.a and libc.a, so flame on! > > Note that much of the toolchain is currently statically linked so > removing libc.a may expose some edge cases in buildworld/installworld. Yes. I don't care about removing threading libs .a's - that seems to make good sense. But I'd like for libc.a to remain. -- -- David (obrien@FreeBSD.org) From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 06:23:54 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA8CE16A417; Sun, 2 Dec 2007 06:23:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 6A24613C458; Sun, 2 Dec 2007 06:23:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.1/8.14.1) with ESMTP id lB26KOrh058917; Sat, 1 Dec 2007 23:20:24 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sat, 01 Dec 2007 23:23:42 -0700 (MST) Message-Id: <20071201.232342.-1622601489.imp@bsdimp.com> To: peter@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <200711302145.lAULj7Ob015371@repoman.freebsd.org> References: <200711302145.lAULj7Ob015371@repoman.freebsd.org> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org Subject: Re: cvs commit: src/sys/conf options.amd64 options.i386 src/sys/dev/sio sio_isa.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 06:23:54 -0000 In message: <200711302145.lAULj7Ob015371@repoman.freebsd.org> Peter Wemm writes: : peter 2007-11-30 21:45:07 UTC : : FreeBSD src repository : : Modified files: : sys/conf options.amd64 options.i386 : sys/dev/sio sio_isa.c : Log: : Allow the sio acpi attachment to be disabled (ie: use hints only). This : hack means you can get the units and flags to match up more easily with : serial consoles on machines with acpi tables that cause the com ports : to be probed in the wrong order (and hence get the wrong sio unit number). : : This replaces the common alternative hack of editing the code to comment : out the acpi attachment. This could go away entirely when device wiring : patches are committed. We should just commit Jhn's wiring hints. I know there are some objections to them that have been voiced here, but they are clearly better than what we have now and don't break anything. Once we get experience with them, we can know the real need for the advanced features that others have advocated for them. Warner From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 06:38:30 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4358216A474 for ; Sun, 2 Dec 2007 06:38:30 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mu-out-0910.google.com (mu-out-0910.google.com [209.85.134.185]) by mx1.freebsd.org (Postfix) with ESMTP id C287B13C468 for ; Sun, 2 Dec 2007 06:38:29 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by mu-out-0910.google.com with SMTP id i10so925659mue for ; Sat, 01 Dec 2007 22:38:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=llqoOijPjF4UjmBckqMgCiTZSW9nixnCmiaa9rTyOhY=; b=W9bejMeN6nOKgNiF1rhTbEBOHqnWoJOXpVHCZburoHIIseo6ser4V8ZZqtu6C/64eGxTSiirSqJeOxFZvEyOcLcVauaZ+xihlCrG6NaTKnZUt2Ua4uPfLjb5xZG3lu/gKYS8bowwu1dXV6Ari4ul0zEVmI64knpvWwiIycLUmso= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=JcaYXPGiz3HZzAiUogN/UwSK2ZEcMZoHWy9ASeWsdqoOOzVFtAINKcroL/YOmj5NOXbNXAnZbe61zyHo+haGtypPZ+ZD7dQqdAg22zb1ABiWDEAgLmDzPpMwaAl1MaZdlANidE9tsFJlshDeUaRfBCnV5hrYVt9ZXl3BGtmspaQ= Received: by 10.86.76.16 with SMTP id y16mr9155916fga.1196577080229; Sat, 01 Dec 2007 22:31:20 -0800 (PST) Received: by 10.86.28.19 with HTTP; Sat, 1 Dec 2007 22:31:20 -0800 (PST) Message-ID: <3bbf2fe10712012231p2945111cma2faed2299167d3a@mail.gmail.com> Date: Sun, 2 Dec 2007 07:31:20 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Poul-Henning Kamp" In-Reply-To: <15391.1196547545@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <15391.1196547545@critter.freebsd.dk> X-Google-Sender-Auth: 89ef2c22ac7d2426 Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 06:38:30 -0000 2007/12/1, Poul-Henning Kamp : > > Here is my proposed new timeout API for 8.x. > > The primary objective is to make it possible to have multiple timeout > "providers" of possibly different kind, so that we can have per-cpu > or per-net-stack timeout handing. I have a question so. Basically, do you plan to have in the so called "per-net-stack" timeout group all these timeouts which doesn't necessarilly want to benefit from CPU affinity? I would expect a tcp callout, which wants to exploit CPU affinity, to be in the "per-cpu0" group (0 is not a typo, it is expressing even on what CPU to run). Is this your idea also? I'm busy with my graduation process in these days, but I hope to came back to you with a better review of this header in a couple of days. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 08:14:47 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F3EE16A41B; Sun, 2 Dec 2007 08:14:47 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 2B85613C45D; Sun, 2 Dec 2007 08:14:46 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 6D24D17105; Sun, 2 Dec 2007 08:14:45 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB28Ei2E017367; Sun, 2 Dec 2007 08:14:45 GMT (envelope-from phk@critter.freebsd.dk) To: "Attilio Rao" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 07:31:20 +0100." <3bbf2fe10712012231p2945111cma2faed2299167d3a@mail.gmail.com> Date: Sun, 02 Dec 2007 08:14:44 +0000 Message-ID: <17366.1196583284@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 08:14:47 -0000 In message <3bbf2fe10712012231p2945111cma2faed2299167d3a@mail.gmail.com>, "Atti lio Rao" writes: >2007/12/1, Poul-Henning Kamp : >> >> Here is my proposed new timeout API for 8.x. >> >> The primary objective is to make it possible to have multiple timeout >> "providers" of possibly different kind, so that we can have per-cpu >> or per-net-stack timeout handing. > >I have a question so. I have no idea what the answer to your question is, I'm focusing on providing the ability, how we subsequently decide to use it is up to others. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 08:50:15 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7AEE316A46B for ; Sun, 2 Dec 2007 08:50:15 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mu-out-0910.google.com (mu-out-0910.google.com [209.85.134.189]) by mx1.freebsd.org (Postfix) with ESMTP id 9BFCF13C458 for ; Sun, 2 Dec 2007 08:50:14 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by mu-out-0910.google.com with SMTP id i10so1111772mue for ; Sun, 02 Dec 2007 00:50:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=zZLaDc+UWVVrS+gUNL+zttZNOLgw6jRNMs8P75m2EGk=; b=ohv3qJhznfdCjSwuyAWIahM7bYyVp92Byxeu++MI9sJ8JlDITHEqCaIZiTc4RoTBH1VmMq7BSC3d6uM20ujXCqbpuUhpeSBLCIEUQH5gJhAmsztUylQdL4K2/UniM3zdEQg1F7Xw7TNQD/S6PqJbt57rLqSDA5HZRBJvrxookL8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=bnciqqyMP4Y3fiL1wg6vlSsaJrmCyMz+/uGtFeQGFRroWAuk3NrZb+jKV2GEhK741GduTk5vmXafVhFRcGY/7mGKhbQG+HsLxYvX5o2iyWO2vURGuZHlBvwE7uilr5CSABWrBFLkzNqdCHc0ubKNCag24M6J0DNO60dG48/JqrQ= Received: by 10.86.51.2 with SMTP id y2mr4396971fgy.1196585412123; Sun, 02 Dec 2007 00:50:12 -0800 (PST) Received: by 10.86.25.3 with HTTP; Sun, 2 Dec 2007 00:50:12 -0800 (PST) Message-ID: <3bbf2fe10712020050p3ae55a30p84ff775dd6c103bd@mail.gmail.com> Date: Sun, 2 Dec 2007 09:50:12 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Poul-Henning Kamp" In-Reply-To: <17366.1196583284@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10712012231p2945111cma2faed2299167d3a@mail.gmail.com> <17366.1196583284@critter.freebsd.dk> X-Google-Sender-Auth: 597a5b4fca5cb8b9 Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 08:50:15 -0000 2007/12/2, Poul-Henning Kamp : > In message <3bbf2fe10712012231p2945111cma2faed2299167d3a@mail.gmail.com>, "Atti > lio Rao" writes: > >2007/12/1, Poul-Henning Kamp : > >> > >> Here is my proposed new timeout API for 8.x. > >> > >> The primary objective is to make it possible to have multiple timeout > >> "providers" of possibly different kind, so that we can have per-cpu > >> or per-net-stack timeout handing. > > > >I have a question so. > > I have no idea what the answer to your question is, I'm focusing on > providing the ability, how we subsequently decide to use it is up > to others. Well, this is part of that too. For things like "per-cpu" timeouts, I expect to have groups bound to the specific CPU you referr to. This kind of group is the only one where I can find a benefit about having a group. I'm not sure what would one expect by having, for example, a tcp-stack group as the generic implementation cannot do optimization on scheduling SWIs. What I'm trying to say is that the 'group' concept is good as an abstraction, but I still can't see what it can bring to us. What really matters about timeouts in SMP systems is how you expolit CPU affinity and how you balance timeouts between them. Doing this on a per-group basis is neither efficient or helpful really as these details should be decided on the consumers side of the matter. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 09:17:37 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A90A816A419; Sun, 2 Dec 2007 09:17:37 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 6613F13C455; Sun, 2 Dec 2007 09:17:37 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 2AC0817105; Sun, 2 Dec 2007 09:17:36 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB29HZiv017696; Sun, 2 Dec 2007 09:17:36 GMT (envelope-from phk@critter.freebsd.dk) To: "Attilio Rao" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 09:50:12 +0100." <3bbf2fe10712020050p3ae55a30p84ff775dd6c103bd@mail.gmail.com> Date: Sun, 02 Dec 2007 09:17:35 +0000 Message-ID: <17695.1196587055@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 09:17:37 -0000 In message <3bbf2fe10712020050p3ae55a30p84ff775dd6c103bd@mail.gmail.com>, "Atti lio Rao" writes: >2007/12/2, Poul-Henning Kamp : >> I have no idea what the answer to your question is, I'm focusing on >> providing the ability, how we subsequently decide to use it is up >> to others. > >Well, this is part of that too. >For things like "per-cpu" timeouts, I expect to have groups bound to >the specific CPU you referr to. >This kind of group is the only one where I can find a benefit about >having a group. I'm not sure what would one expect by having, for >example, a tcp-stack group as the generic implementation cannot do >optimization on scheduling SWIs. There are many possibilities we need to explore, but before we can do that, we need a mechanism that allows us to have multiple timeout providers. >What I'm trying to say is that the 'group' concept is good as an >abstraction, but I still can't see what it can bring to us. What >really matters about timeouts in SMP systems is how you expolit CPU >affinity and how you balance timeouts between them. Doing this on a >per-group basis is neither efficient or helpful really as these >details should be decided on the consumers side of the matter. There may be locking related optimizations to be had, or they may be a wash with the ekstra overhead. Time will show. Implementation wise there is no difference between enabling per-cpu timeouts, and enabling us to have any number of timeout providers, grouped or not. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 11:01:19 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3EE416A417; Sun, 2 Dec 2007 11:01:19 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id A480F13C442; Sun, 2 Dec 2007 11:01:18 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id BFD6617105; Sun, 2 Dec 2007 11:01:17 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2B1HZf018130; Sun, 2 Dec 2007 11:01:17 GMT (envelope-from phk@critter.freebsd.dk) To: Robert Watson From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 10:47:55 GMT." <20071202103833.N74097@fledge.watson.org> Date: Sun, 02 Dec 2007 11:01:17 +0000 Message-ID: <18129.1196593277@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Attilio Rao , arch@FreeBSD.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 11:01:20 -0000 In message <20071202103833.N74097@fledge.watson.org>, Robert Watson writes: >On Sun, 2 Dec 2007, Poul-Henning Kamp wrote: >> I have no idea what the answer to your question is, I'm focusing on >> providing the ability, how we subsequently decide to use it is up to others. > >Well, I think there is an important question to be discussed regarding >combinatorics, context switching, and the ability to provide multiple callout >threads. I still have no way to answer those questions. My aim here is to provide and implement an client API that will let us play with all those things. There are 444 .c or .h files in my src/sys which contains the word "callout". Obviously, getting the API right, so that we will not have to walk all these files once again is a very important point, and the only one I am trying to focus on right now. That is why the proposed API is only the client side. How we implement the timeout service, one which cpus and in which threads we execute the the timeout functions, and all other questions of that sort are not addressed, because I doubt anybody has a definitive answer to those questions yet. But if we get the API right, so that it can express the request from the client code precisely and concisely, then we can start to play with, benchmark and argue about how we execute timeouts. So please look at the proposed API only from the client side code for now. All the other questions are hidden behind the choice of first argument to timeout_init(). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 11:04:24 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75C9A16A419 for ; Sun, 2 Dec 2007 11:04:24 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 2ECF013C447 for ; Sun, 2 Dec 2007 11:04:24 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 3DA4447063; Sun, 2 Dec 2007 05:52:32 -0500 (EST) Date: Sun, 2 Dec 2007 10:47:55 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Poul-Henning Kamp In-Reply-To: <17366.1196583284@critter.freebsd.dk> Message-ID: <20071202103833.N74097@fledge.watson.org> References: <17366.1196583284@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Attilio Rao , arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 11:04:24 -0000 On Sun, 2 Dec 2007, Poul-Henning Kamp wrote: > In message <3bbf2fe10712012231p2945111cma2faed2299167d3a@mail.gmail.com>, "Atti > lio Rao" writes: >> 2007/12/1, Poul-Henning Kamp : >>> >>> Here is my proposed new timeout API for 8.x. >>> >>> The primary objective is to make it possible to have multiple timeout >>> "providers" of possibly different kind, so that we can have per-cpu or >>> per-net-stack timeout handing. >> >> I have a question so. > > I have no idea what the answer to your question is, I'm focusing on > providing the ability, how we subsequently decide to use it is up to others. Well, I think there is an important question to be discussed regarding combinatorics, context switching, and the ability to provide multiple callout threads. People have found the facility to provide their own worker threads and work pools surprisingly useful for taskqueue(9), so I find the concept of providing seperate callout wheels for different sorts of work appealing -- we could group, for example, high priority callouts in a separate thread from low priority callouts, avoiding priority inversion scenarions where high priority callouts in effect wait for low priority callouts due to the scheduling that occurs in callout(9) processing. However, this leads to a few concerns: - If we have several wheels in several threads, we risk significantly increasing the level of context switching if callouts exist in multiple wheels that fire at the same time intervals and same offsets. Today, those "context switches" occur in a single thread and don't require interacting with the system scheduler, saving a full stack, etc, and are effectively make callout handlers into co-routines. - There has been quite a bit of discussion about effectively slapping [MAXCPUS] onto the current callout wheel and lock, and starting up a callout thread per-CPU in order to allow workloads to be load-balanced. If no CPU preference is specified, then it lands on CPU 0 (or the like), and otherwise a consumer can request a preference to run the callout on a specific CPU. Good reasons to do this include avoiding lock contention by introducing affinities for workload, and load balancing for heavy callout users. I specifically have TCP in mind, needless to say, and it is one of our largest callout consumers. How would this strategy play out in the new infrastructure -- are you proposing TCP establish a thread and a group for each CPU, or is that a facility (affinity/CPU binding) that the timeout facility will provide for it, allowing TCP simply to express a CPU preference for a timeout when registering or rescheduling it? - For more naive users of the timeout facility, do you have any thinking on how we might load balance the timeouts as part of the facility you are designing? On busy systems, the callout thread can become quite a CPU hog, and it could be that transparent load balancing offers a benefit for consumers that are not aware of how to do their own load balancing. FWIW, I believe that in cases where we have a non-naive consumer, there are significant benefits to allowing it to manage its own balancing, as it can take into account data affinities, the potential for lock contention, etc. I have plans in the early 8.x development cycle to break down the pcbinfo locks and start balancing TCP work across CPUs via a weak affinity model (processing can happen on other CPUs, but we prefer not to for reasons of lock contention, cache cleanliness, etc). This in practice should also mean assigning the callouts for a TCP connection to run on the CPU it has an affinity for, for exactly the same reasons. This means that, one way or another, I need the ability to do this in the next three months, and I want to make sure that these plans are compatible with, and ideally facilitated by, any reworking of the callout facility. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 11:53:34 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A5DA16A417; Sun, 2 Dec 2007 11:53:34 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 326FC13C457; Sun, 2 Dec 2007 11:53:33 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id AC4B646B82; Sun, 2 Dec 2007 06:58:02 -0500 (EST) Date: Sun, 2 Dec 2007 11:53:25 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Andre Oppermann In-Reply-To: <4752998A.9030007@freebsd.org> Message-ID: <20071202115236.U74097@fledge.watson.org> References: <18129.1196593277@critter.freebsd.dk> <4752998A.9030007@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Attilio Rao , arch@FreeBSD.org, Poul-Henning Kamp Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 11:53:34 -0000 On Sun, 2 Dec 2007, Andre Oppermann wrote: > o TCP's data structure is exported to userspace and contains the > timeout data structures. This complicates timeout handling as > the data structure is not known to userland and we have to do > some hacks to prevent exposure. > > -> The timer facility should provide an opaque userland compat > header definition. I think it is important to do this, but I have some hopes that in 8 we'll have fixed up the muddle involving exported data structures. Obviously, that hasn't happened yet, and I had hoped to do it for 7... Perhaps this time one of us will succeed :-). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 11:58:06 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DB3116A41A; Sun, 2 Dec 2007 11:58:06 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id EAAF713C448; Sun, 2 Dec 2007 11:58:05 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id AF9E217105; Sun, 2 Dec 2007 11:58:04 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2Bw4L2018379; Sun, 2 Dec 2007 11:58:04 GMT (envelope-from phk@critter.freebsd.dk) To: Andre Oppermann From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 12:39:54 +0100." <4752998A.9030007@freebsd.org> Date: Sun, 02 Dec 2007 11:58:04 +0000 Message-ID: <18378.1196596684@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Attilio Rao , arch@freebsd.org, Robert Watson Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 11:58:06 -0000 In message <4752998A.9030007@freebsd.org>, Andre Oppermann writes: > o TCP maintains a number of concurrent, but hierarchical timers for > each session. What predominantly happens is a reschedule of an > existing timer, that means it wasn't close to firing and is moved > out again. This happens for every incoming segment. > > -> The timer facility should make it simple and efficient to move > the deadline into the future. That is more or less the reason for the multiple timescales (ns,us,ms,s) and the TIMEOUT_UNLIKELY flag. For long running timeouts that almost never happen, I don't want to even move them in a linked list when their time is moved further in the future. A timeout that is 60 seconds or more into the future can just be put on any random shelf until it gets a lot closer. The exact mechanism is TBD, but the intent is to not waste time on timeouts that are unlikely to happen, and to avoid them getting in the way of the timeouts that we do happen. > o TCP puts the timer into an allocated structure and upon close of the > session it has to be deallocated including stopping of all currently > running timers. > [...] > -> The timer facility should provide an atomic stop/remove call > that prevent any further callbacks upon return. It should not > do a 'drain' where the callback may be run anyway. > Note: We hold the lock the callback would have to obtain. It is my intent, that the implementation behind the new API will only ever grab the specified lock when it calls the timeout function. When you do a timeout_disable() or timeout_cleanup() you will be sleeping on a mutex internal to the implementation, if the timeout is currently executing. > o TCP has hot and cold CPU/cache affinity. > > -> The timer facility should provide strong, weak and "don't care" > CPU affinity. The affinity should be selected for a timer as > whole, not upon each call. That is the "timeout_p" you pass into timeout_init() is for. What values we will provide there is not decided, apart from NULL meaning "whatever..." > o TCP's data structure is exported to userspace and contains the > timeout data structures. This complicates timeout handling as > the data structure is not known to userland and we have to do > some hacks to prevent exposure. > > -> The timer facility should provide an opaque userland compat > header definition. I don't even want to expose its content to the client code, but I do want its size known at compile time. My current definition looks like: struct timeout { struct timeout_p *_prov; union { uintptr_t _timeout_private_i; void *_timeout_private_p; } _u[10]; }; (for some value of 10) I'm still playing with it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 12:02:52 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E0D116A421; Sun, 2 Dec 2007 12:02:52 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id CA2A613C459; Sun, 2 Dec 2007 12:02:51 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 6BA7517107; Sun, 2 Dec 2007 12:02:50 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2C2ndL018444; Sun, 2 Dec 2007 12:02:50 GMT (envelope-from phk@critter.freebsd.dk) From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 11:58:04 GMT." <18378.1196596684@critter.freebsd.dk> Date: Sun, 02 Dec 2007 12:02:49 +0000 Message-ID: <18443.1196596969@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Attilio Rao , arch@freebsd.org, Robert Watson , Andre Oppermann Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 12:02:52 -0000 In message <18378.1196596684@critter.freebsd.dk>, "Poul-Henning Kamp" writes: >> o TCP has hot and cold CPU/cache affinity. >> >> -> The timer facility should provide strong, weak and "don't care" >> CPU affinity. The affinity should be selected for a timer as >> whole, not upon each call. > >That is the "timeout_p" you pass into timeout_init() is for. > >What values we will provide there is not decided, apart from NULL >meaning "whatever..." I guess I need to elaborate that point some more: If we want CPU affinity, what happens that that we pass a per-cpu timeout provider: timeout_init(&pcpu->timouts, ...) If we want a private timeout group for NFS we pass that in: timeout_init(&nfs_timeouts, ...) Think of the implmentation of the timeouts as an object of which we can have multiple instances with various private properties... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 12:06:32 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D86416A421 for ; Sun, 2 Dec 2007 12:06:32 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id B0EF013C468 for ; Sun, 2 Dec 2007 12:06:31 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 15728 invoked from network); 2 Dec 2007 11:10:58 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 2 Dec 2007 11:10:58 -0000 Message-ID: <4752998A.9030007@freebsd.org> Date: Sun, 02 Dec 2007 12:39:54 +0100 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.13 (Windows/20070809) MIME-Version: 1.0 To: Poul-Henning Kamp References: <18129.1196593277@critter.freebsd.dk> In-Reply-To: <18129.1196593277@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Attilio Rao , arch@FreeBSD.org, Robert Watson Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 12:06:32 -0000 Poul-Henning Kamp wrote: > In message <20071202103833.N74097@fledge.watson.org>, Robert Watson writes: >> On Sun, 2 Dec 2007, Poul-Henning Kamp wrote: > >>> I have no idea what the answer to your question is, I'm focusing on >>> providing the ability, how we subsequently decide to use it is up to others. >> Well, I think there is an important question to be discussed regarding >> combinatorics, context switching, and the ability to provide multiple callout >> threads. > > I still have no way to answer those questions. > > My aim here is to provide and implement an client API that will let > us play with all those things. > > There are 444 .c or .h files in my src/sys which contains the word > "callout". > > Obviously, getting the API right, so that we will not have to walk > all these files once again is a very important point, and the only > one I am trying to focus on right now. For TCP the following features/properties would make the implementation much easier: o TCP maintains a number of concurrent, but hierarchical timers for each session. What predominantly happens is a reschedule of an existing timer, that means it wasn't close to firing and is moved out again. This happens for every incoming segment. -> The timer facility should make it simple and efficient to move the deadline into the future. o TCP puts the timer into an allocated structure and upon close of the session it has to be deallocated including stopping of all currently running timers. At the moment this is not really possible as callout_stop() is not atomic and the callout may already be waiting to be run on a lock. At the moment we just live with this race condition, apply some bandages and pray. Since this only happens on close and deallocation the operation may be more expensive than a normal timer stop call. Race conditions on normal timeout stops like stopping the delack timer are acceptable and can easily be handled with TCP. If it shows up after it was stopped we see it and just return. -> The timer facility should provide an atomic stop/remove call that prevent any further callbacks upon return. It should not do a 'drain' where the callback may be run anyway. Note: We hold the lock the callback would have to obtain. o TCP has hot and cold CPU/cache affinity. For certain timers we want to stay on the same CPU as it is very likely to still have the tcp control block in cache. The delayed ACK timer is the prime example running on some 100ms deadline. On the other hand timeouts farther away like the keepalive timer do not matter as there is almost zero chance that any CPU has it still around. Note: When we get NIC->CPU affinity we may want to keep all timeouts of a particular session always on the same CPU. -> The timer facility should provide strong, weak and "don't care" CPU affinity. The affinity should be selected for a timer as whole, not upon each call. o TCP's data structure is exported to userspace and contains the timeout data structures. This complicates timeout handling as the data structure is not known to userland and we have to do some hacks to prevent exposure. -> The timer facility should provide an opaque userland compat header definition. -- Andre From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 12:35:20 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E98716A46C; Sun, 2 Dec 2007 12:35:20 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 67F0213C465; Sun, 2 Dec 2007 12:35:20 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id D227C46F6E; Sun, 2 Dec 2007 07:39:49 -0500 (EST) Date: Sun, 2 Dec 2007 12:35:12 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Poul-Henning Kamp In-Reply-To: <18443.1196596969@critter.freebsd.dk> Message-ID: <20071202123231.G74097@fledge.watson.org> References: <18443.1196596969@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Attilio Rao , arch@freebsd.org, Andre Oppermann Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 12:35:20 -0000 On Sun, 2 Dec 2007, Poul-Henning Kamp wrote: > I guess I need to elaborate that point some more: > > If we want CPU affinity, what happens that that we pass a per-cpu timeout > provider: > > timeout_init(&pcpu->timouts, ...) > > If we want a private timeout group for NFS we pass that in: > > timeout_init(&nfs_timeouts, ...) > > Think of the implmentation of the timeouts as an object of which we can have > multiple instances with various private properties... The reason affinity is getting raised in particular is that quite a few people are running around thinking that affinity is something that they do want and plan to use. In the above, affinity is a property of the consumer in the event that the consumer has its own timeout instance. If it's a common property across many consmers, it's not something we want every consumer to deal with as it adds a significant degree of complexity to each consumer. I.e., rather than nfs_timeouts, it's pcpu.nfs_timeouts, or nfs_timeouts[cpu], leading to every consumer defining MAXCPUs or a related constant instances instead of a single instance that knows about affinity. Also, if there are multiple instances, that means "migrating" a callout from one CPU to another involves moving it from one instance to another, rather than simply asking the instance to move it in the way it understands best (which might involve less overhead). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 12:36:07 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3885B16A417 for ; Sun, 2 Dec 2007 12:36:07 +0000 (UTC) (envelope-from bushman@freebsd.org) Received: from mail.r61.net (mail.r61.net [195.208.245.249]) by mx1.freebsd.org (Postfix) with ESMTP id 9727D13C46B; Sun, 2 Dec 2007 12:36:06 +0000 (UTC) (envelope-from bushman@freebsd.org) Received: from [192.168.0.43] ([195.161.235.42]) (authenticated bits=0) by mail.r61.net (8.14.1/8.14.1) with ESMTP id lB2C8gtH065999 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sun, 2 Dec 2007 15:08:44 +0300 (MSK) (envelope-from bushman@freebsd.org) Message-Id: <5555F136-D396-4333-837D-C4924416C3CB@freebsd.org> From: Michael Bushkov To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Date: Sun, 2 Dec 2007 15:08:37 +0300 X-Mailer: Apple Mail (2.915) Cc: de@freebsd.org Subject: libc-scoped function and Symbol.map X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 12:36:07 -0000 Hi! If I add the internal libc-scoped function (named __getgroupmembership) to src/lib/libc/gen/getgrent.c, I need to add its name to Symbol.map, don't I? The function definition is: int __getgroupmembership(const char *uname, gid_t agroup, gid_t *groups, int maxgrp, int *grpcnt) The Symbol.map diff is: --- lib/libc/gen/Symbol.map 31 May 2007 13:01:33 -0000 1.6 +++ lib/libc/gen/Symbol.map 23 Oct 2007 14:25:53 -0000 @@ -337,6 +337,8 @@ }; FBSDprivate_1.0 { + __getgroupmembership; + /* needed by thread libraries */ __thr_jtable; With best regards, Michael Bushkov, Southern Federal University From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 12:50:24 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6024B16A418; Sun, 2 Dec 2007 12:50:24 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 1EA3513C447; Sun, 2 Dec 2007 12:50:23 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id C7FC617105; Sun, 2 Dec 2007 12:50:22 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2CoM3r018584; Sun, 2 Dec 2007 12:50:22 GMT (envelope-from phk@critter.freebsd.dk) To: Robert Watson From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 12:35:12 GMT." <20071202123231.G74097@fledge.watson.org> Date: Sun, 02 Dec 2007 12:50:22 +0000 Message-ID: <18583.1196599822@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Attilio Rao , arch@FreeBSD.org, Andre Oppermann Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 12:50:24 -0000 In message <20071202123231.G74097@fledge.watson.org>, Robert Watson writes: >On Sun, 2 Dec 2007, Poul-Henning Kamp wrote: >The reason affinity is getting raised in particular is that quite a few people >are running around thinking that affinity is something that they do want and >plan to use. That's fine and good and all. But before we can play with that sort of stuff, we need some kind of instance handle on the timeout to express cpu affinity to/with. We also need to losse Hz from this API, for a large number of reasons, from efficiency to precision. And we need to get rid of the 20+ lines of "cleanup my callout" code that is infecting more and more code. This API redesign tries to address those three major problems, and getting that right is important because there are 444 sourcefiles to visit. If we find later on that we need to add timeout_fiddle_cpu_affinity(), we can add that, touching only two or three files, so that is two orders of magnitude less interesting right now. The important thing to look at this API, is that it should be able to express our intent, so that we should never need to visit all the 444 files ever again. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 12:53:14 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8220F16A419 for ; Sun, 2 Dec 2007 12:53:14 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id DFC4013C46E for ; Sun, 2 Dec 2007 12:53:13 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 16491 invoked from network); 2 Dec 2007 12:24:21 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 2 Dec 2007 12:24:21 -0000 Message-ID: <4752AABE.6090006@freebsd.org> Date: Sun, 02 Dec 2007 13:53:18 +0100 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.13 (Windows/20070809) MIME-Version: 1.0 To: Poul-Henning Kamp References: <18378.1196596684@critter.freebsd.dk> In-Reply-To: <18378.1196596684@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Attilio Rao , arch@freebsd.org, Robert Watson Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 12:53:14 -0000 Poul-Henning Kamp wrote: > In message <4752998A.9030007@freebsd.org>, Andre Oppermann writes: >> o TCP puts the timer into an allocated structure and upon close of the >> session it has to be deallocated including stopping of all currently >> running timers. >> [...] >> -> The timer facility should provide an atomic stop/remove call >> that prevent any further callbacks upon return. It should not >> do a 'drain' where the callback may be run anyway. >> Note: We hold the lock the callback would have to obtain. > > It is my intent, that the implementation behind the new API will > only ever grab the specified lock when it calls the timeout function. This is the same for the current one and pretty much a given. > When you do a timeout_disable() or timeout_cleanup() you will be > sleeping on a mutex internal to the implementation, if the timeout > is currently executing. This is the problematic part. We can't sleep in TCP when cleaning up the timer. We're not always called from userland but from interrupt context. And when calling the cleanup we currently hold the lock the callout wants to obtain. We can't drop it either as the race would be back again. What you describe here is the equivalent of callout_ drain(). This is unfortunately unworkable in TCP's context. The callout has to go away even if it is already pending and waiting on the lock. Maybe that can only be solved by a flag in the lock saying "give up and go away". -- Andre From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 13:19:41 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B022916A418 for ; Sun, 2 Dec 2007 13:19:41 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id 85FE413C447 for ; Sun, 2 Dec 2007 13:19:41 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id lB2CpZPs007701; Sun, 2 Dec 2007 04:51:35 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id lB2CpZKY007700; Sun, 2 Dec 2007 04:51:35 -0800 (PST) (envelope-from rizzo) Date: Sun, 2 Dec 2007 04:51:35 -0800 From: Luigi Rizzo To: Poul-Henning Kamp Message-ID: <20071202045134.A7421@xorpc.icir.org> References: <15391.1196547545@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <15391.1196547545@critter.freebsd.dk>; from phk@phk.freebsd.dk on Sat, Dec 01, 2007 at 10:19:05PM +0000 Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 13:19:41 -0000 On Sat, Dec 01, 2007 at 10:19:05PM +0000, Poul-Henning Kamp wrote: > > Here is my proposed new timeout API for 8.x. ... remaining limited to the client API, I have the following questions: > /* > * A duration of time, represented in the optimal way for a given provider > * or family of providers (ie: per cpu). > */ > typedef int timeout_time; is this meant to be parsable by client code, or should it be opaque (even though of known size) ? > /* > * Flag values, > * can be used as return from the function or as argument to timeout_arm() > */ > #define TIMEOUT_REARM (1<<0) ... any reason not to use an enum for these ? > /* > * Convert various human compatible time-units to internal units > * Since these are potentially expensive (as in multiple integer divisions) > * caching the return value is adviced for heavy use. > * Choice of function indicates level of precision requested, so > * timeout_ms_time(1000) may be different from timeout_s_time(1), depending > * on the implementation. > */ > timeout_time timeout_ns_time(struct timeout_p *, unsigned nsec); > timeout_time timeout_us_time(struct timeout_p *, unsigned usec); > timeout_time timeout_ms_time(struct timeout_p *, unsigned msec); > timeout_time timeout_s_time(struct timeout_p *, unsigned sec); Two questions here: 1. is there any need for the first argument not to be const ? If all the function do is a conversion, there shouldn't be any need to modify the timeout_p 2. I fully agree that the precision level should be a user-supplied parameter, but wonder whether ns/us/ms/s is really the set of precisions one might want. I could see a need for at least the following requests: "as accurate as possible" "one timecounter tick accuracy, whatever the tick is" "one timer tick (1/HZ) accuracy, whatever the tick is" " So how about moving to a slightly different API to convert timeouts back and forth between Human and Internal representation e.g timeout_time timeout_UtoI(const struct timeout_p *, int units, enum timeout_precision); with enum timeout_precision { TP_NANOSECOND, TP_MICROSECOND, TP_MILLISECOND, TP_SECOND, TP_DEFAULT, /* default: units is us */ TP_HIGHEST, /* highest possible; units is in ns */ TP_TIMECOUNTER, /* one timecounter tick */ TP_TIMECOUNTER, /* one timecounter tick */ ... } The 'unit' argument is in whatever unit applies to the precision, i.e. ns, us, ms, ticks, ... when obvious, or we need to specify a suitable value for TP_DEFAULT and TP_HIGHEST We might also make use of the inverse conversion, something like int timeout_ItoU(const struct timeout_p *, timeout_time t, enum timeout_precision); cheers luigi From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 13:25:20 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94AF816A46B; Sun, 2 Dec 2007 13:25:20 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 4FEB713C4E5; Sun, 2 Dec 2007 13:25:20 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 1209417106; Sun, 2 Dec 2007 13:25:19 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2DPFJk018720; Sun, 2 Dec 2007 13:25:15 GMT (envelope-from phk@critter.freebsd.dk) To: Andre Oppermann From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 13:53:18 +0100." <4752AABE.6090006@freebsd.org> Date: Sun, 02 Dec 2007 13:25:15 +0000 Message-ID: <18719.1196601915@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Attilio Rao , arch@freebsd.org, Robert Watson Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 13:25:20 -0000 In message <4752AABE.6090006@freebsd.org>, Andre Oppermann writes: >> It is my intent, that the implementation behind the new API will >> only ever grab the specified lock when it calls the timeout function. > >This is the same for the current one and pretty much a given. > >> When you do a timeout_disable() or timeout_cleanup() you will be >> sleeping on a mutex internal to the implementation, if the timeout >> is currently executing. > >This is the problematic part. We can't sleep in TCP when cleaning up >the timer. The trouble arises because the current callout implementation will try to sleep on the timeouts lock, and once it does that, you cannot cancel it any more. I'm going to exchange that problem for once that is less severe. My plan is to use non-blocking grabs of the timeouts lock to get around that race. When a timeouts timer expires, the thread that services the timeouts will try to get the lock in a non-blocking fashion, and if it fails, be put on a queue, to be retried after any other expired timeouts have had their chance. That leaves only the question of "how hard to we try to get the lock with non-blocking means". The answer to that will depend on how big a problem it is in practice. Adding timeout_cleanup() as an explicit end of life indicator for the timeout structure and its lock, makes it possible to use blocking methods, at high expense, in those rare cases where non-blocking means keeps failing. But lets hope we will not need that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 13:32:39 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E71B16A469 for ; Sun, 2 Dec 2007 13:32:39 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 041B013C4EC for ; Sun, 2 Dec 2007 13:32:38 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id CE70F17106; Sun, 2 Dec 2007 13:32:37 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2DWbIT018792; Sun, 2 Dec 2007 13:32:37 GMT (envelope-from phk@critter.freebsd.dk) To: Luigi Rizzo From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 04:51:35 PST." <20071202045134.A7421@xorpc.icir.org> Date: Sun, 02 Dec 2007 13:32:37 +0000 Message-ID: <18791.1196602357@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 13:32:39 -0000 In message <20071202045134.A7421@xorpc.icir.org>, Luigi Rizzo writes: >On Sat, Dec 01, 2007 at 10:19:05PM +0000, Poul-Henning Kamp wrote: >> /* >> * A duration of time, represented in the optimal way for a given provider >> * or family of providers (ie: per cpu). >> */ >> typedef int timeout_time; > >is this meant to be parsable by client code, or should it be >opaque (even though of known size) ? Opaque to clients. It is likely to be scaled to some hardware counters clock period, but that is internal to the provider. >> /* >> * Flag values, >> * can be used as return from the function or as argument to timeout_arm() >> */ >> #define TIMEOUT_REARM (1<<0) >... >any reason not to use an enum for these ? Enums make static code checkers barf when your OR their values. C doesn't have bitmapts, which is a strange oversight. >> timeout_time timeout_ns_time(struct timeout_p *, unsigned nsec); >> timeout_time timeout_us_time(struct timeout_p *, unsigned usec); >> timeout_time timeout_ms_time(struct timeout_p *, unsigned msec); >> timeout_time timeout_s_time(struct timeout_p *, unsigned sec); > >Two questions here: >1. is there any need for the first argument not to be const ? > If all the function do is a conversion, there shouldn't be any > need to modify the timeout_p No, it can probably be const. I think I have also convinced myself to change this to be one function with a scaling specified: #define TIMEOUT_NSEC (1 << 9) #define TIMEOUT_USEC (1 << 6) #define TIMEOUT_MSEC (1 << 3) #define TIMEOUT_SEC (1 << 0) timeout_time timeout_conv_time(struct timeout_p *, unsigned val, unsigned scale) ; >2. I fully agree that the precision level should be a user-supplied > parameter, but wonder whether ns/us/ms/s is really the set of > precisions one might want. I could see a need for at least the > following requests: > > "as accurate as possible" > "one timecounter tick accuracy, whatever the tick is" > "one timer tick (1/HZ) accuracy, whatever the tick is" > " Ticks may cease to have importance with hardware like the HPET where we can implement deadlining instead of periodic scheduling. It may make sense to offer ranges or tolerances, but that can be added later if we find that we need it. The goal here and now, in this area of the API, is only to kill HZ as a specification unit of time. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 13:51:48 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53DE616A418 for ; Sun, 2 Dec 2007 13:51:48 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id 415F213C4D9 for ; Sun, 2 Dec 2007 13:51:47 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id lB2DoVg9008303; Sun, 2 Dec 2007 05:50:31 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id lB2DoVlF008302; Sun, 2 Dec 2007 05:50:31 -0800 (PST) (envelope-from rizzo) Date: Sun, 2 Dec 2007 05:50:31 -0800 From: Luigi Rizzo To: Poul-Henning Kamp Message-ID: <20071202055031.A8107@xorpc.icir.org> References: <20071202045134.A7421@xorpc.icir.org> <18791.1196602357@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <18791.1196602357@critter.freebsd.dk>; from phk@phk.freebsd.dk on Sun, Dec 02, 2007 at 01:32:37PM +0000 Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 13:51:48 -0000 On Sun, Dec 02, 2007 at 01:32:37PM +0000, Poul-Henning Kamp wrote: > In message <20071202045134.A7421@xorpc.icir.org>, Luigi Rizzo writes: ... > I think I have also convinced myself to change this to be one > function with a scaling specified: > > #define TIMEOUT_NSEC (1 << 9) > #define TIMEOUT_USEC (1 << 6) > #define TIMEOUT_MSEC (1 << 3) > #define TIMEOUT_SEC (1 << 0) > timeout_time timeout_conv_time(struct timeout_p *, unsigned val, unsigned scale) ; > > > >2. I fully agree that the precision level should be a user-supplied > > parameter, but wonder whether ns/us/ms/s is really the set of > > precisions one might want. I could see a need for at least the > > following requests: > > > > "as accurate as possible" > > "one timecounter tick accuracy, whatever the tick is" > > "one timer tick (1/HZ) accuracy, whatever the tick is" > > " > > Ticks may cease to have importance with hardware like the HPET > where we can implement deadlining instead of periodic scheduling. > > It may make sense to offer ranges or tolerances, but that can be > added later if we find that we need it. > > The goal here and now, in this area of the API, is only to kill HZ > as a specification unit of time. I suppose the goal is to let the program express its real requirements as opposed to "requirements with hardwired assumptions on the timer implementation". So seconds and microseconds etc. are fine in some cases (e.g. playing with hardware, or protocol timeouts, etc.). But in other cases the periodic scheduling is really an application requirement and not an artifact of the hardware - e.g. think of the many drivers or applications that need to do some periodic polling because the hardware doesn't really provide a synchronization signal. In those cases you want to synchronize to what the underlying period is, rather than hardwire your own estimate in the code. This is why i suggest having a 'scale' that can represent '1 tick' (and also don't depend on TIMEOUT_MSEC == 1000 and so on, but keep them opaque and require that the client code uses one of the supported scales). cheers luigi From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 13:55:40 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B96816A417 for ; Sun, 2 Dec 2007 13:55:40 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 64C3513C468 for ; Sun, 2 Dec 2007 13:55:39 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 17232 invoked from network); 2 Dec 2007 13:26:46 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 2 Dec 2007 13:26:46 -0000 Message-ID: <4752B95F.20308@freebsd.org> Date: Sun, 02 Dec 2007 14:55:43 +0100 From: Andre Oppermann User-Agent: Thunderbird 1.5.0.13 (Windows/20070809) MIME-Version: 1.0 To: Poul-Henning Kamp References: <18719.1196601915@critter.freebsd.dk> In-Reply-To: <18719.1196601915@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Attilio Rao , arch@freebsd.org, Robert Watson Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 13:55:40 -0000 Poul-Henning Kamp wrote: > In message <4752AABE.6090006@freebsd.org>, Andre Oppermann writes: > >>> It is my intent, that the implementation behind the new API will >>> only ever grab the specified lock when it calls the timeout function. >> This is the same for the current one and pretty much a given. >> >>> When you do a timeout_disable() or timeout_cleanup() you will be >>> sleeping on a mutex internal to the implementation, if the timeout >>> is currently executing. >> This is the problematic part. We can't sleep in TCP when cleaning up >> the timer. > > The trouble arises because the current callout implementation will > try to sleep on the timeouts lock, and once it does that, you cannot > cancel it any more. It hurts us big time in the TCP code. > I'm going to exchange that problem for once that is less severe. > > My plan is to use non-blocking grabs of the timeouts lock to get > around that race. > > When a timeouts timer expires, the thread that services the timeouts > will try to get the lock in a non-blocking fashion, and if it fails, > be put on a queue, to be retried after any other expired timeouts > have had their chance. In TCP we've got two types of races: o Timer expires on active session but source of timer was just handled (because segment just arrived). To simplify detection of timer races some generation count passed together with the timer may be of value. That way I (or the timer code) can easily detect if this invocation of the callback has become obsolete. o On shutdown we have to get rid of all timers for sure because once we release the lock it is immediately destroyed and the memory is freed and cleared. There is no way the timer must even try to look at the lock again. This is our major problem child in the TCP and socket lifecycle code. There is another fine line. When doing a timer cleanup do I get to know if there is a timeout pending and waiting in the CPU queue? In other words can timeout_cleanup() tell us with certainty that a timeout is no longer active and/or pending? This would help us half way. Other than that is a flag planned saying "try only once" to obtain the lock? This may help the first race. Though the current TCP code is not structured to work that way it could move in that direction. > That leaves only the question of "how hard to we try to get the lock > with non-blocking means". > > The answer to that will depend on how big a problem it is in practice. > > Adding timeout_cleanup() as an explicit end of life indicator for > the timeout structure and its lock, makes it possible to use blocking > methods, at high expense, in those rare cases where non-blocking > means keeps failing. > > But lets hope we will not need that. -- Andre From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:08:44 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04DD516A418 for ; Sun, 2 Dec 2007 15:08:44 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id BA6DB13C478 for ; Sun, 2 Dec 2007 15:08:43 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 02C0617105; Sun, 2 Dec 2007 15:08:41 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2F8fAm019257; Sun, 2 Dec 2007 15:08:41 GMT (envelope-from phk@critter.freebsd.dk) To: Luigi Rizzo From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 05:50:31 PST." <20071202055031.A8107@xorpc.icir.org> Date: Sun, 02 Dec 2007 15:08:41 +0000 Message-ID: <19256.1196608121@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:08:44 -0000 In message <20071202055031.A8107@xorpc.icir.org>, Luigi Rizzo writes: >This is why i suggest having a 'scale' that can represent '1 tick' >(and also don't depend on TIMEOUT_MSEC == 1000 and so on, but keep >them opaque and require that the client code uses one of the supported >scales). Using a deadline timer based in the HPET, the timeout can be scheduled to any 1/14318181th of a second and there will be no concept of "a tick" as we know it now. Clients should say how often they want to be called, and they should express it in terms of time, not based on some implementation detail of a historical implementation of the scheduler. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:16:15 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6630716A417; Sun, 2 Dec 2007 15:16:15 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 21BF613C47E; Sun, 2 Dec 2007 15:16:15 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 1D0BD17105; Sun, 2 Dec 2007 15:16:13 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2FGDLK019296; Sun, 2 Dec 2007 15:16:13 GMT (envelope-from phk@critter.freebsd.dk) To: Andre Oppermann From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 14:55:43 +0100." <4752B95F.20308@freebsd.org> Date: Sun, 02 Dec 2007 15:16:13 +0000 Message-ID: <19295.1196608573@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Attilio Rao , arch@freebsd.org, Robert Watson Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:16:15 -0000 In message <4752B95F.20308@freebsd.org>, Andre Oppermann writes: >> The trouble arises because the current callout implementation will >> try to sleep on the timeouts lock, and once it does that, you cannot >> cancel it any more. > >It hurts us big time in the TCP code. I know, that's one of the major inspirations for this work. >In TCP we've got two types of races: > > o Timer expires on active session but source of timer was just > handled (because segment just arrived). To simplify detection of > timer races some generation count passed together with the timer > may be of value. That way I (or the timer code) can easily detect > if this invocation of the callback has become obsolete. You shouldn't need to do that if my schme works, you can just reschedule the timeout, and it will be rescheduled - _also_ if its time has expired and it is ready to run, once its lock gets freed. > o On shutdown we have to get rid of all timers for sure because once > we release the lock it is immediately destroyed and the memory is > freed and cleared. There is no way the timer must even try to look > at the lock again. This is our major problem child in the TCP and > socket lifecycle code. That should also work ok because as long as you hold the lock, you can safely arm, disable or cleanup the timeout. >There is another fine line. When doing a timer cleanup do I get to >know if there is a timeout pending and waiting in the CPU queue? In >other words can timeout_cleanup() tell us with certainty that a timeout >is no longer active and/or pending? This would help us half way. I'm not sure I understand exactly what you ask about here, do you want to know if there are armed timeouts or do you want to know if they expired ? Either way, the info is there... >Other than that is a flag planned saying "try only once" to obtain >the lock? This may help the first race. Though the current TCP >code is not structured to work that way it could move in that direction. More flags is certainly possible, and we can experiment with which. What I'm looking for right now is people telling me "you obviously need a pointer to a X-mas tree in the timeout_foo() function" before I start on the 444 files... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:22:46 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AA7616A418 for ; Sun, 2 Dec 2007 15:22:46 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.178]) by mx1.freebsd.org (Postfix) with ESMTP id 6663813C467 for ; Sun, 2 Dec 2007 15:22:46 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so4614210waf for ; Sun, 02 Dec 2007 07:22:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer:mime-version:content-type; bh=H3PQO3AyDUbxOC5KGstWY+j3aKyPGIAMlDbia8s2Xe0=; b=jtXCQht95N98mRSzBfW/5LiERElpHVFal0/bZ8Sx1I19kNSYB5Ul2QhwZHILmCZAalshDTC81FG6sL4wKwxjd5PzuH31T0LqwCnnr1gwNTueezGGmKKpYsx50s45uOPfJoA5gfACXt5varyhm1HueDa/PrH1XhYCkzVUNukFQek= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=received:date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer:mime-version:content-type; b=aexfwjSR5FtI1NqCSaIU9K1yDHL8Vvbo4Mo6fSrUh2L/WGBrw2UneID+iCDj7ahbJslDbUh/dkLPLS/1zqUH+rNzpHoTBZEIyVqEfYQADOB5Q3RiqJLumj1IvMf/23M5tCI7XzKet3cG+oAB2+T3F8yl1Oj9F4QuqNb4uVvmPdU= Received: by 10.142.185.13 with SMTP id i13mr1338791wff.1196608966064; Sun, 02 Dec 2007 07:22:46 -0800 (PST) Received: from kan.dnsalias.net ( [24.218.183.247]) by mx.google.com with ESMTPS id 34sm11638135wra.2007.12.02.07.22.44 (version=SSLv3 cipher=OTHER); Sun, 02 Dec 2007 07:22:45 -0800 (PST) Date: Sun, 2 Dec 2007 10:22:38 -0500 From: Alexander Kabaev To: Michael Bushkov Message-ID: <20071202102238.0ad19c7b@kan.dnsalias.net> In-Reply-To: <5555F136-D396-4333-837D-C4924416C3CB@freebsd.org> References: <5555F136-D396-4333-837D-C4924416C3CB@freebsd.org> X-Mailer: Claws Mail 3.0.2 (GTK+ 2.12.1; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/oaTylQIbjG_Q54SeMk_kmer"; protocol="application/pgp-signature"; micalg=PGP-SHA1 Cc: de@freebsd.org, freebsd-arch@freebsd.org Subject: Re: libc-scoped function and Symbol.map X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:22:46 -0000 --Sig_/oaTylQIbjG_Q54SeMk_kmer Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Sun, 2 Dec 2007 15:08:37 +0300 Michael Bushkov wrote: > Hi! > If I add the internal libc-scoped function (named =20 > __getgroupmembership) to src/lib/libc/gen/getgrent.c, I need to add =20 > its name to Symbol.map, don't I? >=20 > The function definition is: > int > __getgroupmembership(const char *uname, gid_t agroup, gid_t *groups, > int maxgrp, int *grpcnt) >=20 > The Symbol.map diff is: > --- lib/libc/gen/Symbol.map 31 May 2007 13:01:33 -0000 > 1.6 +++ lib/libc/gen/Symbol.map 23 Oct 2007 14:25:53 -0000 > @@ -337,6 +337,8 @@ > }; > FBSDprivate_1.0 { > + __getgroupmembership; > + > /* needed by thread libraries */ > __thr_jtable; > If the function is only meant to be available within libc itself, you do not need to add it anywhere. --=20 Alexander Kabaev --Sig_/oaTylQIbjG_Q54SeMk_kmer Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHUs2+Q6z1jMm+XZYRAvzcAKDD3w6RSqhNBbphk/R5ulUZgSyUPwCgw34+ 3m1ehTikE4U1PeVnEQptuag= =07vF -----END PGP SIGNATURE----- --Sig_/oaTylQIbjG_Q54SeMk_kmer-- From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:28:15 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4121716A417 for ; Sun, 2 Dec 2007 15:28:15 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id 2AC1D13C4F4 for ; Sun, 2 Dec 2007 15:28:15 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id lB2FQxxh009376; Sun, 2 Dec 2007 07:26:59 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id lB2FQwfI009375; Sun, 2 Dec 2007 07:26:58 -0800 (PST) (envelope-from rizzo) Date: Sun, 2 Dec 2007 07:26:58 -0800 From: Luigi Rizzo To: Poul-Henning Kamp Message-ID: <20071202072658.A9217@xorpc.icir.org> References: <20071202055031.A8107@xorpc.icir.org> <19256.1196608121@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <19256.1196608121@critter.freebsd.dk>; from phk@phk.freebsd.dk on Sun, Dec 02, 2007 at 03:08:41PM +0000 Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:28:15 -0000 On Sun, Dec 02, 2007 at 03:08:41PM +0000, Poul-Henning Kamp wrote: > In message <20071202055031.A8107@xorpc.icir.org>, Luigi Rizzo writes: > > > >This is why i suggest having a 'scale' that can represent '1 tick' > >(and also don't depend on TIMEOUT_MSEC == 1000 and so on, but keep > >them opaque and require that the client code uses one of the supported > >scales). > > > Using a deadline timer based in the HPET, the timeout can be scheduled > to any 1/14318181th of a second This must obviously come for a price or there would be no reason to provide 'millisecond' and 'second' precisions that you suggested. I agree that the API should not assume that the hardware is based on a periodic timer, but in the same way it should not assume that the hardware can generate arbitrary timeouts. Secondy: > and there will be no concept of "a > tick" as we know it now. > Clients should say how often they want to be called, and they should > express it in terms of time, not based on some implementation detail > of a historical implementation of the scheduler. "periodically with whatever period is convenient to you" is a legitimate request that client code might have. (I understand that this concept of time is very southern european :) It wouldn't make sense to do poll-based event handling at the same rate on a 133MHz soekris or on a 3 GHz multicore cpu. Even if you kick HZ out of your API, it will come back in some indirect form. In any case: in terms of API it costs absolutely nothing to support system-dependent resolutions instead of absolute ones, so i don't quite understand the opposition. cheers luigi From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:34:52 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FD3916A419 for ; Sun, 2 Dec 2007 15:34:52 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 0180513C448 for ; Sun, 2 Dec 2007 15:34:51 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 6DA9917105; Sun, 2 Dec 2007 15:34:50 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2FYnEk019519; Sun, 2 Dec 2007 15:34:49 GMT (envelope-from phk@critter.freebsd.dk) To: Luigi Rizzo From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 07:26:58 PST." <20071202072658.A9217@xorpc.icir.org> Date: Sun, 02 Dec 2007 15:34:49 +0000 Message-ID: <19518.1196609689@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:34:52 -0000 In message <20071202072658.A9217@xorpc.icir.org>, Luigi Rizzo writes: >On Sun, Dec 02, 2007 at 03:08:41PM +0000, Poul-Henning Kamp wrote: >> Using a deadline timer based in the HPET, the timeout can be scheduled >> to any 1/14318181th of a second > >This must obviously come for a price or there would be no reason to >provide 'millisecond' and 'second' precisions that you suggested. >I agree that the API should not assume that the hardware is based on >a periodic timer, but in the same way it should not assume that >the hardware can generate arbitrary timeouts. If the client code needs this finegrained control, the trick is to extend the API to be able to provide this information somehow. But before designing that, we need to have concrete examples to talk about. >"periodically with whatever period is convenient to you" is >a legitimate request that client code might have. >(I understand that this concept of time is very southern european :) I disagree, that case, should be "with a period between N and M, at your choice". And yes, it may be that we need to extend the API for that, but please give a concrete example so we know what we're talking about. >It wouldn't make sense to do poll-based event handling at the same >rate on a 133MHz soekris or on a 3 GHz multicore cpu. Hz is the same for those two, so today we poll at the same rate. >In any case: in terms of API it costs absolutely nothing to support >system-dependent resolutions instead of absolute ones, so >i don't quite understand the opposition. I'm applying Jim Gettys 3. rule from the X11 project: 3.The only thing worse than generalizing from one example is generalizing from no examples at all. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:48:01 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BD2C16A41A for ; Sun, 2 Dec 2007 15:48:01 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id DBAFE13C442 for ; Sun, 2 Dec 2007 15:48:00 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.1/8.14.1) with ESMTP id lB2FhPRA070754; Sun, 2 Dec 2007 08:43:26 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sun, 02 Dec 2007 08:46:48 -0700 (MST) Message-Id: <20071202.084648.-108809549.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <15391.1196547545@critter.freebsd.dk> References: <15391.1196547545@critter.freebsd.dk> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:48:01 -0000 In message: <15391.1196547545@critter.freebsd.dk> Poul-Henning Kamp writes: : Here is my proposed new timeout API for 8.x. : : The primary objective is to make it possible to have multiple timeout : "providers" of possibly different kind, so that we can have per-cpu : or per-net-stack timeout handing. : : A secondary goal, is to shove the anti-race handling in destruction of : timeouts back into the implemenation, rather than force users to spend : 20+ lines doing that. : : A third goal is to enable deadline scheduling of timeouts using hardware : like HPET when and where we have it. : : Comments in general (only B/W) and pointers to client code of : particular interest is most welcome. >>/* >> * A duration of time, represented in the optimal way for a given provider >> * or family of providers (ie: per cpu). >> */ >>typedef int timeout_time; What does this mean? How do I get one of those? Without a unit associated with this number, it becomes hard to do a "tickless" implementation. Right now we have a well defined unit (1/hz). I worry that without having a way to convert time to this thing that we'll have more trouble writing drivers. Warner From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:54:41 2007 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B53316A420 for ; Sun, 2 Dec 2007 15:54:41 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 468AD13C44B for ; Sun, 2 Dec 2007 15:54:41 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.1/8.14.1) with ESMTP id lB2FnrLf070803; Sun, 2 Dec 2007 08:49:53 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sun, 02 Dec 2007 08:53:16 -0700 (MST) Message-Id: <20071202.085316.723205116.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <19256.1196608121@critter.freebsd.dk> References: <20071202055031.A8107@xorpc.icir.org> <19256.1196608121@critter.freebsd.dk> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.ORG Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:54:41 -0000 In message: <19256.1196608121@critter.freebsd.dk> "Poul-Henning Kamp" writes: : In message <20071202055031.A8107@xorpc.icir.org>, Luigi Rizzo writes: : : : >This is why i suggest having a 'scale' that can represent '1 tick' : >(and also don't depend on TIMEOUT_MSEC == 1000 and so on, but keep : >them opaque and require that the client code uses one of the supported : >scales). : : : Using a deadline timer based in the HPET, the timeout can be scheduled : to any 1/14318181th of a second and there will be no concept of "a : tick" as we know it now. : : Clients should say how often they want to be called, and they should : express it in terms of time, not based on some implementation detail : of a historical implementation of the scheduler. Yes. I'd definitely like to move to this sort of thing. I missed the conversion routines in my last email, so ignore that part of things... Does this mean that you're planning a so-called tickless implementation? Warner From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:54:50 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E18416A419 for ; Sun, 2 Dec 2007 15:54:50 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4D48E13C45D for ; Sun, 2 Dec 2007 15:54:50 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id lB2FrYvb012488; Sun, 2 Dec 2007 07:53:34 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id lB2FrYpg012487; Sun, 2 Dec 2007 07:53:34 -0800 (PST) (envelope-from rizzo) Date: Sun, 2 Dec 2007 07:53:34 -0800 From: Luigi Rizzo To: Poul-Henning Kamp Message-ID: <20071202075334.A11104@xorpc.icir.org> References: <20071202072658.A9217@xorpc.icir.org> <19518.1196609689@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <19518.1196609689@critter.freebsd.dk>; from phk@phk.freebsd.dk on Sun, Dec 02, 2007 at 03:34:49PM +0000 Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:54:50 -0000 On Sun, Dec 02, 2007 at 03:34:49PM +0000, Poul-Henning Kamp wrote: > In message <20071202072658.A9217@xorpc.icir.org>, Luigi Rizzo writes: > >On Sun, Dec 02, 2007 at 03:08:41PM +0000, Poul-Henning Kamp wrote: > > >> Using a deadline timer based in the HPET, the timeout can be scheduled > >> to any 1/14318181th of a second > > > >This must obviously come for a price or there would be no reason to > >provide 'millisecond' and 'second' precisions that you suggested. > >I agree that the API should not assume that the hardware is based on > >a periodic timer, but in the same way it should not assume that > >the hardware can generate arbitrary timeouts. > > If the client code needs this finegrained control, the trick is to > extend the API to be able to provide this information somehow. > > But before designing that, we need to have concrete examples > to talk about. > > >"periodically with whatever period is convenient to you" is > >a legitimate request that client code might have. > >(I understand that this concept of time is very southern european :) > > I disagree, that case, should be "with a period between N and M, > at your choice". yes surely this is better. But since your initial interface only had one parameter for this i tried to stick with that. > And yes, it may be that we need to extend the API for that, but > please give a concrete example so we know what we're talking about. DEVICE_POLLING is one example. You poll (and do resource allocation) to avoid livelock generated by too many interrupts, but don't know in advance what's the correct polling rate so rely on the system administrator to set HZ appropriately. (the same might apply for devices with broken or missing interrupt hardware so you have to poll. You'd trash them in an ideal world, but sometimes you have to live with them). dummynet is another one - in principle you'd like precise timing but with thousands of active pipes you probably can't afford that and so again rely on some coarser resolution which is synchronized with some system task. > >It wouldn't make sense to do poll-based event handling at the same > >rate on a 133MHz soekris or on a 3 GHz multicore cpu. > > Hz is the same for those two, so today we poll at the same rate. but you'd bump HZ to 2k or 4k up on a modern CPU, and lower to 500 on a soekris. > >In any case: in terms of API it costs absolutely nothing to support > >system-dependent resolutions instead of absolute ones, so > >i don't quite understand the opposition. > > I'm applying Jim Gettys 3. rule from the X11 project: > > 3.The only thing worse than generalizing from one example > is generalizing from no examples at all. for what matters that would apply to the ns/us/ms/s resolutions that you proposed. They may be intuitive units but it doesn't mean that they are the most appropriate ones. Anyways, I think you got my point. cheers luigi From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:57:34 2007 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 802B616A417 for ; Sun, 2 Dec 2007 15:57:34 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 3E45E13C459 for ; Sun, 2 Dec 2007 15:57:34 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.1/8.14.1) with ESMTP id lB2FqMTr070849; Sun, 2 Dec 2007 08:52:22 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sun, 02 Dec 2007 08:55:45 -0700 (MST) Message-Id: <20071202.085545.177225588.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <19518.1196609689@critter.freebsd.dk> References: <20071202072658.A9217@xorpc.icir.org> <19518.1196609689@critter.freebsd.dk> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.ORG Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:57:34 -0000 In message: <19518.1196609689@critter.freebsd.dk> "Poul-Henning Kamp" writes: : >"periodically with whatever period is convenient to you" is : >a legitimate request that client code might have. : >(I understand that this concept of time is very southern european :) : : I disagree, that case, should be "with a period between N and M, : at your choice". : : And yes, it may be that we need to extend the API for that, but : please give a concrete example so we know what we're talking about. There are a number of places in the tree that use a parameter of '1' today to mean "next time that's convenient." Some of these places are clever and know that HZ is never < 100 or > 1000 (or so they think), while others are just sloppy code. Warner From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 15:58:59 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9CCB16A418 for ; Sun, 2 Dec 2007 15:58:59 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id A98FE13C447 for ; Sun, 2 Dec 2007 15:58:59 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 73DA217105; Sun, 2 Dec 2007 15:58:58 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2FwwLP019723; Sun, 2 Dec 2007 15:58:58 GMT (envelope-from phk@critter.freebsd.dk) To: "M. Warner Losh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 08:46:48 MST." <20071202.084648.-108809549.imp@bsdimp.com> Date: Sun, 02 Dec 2007 15:58:58 +0000 Message-ID: <19722.1196611138@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 15:59:00 -0000 In message <20071202.084648.-108809549.imp@bsdimp.com>, "M. Warner Losh" writes : >In message: <15391.1196547545@critter.freebsd.dk> > Poul-Henning Kamp writes: >>>/* >>> * A duration of time, represented in the optimal way for a given provider >>> * or family of providers (ie: per cpu). >>> */ >>>typedef int timeout_time; > >What does this mean? How do I get one of those? Without a unit >associated with this number, it becomes hard to do a "tickless" >implementation. You get it back from the timeout provider. The point here is that most timeout clients use the same time interval again and again, so precalculating any "magic" values for the specific hardware makes sense. In practice your code could look like: struct foo_softc { timeout_time timeout_duration; }; ... foo_attach() { [...] sc->timeout_duration = timeout_conv_time([...], 200, TIMEOUT_MSEC); } foo_something() { timeout_arm(sc->timeout, sc->timeout_duration, [flags]); } -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 16:00:01 2007 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 150F416A47A for ; Sun, 2 Dec 2007 16:00:01 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id C20EA13C467 for ; Sun, 2 Dec 2007 16:00:00 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id CE09A17105; Sun, 2 Dec 2007 15:59:59 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2Fxxr5019735; Sun, 2 Dec 2007 15:59:59 GMT (envelope-from phk@critter.freebsd.dk) To: "M. Warner Losh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 08:53:16 MST." <20071202.085316.723205116.imp@bsdimp.com> Date: Sun, 02 Dec 2007 15:59:59 +0000 Message-ID: <19734.1196611199@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@FreeBSD.ORG Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 16:00:01 -0000 In message <20071202.085316.723205116.imp@bsdimp.com>, "M. Warner Losh" writes: >Does this mean that you're planning a so-called tickless >implementation? Time permitting, yes. First order of business is getting the API introduced, then to clean up the places where the 20-line hack for cleaning up callouts has happened, and them move on in natural order from there. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 16:04:01 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2A0916A418 for ; Sun, 2 Dec 2007 16:04:01 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id A0FCC13C442 for ; Sun, 2 Dec 2007 16:04:00 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 9A75617105; Sun, 2 Dec 2007 16:03:59 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2G3x3m019766; Sun, 2 Dec 2007 16:03:59 GMT (envelope-from phk@critter.freebsd.dk) To: Luigi Rizzo From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 07:53:34 PST." <20071202075334.A11104@xorpc.icir.org> Date: Sun, 02 Dec 2007 16:03:59 +0000 Message-ID: <19765.1196611439@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 16:04:02 -0000 In message <20071202075334.A11104@xorpc.icir.org>, Luigi Rizzo writes: >> I disagree, that case, should be "with a period between N and M, >> at your choice". > >yes surely this is better. But since your initial interface only >had one parameter for this i tried to stick with that. Right now, I'm looking at the API to replace the current API. New functionality can be added, as we find out what it should be and how it should work. >DEVICE_POLLING is one example. You poll (and do resource allocation) >to avoid livelock generated by too many interrupts, but don't know >in advance what's the correct polling rate so rely on the system administrator >to set HZ appropriately. Ahh, but isn't that the wrong way ? Shouldn't the driver (or central polling) code dynamically adjust the pollrate instead ? >> >It wouldn't make sense to do poll-based event handling at the same >> >rate on a 133MHz soekris or on a 3 GHz multicore cpu. >> >> Hz is the same for those two, so today we poll at the same rate. > >but you'd bump HZ to 2k or 4k up on a modern CPU, and lower to 500 >on a soekris. Nothing prevents dummynet or devicepolling from having a sysctl to adjust the rate. >> I'm applying Jim Gettys 3. rule from the X11 project: >> >> 3.The only thing worse than generalizing from one example >> is generalizing from no examples at all. > >for what matters that would apply to the ns/us/ms/s resolutions that >you proposed. They may be intuitive units but it doesn't mean that they are >the most appropriate ones. There's trivially room for another 28 'unit' flags, but I don't want to define them, until rule three above is satisfied. with respect to ns/us/ms/s, there are plenty of places in device drivers where datasheets specify timeout durations on those terms, and network protocols add another dozen such well defined timeouts. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 16:15:25 2007 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3FB016A41B for ; Sun, 2 Dec 2007 16:15:25 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 941D713C458 for ; Sun, 2 Dec 2007 16:15:25 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 778FB17105; Sun, 2 Dec 2007 16:15:23 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2GFNkH019828; Sun, 2 Dec 2007 16:15:23 GMT (envelope-from phk@critter.freebsd.dk) To: "M. Warner Losh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 08:55:45 MST." <20071202.085545.177225588.imp@bsdimp.com> Date: Sun, 02 Dec 2007 16:15:23 +0000 Message-ID: <19827.1196612123@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@FreeBSD.ORG Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 16:15:25 -0000 In message <20071202.085545.177225588.imp@bsdimp.com>, "M. Warner Losh" writes: >There are a number of places in the tree that use a parameter of '1' >today to mean "next time that's convenient." Some of these places are >clever and know that HZ is never < 100 or > 1000 (or so they think), >while others are just sloppy code. Yes, but those can hardly be called "concrete" in terms of wanting to know what they mean, can they ? :-) The only way I can see we can deal with them in the short term, is to ask for timeouts of "1000000 / hz, TIMEOUT_USEC" -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 16:39:25 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 505C516A480 for ; Sun, 2 Dec 2007 16:39:25 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id D77E713C46B for ; Sun, 2 Dec 2007 16:39:24 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.1/8.14.1) with ESMTP id lB2GWdRK071258; Sun, 2 Dec 2007 09:32:40 -0700 (MST) (envelope-from imp@bsdimp.com) Date: Sun, 02 Dec 2007 09:36:03 -0700 (MST) Message-Id: <20071202.093603.228972203.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <19827.1196612123@critter.freebsd.dk> References: <20071202.085545.177225588.imp@bsdimp.com> <19827.1196612123@critter.freebsd.dk> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 16:39:25 -0000 In message: <19827.1196612123@critter.freebsd.dk> "Poul-Henning Kamp" writes: : In message <20071202.085545.177225588.imp@bsdimp.com>, "M. Warner Losh" writes: : : >There are a number of places in the tree that use a parameter of '1' : >today to mean "next time that's convenient." Some of these places are : >clever and know that HZ is never < 100 or > 1000 (or so they think), : >while others are just sloppy code. : : Yes, but those can hardly be called "concrete" in terms of wanting : to know what they mean, can they ? :-) : : The only way I can see we can deal with them in the short term, : is to ask for timeouts of "1000000 / hz, TIMEOUT_USEC" Or have a "timeout_soon" function like you have the other timeout conversion routines. Warner From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 16:58:19 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11B2016A419 for ; Sun, 2 Dec 2007 16:58:19 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id C4A2413C448 for ; Sun, 2 Dec 2007 16:58:18 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.2/8.14.2/NETPLEX) with ESMTP id lB2GwHJi001773; Sun, 2 Dec 2007 11:58:17 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Sun, 02 Dec 2007 11:58:17 -0500 (EST) Date: Sun, 2 Dec 2007 11:58:17 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Michael Bushkov In-Reply-To: <5555F136-D396-4333-837D-C4924416C3CB@freebsd.org> Message-ID: References: <5555F136-D396-4333-837D-C4924416C3CB@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: libc-scoped function and Symbol.map X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 16:58:19 -0000 On Sun, 2 Dec 2007, Michael Bushkov wrote: > Hi! > If I add the internal libc-scoped function (named __getgroupmembership) to > src/lib/libc/gen/getgrent.c, I need to add its name to Symbol.map, don't I? If the function is meant to be used outside of libc, then yes. If the function is only meant to be used by the FreeBSD base system (not ports or other SW maintained elsewhere), then it should be in the FBSDprivate namespace. If this function is meant to be used outside the FreeBSD base, then it goes in the FBSDpublic namespace. Once in the public namespace and it hits a release, we commit to maintaining an ABI compatible version of it forever (*). (*) Most likely for many years anyway. -- DE From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 17:43:58 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7031316A421 for ; Sun, 2 Dec 2007 17:43:58 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 2CDB413C46E for ; Sun, 2 Dec 2007 17:43:57 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 3BDEB17105; Sun, 2 Dec 2007 17:43:56 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id lB2HhtGC045309; Sun, 2 Dec 2007 17:43:55 GMT (envelope-from phk@critter.freebsd.dk) To: "M. Warner Losh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 02 Dec 2007 09:36:03 MST." <20071202.093603.228972203.imp@bsdimp.com> Date: Sun, 02 Dec 2007 17:43:54 +0000 Message-ID: <45308.1196617434@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@FreeBSD.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 17:43:58 -0000 In message <20071202.093603.228972203.imp@bsdimp.com>, "M. Warner Losh" writes: >In message: <19827.1196612123@critter.freebsd.dk> > "Poul-Henning Kamp" writes: >: In message <20071202.085545.177225588.imp@bsdimp.com>, "M. Warner Losh" writes: >: >: >There are a number of places in the tree that use a parameter of '1' >: >today to mean "next time that's convenient." Some of these places are >: >clever and know that HZ is never < 100 or > 1000 (or so they think), >: >while others are just sloppy code. >: >: Yes, but those can hardly be called "concrete" in terms of wanting >: to know what they mean, can they ? :-) >: >: The only way I can see we can deal with them in the short term, >: is to ask for timeouts of "1000000 / hz, TIMEOUT_USEC" > >Or have a "timeout_soon" function like you have the other timeout >conversion routines. I'm not very keen on offering too much rope. Intelligent decisions need to be made about these polling rates and making it too easy to not think about it would be to encourage bad practices. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun Dec 2 19:06:40 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32A1E16A418 for ; Sun, 2 Dec 2007 19:06:40 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id 182F013C455 for ; Sun, 2 Dec 2007 19:06:40 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id lB2J5MBW022665; Sun, 2 Dec 2007 11:05:22 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id lB2J5MBO022664; Sun, 2 Dec 2007 11:05:22 -0800 (PST) (envelope-from rizzo) Date: Sun, 2 Dec 2007 11:05:22 -0800 From: Luigi Rizzo To: Poul-Henning Kamp Message-ID: <20071202110522.A22571@xorpc.icir.org> References: <20071202.093603.228972203.imp@bsdimp.com> <45308.1196617434@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <45308.1196617434@critter.freebsd.dk>; from phk@phk.freebsd.dk on Sun, Dec 02, 2007 at 05:43:54PM +0000 Cc: arch@FreeBSD.org Subject: Re: New "timeout" api, to replace callout X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2007 19:06:40 -0000 On Sun, Dec 02, 2007 at 05:43:54PM +0000, Poul-Henning Kamp wrote: > In message <20071202.093603.228972203.imp@bsdimp.com>, "M. Warner Losh" writes: > >In message: <19827.1196612123@critter.freebsd.dk> > > "Poul-Henning Kamp" writes: > >: In message <20071202.085545.177225588.imp@bsdimp.com>, "M. Warner Losh" writes: > >: > >: >There are a number of places in the tree that use a parameter of '1' > >: >today to mean "next time that's convenient." Some of these places are > >: >clever and know that HZ is never < 100 or > 1000 (or so they think), > >: >while others are just sloppy code. > >: > >: Yes, but those can hardly be called "concrete" in terms of wanting > >: to know what they mean, can they ? :-) > >: > >: The only way I can see we can deal with them in the short term, > >: is to ask for timeouts of "1000000 / hz, TIMEOUT_USEC" > > > >Or have a "timeout_soon" function like you have the other timeout > >conversion routines. > > I'm not very keen on offering too much rope. > > Intelligent decisions need to be made about these polling rates and > making it too easy to not think about it would be to encourage > bad practices. the approach 'no less than N, preferably* no more than M' {n|u|m}seconds that phk suggested looks to me like the correct approach. The programmer can make his assumptions explicit, and the system can make convenient decisions without making arbitrary assumptions. cheers luigi From owner-freebsd-arch@FreeBSD.ORG Mon Dec 3 02:44:39 2007 Return-Path: Delivered-To: freebsd-arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7C3A16A418; Mon, 3 Dec 2007 02:44:39 +0000 (UTC) (envelope-from davidxu@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 89A4713C457; Mon, 3 Dec 2007 02:44:39 +0000 (UTC) (envelope-from davidxu@FreeBSD.org) Received: from [127.0.0.1] (root@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id lB32iYIT064731; Mon, 3 Dec 2007 02:44:36 GMT (envelope-from davidxu@freebsd.org) Message-ID: <47536DCF.3040506@freebsd.org> Date: Mon, 03 Dec 2007 10:45:35 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.13) Gecko/20070516 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: <20071128211022.GA74762@lor.one-eyed-alien.net> <20071128213947.Q7555@fledge.watson.org> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: Brooks Davis , Robert Watson , freebsd-arch@FreeBSD.org Subject: Re: RFC: libkse*.a in 7.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Dec 2007 02:44:39 -0000 Daniel Eischen wrote: > > I argued for removing libc.a as well as lib.a a couple of > years ago and was met with opposition, mostly because statically > linked applications are faster. > > I think we should remove libthr.a, libkse.a and libc.a, so flame on! > Keep libc.a, and add my 1 cent for removing libthr.a, libkse.a. :-) From owner-freebsd-arch@FreeBSD.ORG Mon Dec 3 23:26:16 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35B0316A421 for ; Mon, 3 Dec 2007 23:26:16 +0000 (UTC) (envelope-from BearPerson@gmx.net) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 6C62F13C442 for ; Mon, 3 Dec 2007 23:26:15 +0000 (UTC) (envelope-from BearPerson@gmx.net) Received: (qmail invoked by alias); 03 Dec 2007 22:59:34 -0000 Received: from port-83-236-56-222.dynamic.qsc.de (EHLO gmx.net) [83.236.56.222] by mail.gmx.net (mp003) with SMTP; 03 Dec 2007 23:59:34 +0100 X-Authenticated: #20254835 X-Provags-ID: V01U2FsdGVkX19fs5xarR5ft6qKMWcNaV5HfSQ8za8YJRk/V3ALrj q4EvUqcHYatYOq Date: Mon, 3 Dec 2007 23:59:29 +0100 From: Karsten Behrmann To: freebsd-arch@freebsd.org Message-ID: <20071203235929.685d3674@Karsten.Behrmanns.Kasten> In-Reply-To: <20071128.151021.709401576.imp@bsdimp.com> References: <20071128.151021.709401576.imp@bsdimp.com> X-Mailer: Claws Mail 2.9.2 (GTK+ 2.8.18; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; boundary=Sig_w9H.JONpwy2BbOTeikusyyT; protocol="application/pgp-signature"; micalg=PGP-SHA1 X-Y-GMX-Trusted: 0 Subject: Re: Code review request: small optimization to localtime.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Dec 2007 23:26:16 -0000 --Sig_w9H.JONpwy2BbOTeikusyyT Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi there, [ we do ] > pthread_mutex_lock(); > if (!is_set) { > is_set =3D true; > // do something > } > pthread_mutex_unlock(); [couldn't we do] > if (!is_set) { > pthread_mutex_lock(); > if (!is_set) { > is_set =3D true; > // do something > } > pthread_mutex_unlock(); > } Ah, the old "double-checked locking" thing. Take what I say here with a grain of salt: I'm a plain programmer, not very versed in different architectures, so I just pick up what I hear. essentially, the problem with the scheme is that plain variable writes may not be ordered as expected, making them unsuitable to communicate between threads and processors, depending on compiler and architecture. Let's take the example pthread_mutex_lock(); if (!is_set) { a =3D 3; b =3D 4; c =3D a * b; is_set =3D true; } pthread_mutex_unlock(); Now, an optimizing compiler might put a and b into registers, compute a * b while storing a 1 to is_set, and then store a, b, and c from registers into memory. As I said, I'm not actually sure where compilers are allowed to do this. Also, on some architectures, the caching structure may cause writes to is_set to show up earlier on another CPU than writes to other parts of memory, if you're unlucky. So the bottom line is, in some situations, writes to memory don't neccessarily take effect everywhere in the same order as the code would suggest, breaking double-checked locking. is_set could end up getting set before the "something" part has been executed. You need the synchronization to ensure consistency of data before being sure your checks do what you think they do. In fact, in your patch, you sometimes actually set the variable getting checked before setting the data it is supposed to protect ;-) > - _MUTEX_LOCK(&gmt_mutex); > if (!gmt_is_set) { > - gmt_is_set =3D TRUE; > + _MUTEX_LOCK(&gmt_mutex); > + if (!gmt_is_set) { > + gmt_is_set =3D TRUE; XXX - at this point, gmt_is_set is TRUE but gmtptr is not initialized yet. > #ifdef ALL_STATE > - gmtptr =3D (struct state *) malloc(sizeof *gmtptr); > - if (gmtptr !=3D NULL) > + gmtptr =3D (struct state *) malloc(sizeof *gmtptr); > + if (gmtptr !=3D NULL) > #endif /* defined ALL_STATE */ > - gmtload(gmtptr); > + gmtload(gmtptr); > + } > + _MUTEX_UNLOCK(&gmt_mutex); } - _MUTEX_UNLOCK(&gmt_mutex); So Far, Karsten Behrmann --Sig_w9H.JONpwy2BbOTeikusyyT Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (GNU/Linux) iD8DBQFHVIpVAksKLoO3vywRAj3gAKCTAcmVaXGGUiSa6feF1UveYDbhNACaA8B2 PMVBUfz8oM9Kjv+KHZ/bfxM= =I8Wk -----END PGP SIGNATURE----- --Sig_w9H.JONpwy2BbOTeikusyyT-- From owner-freebsd-arch@FreeBSD.ORG Tue Dec 4 01:46:24 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5DDD716A418 for ; Tue, 4 Dec 2007 01:46:24 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 52D9C13C457 for ; Tue, 4 Dec 2007 01:46:24 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 621751A4D7E; Mon, 3 Dec 2007 17:46:14 -0800 (PST) Date: Mon, 3 Dec 2007 17:46:14 -0800 From: Alfred Perlstein To: Karsten Behrmann Message-ID: <20071204014614.GE76623@elvis.mu.org> References: <20071128.151021.709401576.imp@bsdimp.com> <20071203235929.685d3674@Karsten.Behrmanns.Kasten> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071203235929.685d3674@Karsten.Behrmanns.Kasten> User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: Code review request: small optimization to localtime.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Dec 2007 01:46:24 -0000 Karsten, _typically_ (but not always) an "unlock" operation requires that writes prior to the unlock be globally visible. This is why it works almost everywhere. * Karsten Behrmann [071203 15:26] wrote: > Hi there, > > [ we do ] > > > pthread_mutex_lock(); > > if (!is_set) { > > is_set = true; > > // do something > > } > > pthread_mutex_unlock(); > > [couldn't we do] > > > if (!is_set) { > > pthread_mutex_lock(); > > if (!is_set) { > > is_set = true; > > // do something > > } > > pthread_mutex_unlock(); > > } > > Ah, the old "double-checked locking" thing. Take what I say here > with a grain of salt: I'm a plain programmer, not very versed in > different architectures, so I just pick up what I hear. > > essentially, the problem with the scheme is that plain variable > writes may not be ordered as expected, making them unsuitable to > communicate between threads and processors, depending on compiler > and architecture. > > Let's take the example > > pthread_mutex_lock(); > if (!is_set) { > a = 3; > b = 4; > c = a * b; > is_set = true; > } > pthread_mutex_unlock(); > > Now, an optimizing compiler might put a and b into registers, > compute a * b while storing a 1 to is_set, and then store a, b, and > c from registers into memory. As I said, I'm not actually sure > where compilers are allowed to do this. > > Also, on some architectures, the caching structure may cause writes > to is_set to show up earlier on another CPU than writes to other > parts of memory, if you're unlucky. > > So the bottom line is, in some situations, writes to memory don't > neccessarily take effect everywhere in the same order as the code > would suggest, breaking double-checked locking. is_set could end > up getting set before the "something" part has been executed. > You need the synchronization to ensure consistency of data before > being sure your checks do what you think they do. > > In fact, in your patch, you sometimes actually set the variable > getting checked before setting the data it is supposed to protect ;-) > > > - _MUTEX_LOCK(&gmt_mutex); > > if (!gmt_is_set) { > > - gmt_is_set = TRUE; > > + _MUTEX_LOCK(&gmt_mutex); > > + if (!gmt_is_set) { > > + gmt_is_set = TRUE; > > XXX - at this point, gmt_is_set is TRUE but gmtptr is not initialized yet. > > > #ifdef ALL_STATE > > - gmtptr = (struct state *) malloc(sizeof *gmtptr); > > - if (gmtptr != NULL) > > + gmtptr = (struct state *) malloc(sizeof *gmtptr); > > + if (gmtptr != NULL) > > #endif /* defined ALL_STATE */ > > - gmtload(gmtptr); > > + gmtload(gmtptr); > > + } > > + _MUTEX_UNLOCK(&gmt_mutex); > } > - _MUTEX_UNLOCK(&gmt_mutex); > > > So Far, > Karsten Behrmann -- - Alfred Perlstein From owner-freebsd-arch@FreeBSD.ORG Tue Dec 4 02:00:18 2007 Return-Path: Delivered-To: freebsd-arch@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8919816A419 for ; Tue, 4 Dec 2007 02:00:18 +0000 (UTC) (envelope-from davidxu@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7BCFB13C447; Tue, 4 Dec 2007 02:00:18 +0000 (UTC) (envelope-from davidxu@FreeBSD.org) Received: from [127.0.0.1] (root@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id lB420F4q016775; Tue, 4 Dec 2007 02:00:16 GMT (envelope-from davidxu@freebsd.org) Message-ID: <4754B4EC.5040005@freebsd.org> Date: Tue, 04 Dec 2007 10:01:16 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.13) Gecko/20070516 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Karsten Behrmann References: <20071128.151021.709401576.imp@bsdimp.com> <20071203235929.685d3674@Karsten.Behrmanns.Kasten> In-Reply-To: <20071203235929.685d3674@Karsten.Behrmanns.Kasten> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: Code review request: small optimization to localtime.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Dec 2007 02:00:18 -0000 Karsten Behrmann wrote: > Hi there, > > [ we do ] > > >> pthread_mutex_lock(); >> if (!is_set) { >> is_set = true; >> // do something >> } >> pthread_mutex_unlock(); > > > [couldn't we do] > > >> if (!is_set) { >> pthread_mutex_lock(); >> if (!is_set) { >> is_set = true; >> // do something >> } >> pthread_mutex_unlock(); >> } > > > Ah, the old "double-checked locking" thing. Take what I say here > with a grain of salt: I'm a plain programmer, not very versed in > different architectures, so I just pick up what I hear. > > essentially, the problem with the scheme is that plain variable > writes may not be ordered as expected, making them unsuitable to > communicate between threads and processors, depending on compiler > and architecture. > > Let's take the example > > pthread_mutex_lock(); > if (!is_set) { > a = 3; > b = 4; > c = a * b; > is_set = true; > } > pthread_mutex_unlock(); > > Now, an optimizing compiler might put a and b into registers, > compute a * b while storing a 1 to is_set, and then store a, b, and > c from registers into memory. As I said, I'm not actually sure > where compilers are allowed to do this. > > Also, on some architectures, the caching structure may cause writes > to is_set to show up earlier on another CPU than writes to other > parts of memory, if you're unlucky. > > So the bottom line is, in some situations, writes to memory don't > neccessarily take effect everywhere in the same order as the code > would suggest, breaking double-checked locking. is_set could end > up getting set before the "something" part has been executed. > You need the synchronization to ensure consistency of data before > being sure your checks do what you think they do. > > In fact, in your patch, you sometimes actually set the variable > getting checked before setting the data it is supposed to protect ;-) > > >>- _MUTEX_LOCK(&gmt_mutex); >> if (!gmt_is_set) { >>- gmt_is_set = TRUE; >>+ _MUTEX_LOCK(&gmt_mutex); >>+ if (!gmt_is_set) { >>+ gmt_is_set = TRUE; > > > XXX - at this point, gmt_is_set is TRUE but gmtptr is not initialized yet. > > >> #ifdef ALL_STATE >>- gmtptr = (struct state *) malloc(sizeof *gmtptr); >>- if (gmtptr != NULL) >>+ gmtptr = (struct state *) malloc(sizeof *gmtptr); >>+ if (gmtptr != NULL) >> #endif /* defined ALL_STATE */ >>- gmtload(gmtptr); >>+ gmtload(gmtptr); >>+ } >>+ _MUTEX_UNLOCK(&gmt_mutex); > > } > - _MUTEX_UNLOCK(&gmt_mutex); > > > So Far, > Karsten Behrmann You are correct here, I think it should either use pthread_once or use some atomic instructions to implicitly set memory barrieries. The pthread_once function in libthr is already using mutex and aware of memory barrieries. Regards, David Xu From owner-freebsd-arch@FreeBSD.ORG Tue Dec 4 05:16:29 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18CD216A41B for ; Tue, 4 Dec 2007 05:16:29 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail12.syd.optusnet.com.au (mail12.syd.optusnet.com.au [211.29.132.193]) by mx1.freebsd.org (Postfix) with ESMTP id A09A213C4CC for ; Tue, 4 Dec 2007 05:16:28 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-219-213.carlnfd3.nsw.optusnet.com.au (c211-30-219-213.carlnfd3.nsw.optusnet.com.au [211.30.219.213]) by mail12.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id lB45GNpG000435 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Dec 2007 16:16:25 +1100 Date: Tue, 4 Dec 2007 16:16:23 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Karsten Behrmann In-Reply-To: <20071203235929.685d3674@Karsten.Behrmanns.Kasten> Message-ID: <20071204153851.C3662@delplex.bde.org> References: <20071128.151021.709401576.imp@bsdimp.com> <20071203235929.685d3674@Karsten.Behrmanns.Kasten> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: Code review request: small optimization to localtime.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Dec 2007 05:16:29 -0000 On Mon, 3 Dec 2007, Karsten Behrmann wrote: > Ah, the old "double-checked locking" thing. Take what I say here > with a grain of salt: I'm a plain programmer, not very versed in > different architectures, so I just pick up what I hear. > > essentially, the problem with the scheme is that plain variable > writes may not be ordered as expected, making them unsuitable to > communicate between threads and processors, depending on compiler > and architecture. > > Let's take the example > > pthread_mutex_lock(); > if (!is_set) { > a = 3; > b = 4; > c = a * b; > is_set = true; > } > pthread_mutex_unlock(); > > Now, an optimizing compiler might put a and b into registers, > compute a * b while storing a 1 to is_set, and then store a, b, and > c from registers into memory. As I said, I'm not actually sure > where compilers are allowed to do this. > > Also, on some architectures, the caching structure may cause writes > to is_set to show up earlier on another CPU than writes to other > parts of memory, if you're unlucky. As alfred said, the locking implementation is supposed to hide these problems so that most code doesn't need to worry about them. I don't know what pthread does, but in for kernel mutexes on i386 with CC=gcc, the first problem is handled by putting a "memory" clobber in the asm for atomic_cmpset to force the compiler to load and store all memory variables (not just the ones in args for the asm), while the second problem is handled implicitly by using a locked instruction for cmpset and/or other magic i386 cache behaviour. The atomic cmpset is, more precisely, atomic_cmpset_acq for locking and atomic_cmpset_rel for unlocking. It is acquire/release semantics that handles the second problem. On i386's acquire/release semantics happen almost automatically so at the lowest level there is only atomic_cmpset_int and everything else is #defined as that. Some sloppy (buggy?) callers of the atomic_ cmpset family use only atomic_cmpset_int, and one of the examples in atomic.9 doesn't mention acq or rel. It isn't clear what a non- acquiring non-releasing cmpset is good for. > In fact, in your patch, you sometimes actually set the variable > getting checked before setting the data it is supposed to protect ;-) > >> - _MUTEX_LOCK(&gmt_mutex); >> if (!gmt_is_set) { >> - gmt_is_set = TRUE; >> + _MUTEX_LOCK(&gmt_mutex); >> + if (!gmt_is_set) { >> + gmt_is_set = TRUE; > > XXX - at this point, gmt_is_set is TRUE but gmtptr is not initialized yet. This is a bug. It shows how tricky locking becomes again if you don't use the normal mechanism of always acquiring the lock first. With the normal mechanism, the ordering inside locked regions doesn't matter since variables are only accessed inside of locked regions, but the optimized version has accesses outside of locked regions. Setting gmt_is_set after initializing gmtptr isn't much better, since the unlock hasn't had a chance to force everything to memory so gmt_is_set might reach memory before gmtptr. I think the order needs to be something like: gmtptr = ... unlock lock gmt_is_set = TRUE; unlock, or slightly shorter using explicit barrier operations instead of implicit ones in unlock. Even if we could guarantee that gmtptr and gmt_is_set are not written prior to the unlock, there would still be a problem with the optimized version unless we could guarantee the order of the writes triggered by the unlock. The unoptimizated version is not affected by this complication either. Bruce From owner-freebsd-arch@FreeBSD.ORG Tue Dec 4 09:15:37 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9115216A4DE for ; Tue, 4 Dec 2007 09:15:37 +0000 (UTC) (envelope-from jan.grant@bristol.ac.uk) Received: from diri.bris.ac.uk (diri.bris.ac.uk [137.222.10.112]) by mx1.freebsd.org (Postfix) with ESMTP id 5AA4613C447 for ; Tue, 4 Dec 2007 09:15:37 +0000 (UTC) (envelope-from jan.grant@bristol.ac.uk) Received: from mail.ilrt.bris.ac.uk ([137.222.16.62]) by diri.bris.ac.uk with esmtp (Exim 4.67) (envelope-from ) id 1IzTcQ-0000rE-EJ; Tue, 04 Dec 2007 08:59:09 +0000 Received: from cse-jg.cse.bris.ac.uk ([137.222.12.37]:54951) by mail.ilrt.bris.ac.uk with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1IzTcD-000840-PS; Tue, 04 Dec 2007 08:58:53 +0000 Date: Tue, 4 Dec 2007 08:58:53 +0000 (GMT) From: Jan Grant X-X-Sender: cmjg@tribble.ilrt.bris.ac.uk To: Alfred Perlstein In-Reply-To: <20071204014614.GE76623@elvis.mu.org> Message-ID: <20071204085502.N83722@tribble.ilrt.bris.ac.uk> References: <20071128.151021.709401576.imp@bsdimp.com> <20071203235929.685d3674@Karsten.Behrmanns.Kasten> <20071204014614.GE76623@elvis.mu.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-ILRT-MailScanner: Found to be clean X-ILRT-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-2.538, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL 1.86, BAYES_00 -2.60) X-ILRT-MailScanner-From: jan.grant@bristol.ac.uk X-Spam-Status: No X-Spam-Score: -0.8 X-Spam-Level: / Cc: Karsten Behrmann , freebsd-arch@freebsd.org Subject: Re: Code review request: small optimization to localtime.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Dec 2007 09:15:37 -0000 On Mon, 3 Dec 2007, Alfred Perlstein wrote: [on the double-checked locking idiom] > Karsten, _typically_ (but not always) an "unlock" operation > requires that writes prior to the unlock be globally visible. > > This is why it works almost everywhere. Perhaps, but if you use it you should probably mark the code with /* XXX not guaranteed to be correct by POSIX */ Double-checked locking is broken without an appropriate barrier. "Correctness over speed" should surely be our watchword :-) Cheers, jan -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ Hang on, wasn't he holding a wooden parrot? No! It was a porcelain owl. From owner-freebsd-arch@FreeBSD.ORG Sat Dec 8 00:16:12 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EE2F16A417 for ; Sat, 8 Dec 2007 00:16:12 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with SMTP id E7F1713C468 for ; Sat, 8 Dec 2007 00:16:11 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: (qmail 21828 invoked by uid 399); 7 Dec 2007 23:49:30 -0000 Received: from localhost (HELO lap.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTP; 7 Dec 2007 23:49:30 -0000 X-Originating-IP: 127.0.0.1 Message-ID: <4759DC08.9070600@FreeBSD.org> Date: Fri, 07 Dec 2007 15:49:28 -0800 From: Doug Barton Organization: http://www.FreeBSD.org/ User-Agent: Thunderbird 2.0.0.9 (X11/20071119) MIME-Version: 1.0 To: freebsd-arch@freebsd.org X-Enigmail-Version: 0.95.5 OpenPGP: id=D5B2F0FB Content-Type: multipart/signed; micalg=pgp-ripemd160; protocol="application/pgp-signature"; boundary="------------enig7D950AC219629E304C995BEA" Subject: Should libgssapi be hidden behind the MK_KERBEROS knob? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Dec 2007 00:16:12 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig7D950AC219629E304C995BEA Content-Type: multipart/mixed; boundary="------------040507040604010604050709" This is a multi-part message in MIME format. --------------040507040604010604050709 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable If there is a better list for this, don't hesitate to let me know. I use WITHOUT_KERBEROS=3Dtrue in /etc/{make|src}.conf, since I don't need or use it. However, this leads to a problem with building the kdelibs3 port. The configure script looks for the presence of libgssapi and the associated headers, and takes that to mean that kerberos is available, and sets things up accordingly. This causes the build to fail when it tries to actually link something to a kerberos library. I realize that GSS can be used for other things besides kerberos, but are we really losing anything by hiding them both under the same knob? If the answer to that is yes, is there any objection to a WITHOUT_GSS knob? Thanks, Doug --=20 This .signature sanitized for your protection --------------040507040604010604050709 Content-Type: text/plain; name="gss-no-krb.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline; filename="gss-no-krb.diff" Index: Makefile =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /usr/local/ncvs/src/lib/Makefile,v retrieving revision 1.226 diff -u -r1.226 Makefile --- Makefile 17 Nov 2007 21:29:02 -0000 1.226 +++ Makefile 3 Dec 2007 18:58:08 -0000 @@ -31,7 +31,7 @@ libbegemot ${_libbluetooth} libbsnmp libbz2 \ libcalendar libcam libcompat libdevinfo libdevstat libdisk \ libedit libexpat libfetch libftpio libgeom ${_libgpib} \ - libgssapi libipsec \ + ${_libgssapi} libipsec \ ${_libipx} libkiconv libmagic libmemstat ${_libmilter} ${_libmp} \ ${_libncp} ${_libngatm} libopie libpam libpcap \ libpmc ${_libkse} librt ${_libsdp} ${_libsm} ${_libsmb} \ @@ -120,4 +120,8 @@ _libgpib=3D libgpib .endif =20 +.if ${MK_KERBEROS} !=3D "no" +_libgssapi=3D libgssapi +.endif + --------------040507040604010604050709-- --------------enig7D950AC219629E304C995BEA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (FreeBSD) iD8DBQFHWdwJyIakK9Wy8PsRA9MMAKDI5w0b/8MJbNwrO3htYfaYqfGGTQCdEB8w CvwaGXUvsjU2wcVekVxLF7M= =S0t4 -----END PGP SIGNATURE----- --------------enig7D950AC219629E304C995BEA-- From owner-freebsd-arch@FreeBSD.ORG Sat Dec 8 16:38:59 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9969216A468; Sat, 8 Dec 2007 16:38:59 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 1C9BD13C474; Sat, 8 Dec 2007 16:38:58 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.1/8.13.8) with ESMTP id lB8GcwRp092245; Sat, 8 Dec 2007 10:38:58 -0600 (CST) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.1/8.13.8/Submit) id lB8GcvjJ092244; Sat, 8 Dec 2007 10:38:57 -0600 (CST) (envelope-from brooks) Date: Sat, 8 Dec 2007 10:38:57 -0600 From: Brooks Davis To: Doug Barton Message-ID: <20071208163857.GC91919@lor.one-eyed-alien.net> References: <4759DC08.9070600@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Bu8it7iiRSEf40bY" Content-Disposition: inline In-Reply-To: <4759DC08.9070600@FreeBSD.org> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Sat, 08 Dec 2007 10:38:58 -0600 (CST) Cc: freebsd-arch@freebsd.org Subject: Re: Should libgssapi be hidden behind the MK_KERBEROS knob? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Dec 2007 16:38:59 -0000 --Bu8it7iiRSEf40bY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 07, 2007 at 03:49:28PM -0800, Doug Barton wrote: > If there is a better list for this, don't hesitate to let me know. >=20 > I use WITHOUT_KERBEROS=3Dtrue in /etc/{make|src}.conf, since I don't > need or use it. However, this leads to a problem with building the > kdelibs3 port. The configure script looks for the presence of > libgssapi and the associated headers, and takes that to mean that > kerberos is available, and sets things up accordingly. This causes > the build to fail when it tries to actually link something to a > kerberos library. >=20 > I realize that GSS can be used for other things besides kerberos, but > are we really losing anything by hiding them both under the same knob? > If the answer to that is yes, is there any objection to a WITHOUT_GSS > knob? We wouldn't loose anything today, but a without GSS knob makes more sense to me. There's at least one other GSS system in fairly wide use in the high performance computing world today. -- Brooks --Bu8it7iiRSEf40bY Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFHWsihXY6L6fI4GtQRAtscAJ9D9UYlfrT5nT/DznXWPS1Ggktt/QCdHF6q dRsoJ2gmd1D6zOCtITpAq+E= =YBVQ -----END PGP SIGNATURE----- --Bu8it7iiRSEf40bY-- From owner-freebsd-arch@FreeBSD.ORG Sat Dec 8 21:16:57 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F72516A418; Sat, 8 Dec 2007 21:16:57 +0000 (UTC) (envelope-from tetlowgm@mac.com) Received: from smtpoutm.mac.com (smtpoutm.mac.com [17.148.16.82]) by mx1.freebsd.org (Postfix) with ESMTP id 077B013C4D1; Sat, 8 Dec 2007 21:16:56 +0000 (UTC) (envelope-from tetlowgm@mac.com) Received: from mac.com (asmtp002-s [10.150.69.65]) by smtpoutm.mac.com (Xserve/smtpout019/MantshX 4.0) with ESMTP id lB8L16KA021313; Sat, 8 Dec 2007 13:01:06 -0800 (PST) Received: from [192.168.1.2] (cpe-66-75-32-81.san.res.rr.com [66.75.32.81]) (authenticated bits=0) by mac.com (Xserve/asmtp002/MantshX 4.0) with ESMTP id lB8L14DG010138 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 8 Dec 2007 13:01:04 -0800 (PST) Message-Id: From: Gordon M Tetlow To: Brooks Davis In-Reply-To: <20071208163857.GC91919@lor.one-eyed-alien.net> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Date: Sat, 8 Dec 2007 11:42:48 -0800 References: <4759DC08.9070600@FreeBSD.org> <20071208163857.GC91919@lor.one-eyed-alien.net> X-Mailer: Apple Mail (2.915) Cc: Doug Barton , freebsd-arch@freebsd.org Subject: Re: Should libgssapi be hidden behind the MK_KERBEROS knob? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Dec 2007 21:16:57 -0000 On Dec 8, 2007, at 8:38 AM, Brooks Davis wrote: > On Fri, Dec 07, 2007 at 03:49:28PM -0800, Doug Barton wrote: >> If there is a better list for this, don't hesitate to let me know. >> >> I use WITHOUT_KERBEROS=true in /etc/{make|src}.conf, since I don't >> need or use it. However, this leads to a problem with building the >> kdelibs3 port. The configure script looks for the presence of >> libgssapi and the associated headers, and takes that to mean that >> kerberos is available, and sets things up accordingly. This causes >> the build to fail when it tries to actually link something to a >> kerberos library. >> >> I realize that GSS can be used for other things besides kerberos, but >> are we really losing anything by hiding them both under the same >> knob? >> If the answer to that is yes, is there any objection to a WITHOUT_GSS >> knob? > > We wouldn't loose anything today, but a without GSS knob makes more > sense to me. There's at least one other GSS system in fairly wide use > in the high performance computing world today. How about WITHOUT_KERBEROS implies WITHOUT_GSSAPI unless people specifically ask for GSSAPI? Is that too obscure? -gordon From owner-freebsd-arch@FreeBSD.ORG Sat Dec 8 23:42:12 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F58F16A419 for ; Sat, 8 Dec 2007 23:42:12 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with SMTP id 0530B13C455 for ; Sat, 8 Dec 2007 23:42:11 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: (qmail 19277 invoked by uid 399); 8 Dec 2007 23:42:11 -0000 Received: from localhost (HELO lap.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTP; 8 Dec 2007 23:42:11 -0000 X-Originating-IP: 127.0.0.1 Message-ID: <475B2BD1.7000303@FreeBSD.org> Date: Sat, 08 Dec 2007 15:42:09 -0800 From: Doug Barton Organization: http://www.FreeBSD.org/ User-Agent: Thunderbird 2.0.0.9 (X11/20071119) MIME-Version: 1.0 To: Gordon M Tetlow References: <4759DC08.9070600@FreeBSD.org> <20071208163857.GC91919@lor.one-eyed-alien.net> In-Reply-To: X-Enigmail-Version: 0.95.5 OpenPGP: id=D5B2F0FB Content-Type: multipart/mixed; boundary="------------080306060905080008050301" Cc: Brooks Davis , freebsd-arch@freebsd.org Subject: Re: Should libgssapi be hidden behind the MK_KERBEROS knob? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Dec 2007 23:42:12 -0000 This is a multi-part message in MIME format. --------------080306060905080008050301 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Gordon M Tetlow wrote: > > On Dec 8, 2007, at 8:38 AM, Brooks Davis wrote: > >> On Fri, Dec 07, 2007 at 03:49:28PM -0800, Doug Barton wrote: >>> If there is a better list for this, don't hesitate to let me know. >>> >>> I use WITHOUT_KERBEROS=true in /etc/{make|src}.conf, since I don't >>> need or use it. However, this leads to a problem with building the >>> kdelibs3 port. The configure script looks for the presence of >>> libgssapi and the associated headers, and takes that to mean that >>> kerberos is available, and sets things up accordingly. This causes >>> the build to fail when it tries to actually link something to a >>> kerberos library. >>> >>> I realize that GSS can be used for other things besides kerberos, but >>> are we really losing anything by hiding them both under the same knob? >>> If the answer to that is yes, is there any objection to a WITHOUT_GSS >>> knob? >> >> We wouldn't loose anything today, but a without GSS knob makes more >> sense to me. There's at least one other GSS system in fairly wide use >> in the high performance computing world today. > > How about WITHOUT_KERBEROS implies WITHOUT_GSSAPI unless people > specifically ask for GSSAPI? Is that too obscure? That sounds totally reasonable. How does the attached look? Doug -- This .signature sanitized for your protection --------------080306060905080008050301 Content-Type: text/plain; name="gss-no-krb.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="gss-no-krb.diff" Index: lib/Makefile =================================================================== RCS file: /usr/local/ncvs/src/lib/Makefile,v retrieving revision 1.226 diff -u -r1.226 Makefile --- lib/Makefile 17 Nov 2007 21:29:02 -0000 1.226 +++ lib/Makefile 8 Dec 2007 23:24:47 -0000 @@ -31,7 +31,7 @@ libbegemot ${_libbluetooth} libbsnmp libbz2 \ libcalendar libcam libcompat libdevinfo libdevstat libdisk \ libedit libexpat libfetch libftpio libgeom ${_libgpib} \ - libgssapi libipsec \ + ${_libgssapi} libipsec \ ${_libipx} libkiconv libmagic libmemstat ${_libmilter} ${_libmp} \ ${_libncp} ${_libngatm} libopie libpam libpcap \ libpmc ${_libkse} librt ${_libsdp} ${_libsm} ${_libsmb} \ @@ -62,6 +62,14 @@ _libsdp= libsdp .endif +.if ${MK_KERBEROS} != "no" +_libgssapi= libgssapi +.else +.if ${MK_GSSAPI} = "yes" +_libgssapi= libgssapi +.endif +.endif + .if ${MK_IPX} != "no" _libipx= libipx .endif Index: share/man/man5/src.conf.5 =================================================================== RCS file: /usr/local/ncvs/src/share/man/man5/src.conf.5,v retrieving revision 1.20 diff -u -r1.20 src.conf.5 --- share/man/man5/src.conf.5 19 Oct 2007 14:03:05 -0000 1.20 +++ share/man/man5/src.conf.5 8 Dec 2007 23:40:23 -0000 @@ -288,6 +288,10 @@ .\" from FreeBSD: src/tools/build/options/WITHOUT_GROFF,v 1.1 2006/03/21 07:50:49 ru Exp Set to not build .Xr groff 1 . +.It Va WITH_GSSAPI +Set to build libgssapi when +.Va WITHOUT_KERBEROS +is set. .It Va WITH_HESIOD .\" from FreeBSD: src/tools/build/options/WITH_HESIOD,v 1.1 2006/03/21 07:50:50 ru Exp Set to build Hesiod support. @@ -347,6 +351,10 @@ .Bl -item -compact .It .Va WITHOUT_KERBEROS_SUPPORT +.It +.Va WITHOUT_GSSAPI +(unless overridden by +.Va WITH_GSSAPI ) .El .It Va WITHOUT_KERBEROS_SUPPORT .\" from FreeBSD: src/tools/build/options/WITHOUT_KERBEROS_SUPPORT,v 1.1 2006/03/21 07:50:50 ru Exp Index: share/mk/bsd.own.mk =================================================================== RCS file: /usr/local/ncvs/src/share/mk/bsd.own.mk,v retrieving revision 1.69 diff -u -r1.69 bsd.own.mk --- share/mk/bsd.own.mk 20 Oct 2007 19:01:49 -0000 1.69 +++ share/mk/bsd.own.mk 8 Dec 2007 23:29:05 -0000 @@ -381,6 +381,7 @@ # .for var in \ BIND_LIBS \ + GSSAPI \ HESIOD \ IDEA .if defined(WITH_${var}) && defined(WITHOUT_${var}) --------------080306060905080008050301-- From owner-freebsd-arch@FreeBSD.ORG Sat Dec 8 23:44:29 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35C4A16A514; Sat, 8 Dec 2007 23:44:28 +0000 (UTC) (envelope-from remko@FreeBSD.org) Received: from galain.elvandar.org (galain.elvandar.org [217.148.169.56]) by mx1.freebsd.org (Postfix) with ESMTP id 25A6E13C467; Sat, 8 Dec 2007 23:44:27 +0000 (UTC) (envelope-from remko@FreeBSD.org) Received: from evilcoder.xs4all.nl ([195.64.94.120] helo=elvandar.local) by galain.elvandar.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1J19LO-000G3m-MS; Sun, 09 Dec 2007 00:44:26 +0100 Message-ID: <475B2C7C.40903@FreeBSD.org> Date: Sun, 09 Dec 2007 00:45:00 +0100 From: Remko Lodder User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Doug Barton References: <4759DC08.9070600@FreeBSD.org> <20071208163857.GC91919@lor.one-eyed-alien.net> <475B2BD1.7000303@FreeBSD.org> In-Reply-To: <475B2BD1.7000303@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Gordon M Tetlow , Brooks Davis , freebsd-arch@freebsd.org Subject: Re: Should libgssapi be hidden behind the MK_KERBEROS knob? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Dec 2007 23:44:29 -0000 Doug Barton wrote: > Gordon M Tetlow wrote: >> On Dec 8, 2007, at 8:38 AM, Brooks Davis wrote: >> >>> On Fri, Dec 07, 2007 at 03:49:28PM -0800, Doug Barton wrote: >>>> If there is a better list for this, don't hesitate to let me know. >>>> >>>> I use WITHOUT_KERBEROS=true in /etc/{make|src}.conf, since I don't >>>> need or use it. However, this leads to a problem with building the >>>> kdelibs3 port. The configure script looks for the presence of >>>> libgssapi and the associated headers, and takes that to mean that >>>> kerberos is available, and sets things up accordingly. This causes >>>> the build to fail when it tries to actually link something to a >>>> kerberos library. >>>> >>>> I realize that GSS can be used for other things besides kerberos, but >>>> are we really losing anything by hiding them both under the same knob? >>>> If the answer to that is yes, is there any objection to a WITHOUT_GSS >>>> knob? >>> We wouldn't loose anything today, but a without GSS knob makes more >>> sense to me. There's at least one other GSS system in fairly wide use >>> in the high performance computing world today. >> How about WITHOUT_KERBEROS implies WITHOUT_GSSAPI unless people >> specifically ask for GSSAPI? Is that too obscure? > > That sounds totally reasonable. How does the attached look? > > Doug > The attached patch looks good to me, and the proposals as well.. Cheers remko -- /"\ Best regards, | remko@FreeBSD.org \ / Remko Lodder | remko@EFnet X http://www.evilcoder.org/ | / \ ASCII Ribbon Campaign | Against HTML Mail and News