From owner-freebsd-smp Sun Mar 31 9:47:57 2002 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id EB0FE37B41D for ; Sun, 31 Mar 2002 09:47:51 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g2VHlII89488; Sun, 31 Mar 2002 09:47:18 -0800 (PST) (envelope-from dillon) Date: Sun, 31 Mar 2002 09:47:18 -0800 (PST) From: Matthew Dillon Message-Id: <200203311747.g2VHlII89488@apollo.backplane.com> To: John Polstra Cc: smp@FreeBSD.ORG Subject: Re: RE: Syscall contention tests return, userret() bugs/issues. References: <200203292207.g2TM7Fi67491@apollo.backplane.com> <200203311729.g2VHT5h18352@vashon.polstra.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org :> the cache, on IA32, it's write-through to main-memory which :> invalidates all other cpu's caches for that cache line. : :No, it's not nearly that bad. See section 9.2 of the IA-32 Intel :Architecture Software Developer's Manual, Volume 3 (24547203.pdf from :developer.intel.com). It describes what happens in a couple of :different scenarios, for example: This is not a case that typically occurs. Because intel caches are write-through the processor that does the write will flush the dirty cache line to main memory in fairly short order. I'm fairly sure that the flush causes all other processors holding that cache line to invalidate their cache line rather then snoop/update it. The case you describe below occurs when a processor does not have *ANY* data cached and attempts to read a memory location. Since it is leaving the cpu, this is a snoopable transaction. Any other cpu in the system that has the cache line in question can theoretically supply the data to the originating cpu rather then having to wait for the memory controller to supply it. The manual section you describe below indicates that Intel implements this at least when the data is dirty in some cpu's cache. You or I would have to read-up on the specs to determine whether it also occurs if the data is clean. In anycase, I ran real live tests. The stall is real and just as bad as if you had used a locked bus cycle. I don't know whether it is retrieving the data via snooping or via the memory controller but the stall is there either way. -Matt Matthew Dillon : Beginning with the P6 family processors, if a processor detects : (through snooping) that another processor is trying to access a : memory location that it has modified in its cache, but has not : yet written back to system memory, the snooping processor will : signal the other processor (by means of the HITM# signal) that : the cache line is held in modified state and will preform an : implicit write-back of the modified data. The implicit write-back : is transferred directly to the initial requesting processor and : snooped by the memory controller to assure that system memory has : been updated. Here, the processor with the valid data may pass : the data to the other proces-sors without actually writing it to : system memory; however, it is the responsibility of the memory : controller to snoop this operation and update memory. : :John :-- : John Polstra : John D. Polstra & Co., Inc. Seattle, Washington USA : "Disappointment is a good sign of basic intelligence." -- Chögyam Trungpa To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message