From owner-freebsd-smp Sun Feb 24 9:10:12 2002 Delivered-To: freebsd-smp@freebsd.org Received: from voi.aagh.net (pc1-hart4-0-cust168.mid.cable.ntl.com [62.254.84.168]) by hub.freebsd.org (Postfix) with ESMTP id F181337B400; Sun, 24 Feb 2002 09:10:07 -0800 (PST) Received: from freaky by voi.aagh.net with local (Exim 3.35 #1) id 16f2A2-000NoO-00; Sun, 24 Feb 2002 17:10:06 +0000 Date: Sun, 24 Feb 2002 17:10:06 +0000 From: Thomas Hurst To: freebsd-smp@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: TWiki as promised... Message-ID: <20020224171006.GB90595@voi.aagh.net> Mail-Followup-To: freebsd-smp@FreeBSD.ORG, freebsd-current@FreeBSD.ORG References: <200202240607.WAA929178@meer.meer.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200202240607.WAA929178@meer.meer.net> User-Agent: Mutt/1.3.27i Organization: Not much. X-Operating-System: FreeBSD/4.5-PRERELEASE (i386) X-Uptime: 5:06PM up 66 days, 1:51, 5 users, load averages: 2.06, 2.05, 2.01 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * George V. Neville-Neil (gnn@neville-neil.com) wrote: > > BTW, why don't you want a FreeBSD WikiTerm? > > Well it's a bit odd. Anywhere you put the word FreeBSD lights up as a > link. I guess we could put a generic page under it or something but > it's annoying (to me at least) to see FreeBSD as a link everywhere > when it's mostly just a name. You can always style it to look like normal text. a[href=""] { text-decoration: none; color: black; background-color: inherit; } You probably want to do the same for the :link/:hover/:active/:visited elements too. And you'll want to use a browser with a decent CSS implimentation (i.e. not MSIE :) You can even just put it in your user stylesheet if you don't want it to be on the site :) -- Thomas 'Freaky' Hurst - freaky@aagh.net - http://www.aagh.net/ - On the whole, I'd rather be in Philadelphia. -- W.C. Fields' epitaph To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Feb 25 21: 3: 7 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mail014.syd.optusnet.com.au (mail014.syd.optusnet.com.au [203.2.75.175]) by hub.freebsd.org (Postfix) with ESMTP id 7119537B417 for ; Mon, 25 Feb 2002 21:02:46 -0800 (PST) Received: from BENDER (adlax4-149.dialup.optusnet.com.au [198.142.83.149]) by mail014.syd.optusnet.com.au (8.11.1/8.11.1) with ESMTP id g1Q52cM23495 for ; Tue, 26 Feb 2002 16:02:39 +1100 From: "Martin Minkus" To: Subject: FreeBSD 4.5 Date: Tue, 26 Feb 2002 15:31:51 +1030 Message-ID: <001001c1be82$b74ae7d0$0200000a@BENDER> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0011_01C1BEDA.B9D92BD0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org This is a multi-part message in MIME format. ------=_NextPart_000_0011_01C1BEDA.B9D92BD0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit I'm wondering if anyone knows of any issues in SMP in FreeBSD 4.5-RELEASE. I just recently installed 4.5-R on my old Dual P200, and on IO, it CRRRAAAWWWLLLSSSSS. The keyboard is unusable, i can type and several seconds later something comes up, it really lags, etc, top says the cpu is in System almost all the time. And its not because its swapping (i've seen FreeBSD trash so bad the keyboard doesn't work; no swap is being used, its basically the kernel isn't handling the keyboard interrupts and doing what its meant to be i guess). I couldn't understand it. I booted back to Generic kernel, and all is fine and speedy. I recompiled my custom kernel without SMP, and also, nice and fine and speedy. I recompile my custom kernel with SMP in it again, and it CRAWLS like a dog onec again. So whats up? I *know* i used to run FreeBSD on this box before (3.x, and 4.0, 4.1, 4.2, following the STABLE branch till 4.3 probably) and this problem never existed. The machine then ran Windows XP for a little while before getting 4.5-RELEASE on it, and now this problem has arisen. So something must have broken somewhere... Kernel book WITH SMP: Feb 23 11:34:19 silence /kernel: Copyright (c) 1992-2002 The FreeBSD Project. Feb 23 11:34:19 silence /kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 Feb 23 11:34:19 silence /kernel: The Regents of the University of California. All rights reserved. Feb 23 11:34:19 silence /kernel: FreeBSD 4.5-RELEASE #0: Fri Feb 22 21:27:35 CST 2002 Feb 23 11:34:19 silence /kernel: root@silence.diskiller.net:/usr/src/sys/compile/SILENCE Feb 23 11:34:19 silence /kernel: Timecounter "i8254" frequency 1193182 Hz Feb 23 11:34:19 silence /kernel: CPU: Pentium/P55C (199.31-MHz 586-class CPU) Feb 23 11:34:19 silence /kernel: Origin = "GenuineIntel" Id = 0x543 Stepping = 3 Feb 23 11:34:19 silence /kernel: Features=0x8003bf Feb 23 11:34:19 silence /kernel: real memory = 67108864 (65536K bytes) Feb 23 11:34:19 silence /kernel: avail memory = 62615552 (61148K bytes) Feb 23 11:34:19 silence /kernel: Programming 24 pins in IOAPIC #0 Feb 23 11:34:19 silence /kernel: IOAPIC #0 intpin 2 -> irq 0 Feb 23 11:34:19 silence /kernel: IOAPIC #0 intpin 18 -> irq 11 Feb 23 11:34:19 silence /kernel: IOAPIC #0 intpin 19 -> irq 10 Feb 23 11:34:19 silence /kernel: FreeBSD/SMP: Multiprocessor motherboard Feb 23 11:34:19 silence /kernel: cpu0 (BSP): apic id: 0, version: 0x00030010, at 0xfee00000 Feb 23 11:34:19 silence /kernel: cpu1 (AP): apic id: 1, version: 0x00030010, at 0xfee00000 Feb 23 11:34:19 silence /kernel: io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000 Feb 23 11:34:19 silence /kernel: Preloaded elf kernel "kernel" at 0xc02b3000. Feb 23 11:34:19 silence /kernel: Intel Pentium detected, installing workaround for F00F bug Feb 23 11:34:19 silence /kernel: Using $PIR table, 5 entries at 0xc00fdad0 Feb 23 11:34:19 silence /kernel: npx0: on motherboard Feb 23 11:34:19 silence /kernel: npx0: INT 16 interface Feb 23 11:34:19 silence /kernel: pcib0: on motherboard Feb 23 11:34:19 silence /kernel: pci0: on pcib0 Feb 23 11:34:19 silence /kernel: isab0: at device 7.0 on pci0 Feb 23 11:34:19 silence /kernel: isa0: on isab0 Feb 23 11:34:19 silence /kernel: atapci0: port 0xf000-0xf00f at device 7.1 on pci0 Feb 23 11:34:19 silence /kernel: ata0: at 0x1f0 irq 14 on atapci0 Feb 23 11:34:19 silence /kernel: ata1: at 0x170 irq 15 on atapci0 Feb 23 11:34:19 silence /kernel: rl0: port 0x6400-0x64ff mem 0xe1000000-0xe10000ff irq 11 at device 16.0 on pci0 Feb 23 11:34:19 silence /kernel: rl0: Ethernet address: 00:00:21:ee:49:b9 Feb 23 11:34:19 silence /kernel: miibus0: on rl0 Feb 23 11:34:19 silence /kernel: rlphy0: on miibus0 Feb 23 11:34:19 silence /kernel: rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto Feb 23 11:34:19 silence /kernel: pci0: at 17.0 irq 10 Feb 23 11:34:19 silence /kernel: orm0:
hi :
 
  I would like to confirm = whether 4.5=20 support compaq ProLiant 6000) before i purchase.  That one is a SMP = motherborad with Xeon II 400 cpu.  (pretty old model, 2 yr=20 ago)
 
Thank for your help...
 
Vincent Ma
------=_NextPart_000_0007_01C1C047.AC9D3740-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Feb 27 19:33:17 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 72DA037B402; Wed, 27 Feb 2002 19:33:14 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id OAA11185; Thu, 28 Feb 2002 14:32:44 +1100 Date: Thu, 28 Feb 2002 14:33:15 +1100 (EST) From: Bruce Evans X-X-Sender: To: Kelly Yancey Cc: Bob Van Valzah , Robert Watson , Jorge Aldana , Garance A Drosihn , Subject: Re: Performance vs. Stable In-Reply-To: <20020227121320.X8086-100000@gateway.posi.net> Message-ID: <20020228143126.P51203-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Wed, 27 Feb 2002, Kelly Yancey wrote: > On Wed, 27 Feb 2002, Bruce Evans wrote: > > > > > > > Processor, Processes - factor slower than the best > > > -------------------------------------------------- > > > > I think you are misinterpreting them. The non-starrd results are > > absolute times. E.g., they say that the "null" syscall takes 6.1 usec > > in 4.5-S and 6.1 usec in -current. This is about right. The "null" > > ... > > I know I am going out on a limb to doubt you, but are you sure? The chart No. I just didn't believe the results, and conveniently forgot the details of their annotation. See the followup for corrected results. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 1:50:45 2002 Delivered-To: freebsd-smp@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id E4E3837B417; Thu, 28 Feb 2002 01:50:37 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id B9BDFAE279; Thu, 28 Feb 2002 01:50:37 -0800 (PST) Date: Thu, 28 Feb 2002 01:50:37 -0800 From: Alfred Perlstein To: smp@freebsd.org Cc: dillon@freebsd.org, jhb@freebsd.org, smp@freebsd.org, Chad David Subject: select/poll locking wrong? Message-ID: <20020228095037.GJ80761@elvis.mu.org> References: <20020227111605.GH80761@elvis.mu.org> <20020227122606.A15980@colnta.acns.ab.ca> <20020227193059.GP80761@elvis.mu.org> <20020227124514.A27497@colnta.acns.ab.ca> <20020227202205.GT80761@elvis.mu.org> <20020227235736.A28776@colnta.acns.ab.ca> <20020228073743.GD80761@elvis.mu.org> <20020228011644.A28982@colnta.acns.ab.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020228011644.A28982@colnta.acns.ab.ca> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * Chad David [020228 00:16] wrote: > > A question: > > condvar(9) says that the same mtx must be used with a given cv, but doesn't > select() call cv_*_sig() with selwait and many proc mtx's; as well, > selwakeup() broadcasts on selwait without owning the locks? Am I missing > something here, or are the rules no quite as strict as documented? A condvar should record the first mutex it is used with, if it is used with another mutex before calling some sort of special "reset" function it should trigger an assertion. Ok, based on that it looks like using the proc lock for select/poll seems wrong. The proper way to do this seems to require a global mutex lock that is held for select ops. One of the problems with doing that is that selinfo uses a pid to record which processes/threads are watching it. This makes selwakeup/selrecord call pfind which in turn needs the allproc lock. It's hard to express the badness associated with that so I'll just move on to a proposed solution. Solution: One global lock for select/poll operations. (BSD/os style) cv_wait* interlocks with the global lock. struct selinfo is modified (and bloated :() to contain a pointer to the thread as well as a TAILQ so all the selinfo's registered to a particular thread are on a list hung off of that thread struct. all the lists are protected by the select mutex. selrecord is modified to link the selinfo off the thread and set the back pointer to the thread. (under select lock) selwakeup is modified to grab the select lock, do its checks. On return from from select/poll you walk the list and point all the selinfos to null and remove them from yourself. basically, every time you see: sip->si_pid = 0; you want to remove that selinfo from the list hung off the proc. each time you see sip->si_pid = mypid; you want to link that selinfo into yourself. Because selwakeup is only called when someone is interested in the socket/pipe/whatever it's not going to be a global contention point other than with other threads heavily engaged in select/poll activity. I'd really appreciate some feedback, I'd like it even more if someone did this work, failing that I'll take a shot at it. Or is this not going to work? :) thanks, -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 6:58:51 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id 3E19437B421 for ; Thu, 28 Feb 2002 06:58:35 -0800 (PST) Received: (qmail 14083 invoked from network); 28 Feb 2002 14:58:02 -0000 Received: from unknown (HELO server.baldwin.cx) ([65.91.152.52]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 28 Feb 2002 14:58:02 -0000 Received: from laptop.baldwin.cx (john@laptop.baldwin.cx [192.168.0.4]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g1SEvsG40886; Thu, 28 Feb 2002 09:57:55 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020227110249.GG80761@elvis.mu.org> Date: Thu, 28 Feb 2002 09:57:48 -0500 (EST) From: John Baldwin To: Alfred Perlstein Subject: RE: MPsafe sigio stuff? Cc: smp@freebsd.org, Seigo Tanimura Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On 27-Feb-02 Alfred Perlstein wrote: > Ok, How mpsafe are functions like: > > fgetown/fsetown/funsetown/pgsigio/selwakeup/selrecord ? > > Will they block? What mutexes (if any) do they require? selwakeup/selrecord need the actual selinfo passed in to be locked. They have other issues due to using pfind() but those can be addressed later. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 7:13:13 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by hub.freebsd.org (Postfix) with ESMTP id C894F37B47E for ; Thu, 28 Feb 2002 07:12:07 -0800 (PST) Received: (qmail 20864 invoked from network); 28 Feb 2002 15:12:05 -0000 Received: from unknown (HELO server.baldwin.cx) ([65.90.117.97]) (envelope-sender ) by mail6.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 28 Feb 2002 15:12:05 -0000 Received: from laptop.baldwin.cx (john@laptop.baldwin.cx [192.168.0.4]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g1SF9vG40911 for ; Thu, 28 Feb 2002 10:09:57 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Thu, 28 Feb 2002 10:09:51 -0500 (EST) From: John Baldwin To: smp@FreeBSD.org Subject: Some thoughts on Giant instrumentation Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org This past weekend I took the weekend off from e-mail so I could cool down and take a deeper look at the Giant instrumentation stuff. Mostly I wanted to figure out exactly what about it irked me and see if there was a way to still do instrumentation while addressing the problems I had. I ended up with this: - I didn't like having a lot of code in all the syscalls that we would have to rip out later. I would much prefer the changes to support instrumentation to be more localized. - The current instrumentation covers both safe and unsafe code. Since it covers unsafe code, we have to still leave Giant on all the time, thus the safe code is never out from under Giant and the whole thing is a waste. To fix this, we need to only instrument things that are actually safe and then have the variables default to not getting Giant. Places that aren't safe should still explicitly acquire/release Giant until they are safe. This also allows for us to tell safe code with locking apart from unsafe code that uses similar locking. - As the number of subsystems that are locked grows, we may end up with syscalls that need to check a number of variables. This could result in lots of code bloat in the syscalls, lots of temporary variables, possibly acquiring and releasing Giant many times, etc. I also noted a few other things about instrumentation: - Since this is a debugging aid, it doesn't hurt if it is slightly more coarse grained than it could strictly be. In other words, as this is only temporary debugging code, it shouldn't hurt to hold Giant for a few things that may not need it when a given subsystem has its Giant lock on. - It would be nice if all of the instrumentation checking code could be conditionally compiled out. One other thing regarding Giant: - Since Giant must always be acquired with no other locks held (aside from the magic release/reacquire when a thread sleeps in the kernel) it is easiest if Giant is acquired very soon after a thread enters the kernel from userland. Now putting all this together leads to one possible design: - We attach metadata to each syscall specifying what variables to check as far as if we should acquire Giant or not for a given syscall. - We use a similar instance of metadata for userland traps. - In trap() and syscall() we conditionally acquire Giant based on the appropriate metadata and release Giant after the trap or syscall is finished if it was acquired. One possible implementation: - The metadata consists of an array of pointers to integers terminated with a NULL. The arrays' contents are pointers to the various variables to be checked. Populating these arrays would need to be done separately for traps and syscalls. For traps the array can be statically compiled into each architecture's trap.c. For syscalls, we can add a pointer to the array of variables to check in struct sysent. The arrays can be stored in init_sysent.c as static variables and then store the pointers to the arrays in the sysent array in that file. The arrays can be generated by appending a new optional field to the end of each line that is a comma delimited list of variables to check. The names could be abbreviated (e.g. proc,vfs) and expanded to their fullnames by the parser. If desired this metadata could be stored in another file altogether. - A new function mtx_check_giant(int *vars) would be added that returns true if any of the variables in its array are non-zero. Thus: int mtx_check_giant(int *vars) { int *var; var = vars; while (var != NULL) { if (*var != 0) return (1); var++; } return (0); } The syscall and trap code could then operate like so similar to the existing model: int s; ... s = mtx_check_giant(callp->sy_giantvars); ... if (s) mtx_unlock(&Giant); ... Since the code to do all the instrumentation checks would be localized into trap() and syscall() it could easily be conditionally compiled. Also, simply adding an item to a static array or to a comma-separated list of words is easier to do (and read) than adding to the code. Comments? -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 10:39:19 2002 Delivered-To: freebsd-smp@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id C9D8C37B405; Thu, 28 Feb 2002 10:39:13 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g1SId8D72080; Thu, 28 Feb 2002 13:39:08 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Thu, 28 Feb 2002 13:39:08 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: John Baldwin Cc: smp@FreeBSD.org Subject: Re: Some thoughts on Giant instrumentation In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I'm not opposed to the idea of instrumentation, although that's probably simply that so far we haven't managed to shoot ourselves quite enough to have me wish I had it. One of my concerns with instrumentation has always been the fine-grained issue: throwing Giant back into the mix will change a fair amount about the codepath, and that if we adopt a phased locking of a large structure (proc, for example), then we don't have all that much granularity, suggesting that we'll probably want to look at something that has more complexity than the current scheme. One of my particular concerns has been regarding the following: suppose a structure is covered by three or four relevant locks. We decide to selectively instrument one of those sets of locks (perhaps because we just added it, having determined the others are stable). do we want to be able to say: s = giant_instrument(PROCLOCK_INST | PGRPLOCK_INST | ALLPROC_INST); Where the test can check to see if any of the particular instrumented bits of the mask require Giant, and then DTRT? This would help somewhat with the "collapse the instrumentation" some, assuming the simplification that we don't do too much instrumentation below the system call where it gets complex. Which appears to be something we should avoid doing anyway so that Giant doesn't end up in the wrong place in the lock order. It's probably worth mentioning that instrumentation of this sort has actually been very useful in the TrustedBSD code with regards to enforcement. We can selectively turn off enforcement of MAC policies along two axies: first, we can disable specific policies, and second, we can disable the impact of enforcement across particular types of enforcement. For example, there are sysctls to enable/disable enforcement for inter-process operations, filesystem operations, network operations, socket operations, etc. Likewise, we can turn off enforcement for MLS, Biba, TE, and so on, independently. They are also loader tunables so that you can stick things in loader.conf to recover a machine that's very hosed, or turn things on later when getting past special cases in the boot, or to test a particular system call. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 11:44:11 2002 Delivered-To: freebsd-smp@freebsd.org Received: from hebe.or.intel.com (jffdns02.or.intel.com [134.134.248.4]) by hub.freebsd.org (Postfix) with ESMTP id 9EB8237B41A; Thu, 28 Feb 2002 11:43:56 -0800 (PST) Received: from orsmsxvs040.jf.intel.com (orsmsxvs040.jf.intel.com [192.168.65.206]) by hebe.or.intel.com (8.9.1a+p1/8.9.1/d: relay.m4,v 1.51 2002/02/19 21:12:32 root Exp $) with SMTP id TAA22816; Thu, 28 Feb 2002 19:43:56 GMT Received: from orsmsx26.jf.intel.com ([192.168.65.26]) by orsmsxvs040.jf.intel.com (NAVGW 2.5.1.16) with SMTP id M2002022811492810310 ; Thu, 28 Feb 2002 11:49:28 -0800 Received: by orsmsx26.jf.intel.com with Internet Mail Service (5.5.2653.19) id <1YPL9ZTQ>; Thu, 28 Feb 2002 11:43:56 -0800 Message-ID: From: "Frost, Stephen C" To: "'freebsd-questions@freebsd.org'" , "'freebsd-hackers@freebsd.org'" , "'freebsd-smp@freebsd.org'" Cc: "Frost, Stephen C" Subject: RE: FreeBSD, SMP and Performance Speeds? Date: Thu, 28 Feb 2002 11:43:53 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I'm crossposting to freebsd-smp@freebsd.org, as per suggestion. My original post, edited: > > ... why any kernels compiled with SMP enabled seem > > to be slowing the whole system down? Throughput goes down by 40%. Tasks > > take twice as long to run, etc, etc... > > ... it appears to be system-wide. And is directly linked to > > SMP: two kernels, identical EXCEPT that one has SMP enabled, the other not. > > The enabled kernel that *should* be fully utilizing multi-procs is suddenly > > effectively running at half speed. Thanks to all for replies. Regarding my SMP query, Doc asks: > What sort of throughput? What sort of processes are you > running? Do you > actually have multiple processes fighting for CPU? Yes, I'm using netperf, iperf or nttcp to measure TCP throughput using the server (the box in question) in response to ten simultaneous clients. Chariot allegedly did not show the performance hit. But then, even measuring the process time to run a single simple script shows ~half the speed with SMP enabled. Chris F. asks: > Is this an old Pentium? If so, update to a recent -stable; > a fix was committed a few weeks ago fixing a problem where > the caches on both processors were not enabled on Pentiums. > Otherwise, we have a few PII and PIII boxes here that work > quite under 4.5. This includes multiple configurations, incl: dual PIII 700s, dual PIII 800s, quad PIII Zeon 550s, etc... No old procs, per se. I'm running the released version of 4.5. Was a proc-specific fix implemented *after* its release? Greg L states: > It would also be interesting to see if you get the same results > running 5-CURRENT. While this version isn't suited to production use, > it's based on a very different implementation, and the information > would help us work out what's going on here. Unfortunately, I do not get a whole lot of time to get experimental due to compressed testing schedules but, if a hole opens up, I will attempt to get some testing done using 5-CURRENT. Will report any results to you. Thanks for your interest. This scenario has been replicated on several (virtually any and all) test boxes by multiple engineers. Any other tips are greatly appreciated. TIA - -=C. Stephen Frost=- Intel Corp. ICG - Network Quality Labs Software Test Engineer 503.264.8300 All opinions are my own, not those of Intel Corporation To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 11:47:29 2002 Delivered-To: freebsd-smp@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id B681F37B4CD; Thu, 28 Feb 2002 11:46:08 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id 6C866AE279; Thu, 28 Feb 2002 11:46:03 -0800 (PST) Date: Thu, 28 Feb 2002 11:46:03 -0800 From: Alfred Perlstein To: "Frost, Stephen C" Cc: "'freebsd-questions@freebsd.org'" , "'freebsd-hackers@freebsd.org'" , "'freebsd-smp@freebsd.org'" Subject: Re: FreeBSD, SMP and Performance Speeds? Message-ID: <20020228194603.GC77980@elvis.mu.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.27i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * Frost, Stephen C [020228 11:44] wrote: > > This includes multiple configurations, incl: dual PIII 700s, dual PIII 800s, > quad PIII Zeon 550s, etc... No old procs, per se. I'm running the released > version of 4.5. Was a proc-specific fix implemented *after* its release? Yes. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 12:36:44 2002 Delivered-To: freebsd-smp@freebsd.org Received: from hawk.mail.pas.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by hub.freebsd.org (Postfix) with ESMTP id 092C237B405; Thu, 28 Feb 2002 12:36:40 -0800 (PST) Received: from pool0091.cvx21-bradley.dialup.earthlink.net ([209.179.192.91] helo=mindspring.com) by hawk.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 16gXHu-0001zw-00; Thu, 28 Feb 2002 12:36:26 -0800 Message-ID: <3C7E94BD.B22144BB@mindspring.com> Date: Thu, 28 Feb 2002 12:36:13 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Frost, Stephen C" Cc: "'freebsd-questions@freebsd.org'" , "'freebsd-hackers@freebsd.org'" , "'freebsd-smp@freebsd.org'" Subject: Re: FreeBSD, SMP and Performance Speeds? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org "Frost, Stephen C" wrote: > This includes multiple configurations, incl: dual PIII 700s, dual PIII 800s, > quad PIII Zeon 550s, etc... No old procs, per se. I'm running the released > version of 4.5. Was a proc-specific fix implemented *after* its release? There is code in 4.5 that is incredibly slow on older hardware. THis has been fixed in -current and -stable. Please see the list archives for the patch, if you can not update to -stable. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 13: 1:20 2002 Delivered-To: freebsd-smp@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id AF88E37B437; Thu, 28 Feb 2002 13:00:42 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020228210042.GCGN1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Thu, 28 Feb 2002 21:00:42 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA07396; Thu, 28 Feb 2002 12:47:00 -0800 (PST) Date: Thu, 28 Feb 2002 12:46:59 -0800 (PST) From: Julian Elischer To: Terry Lambert Cc: "Frost, Stephen C" , "'freebsd-questions@freebsd.org'" , "'freebsd-hackers@freebsd.org'" , "'freebsd-smp@freebsd.org'" Subject: Re: FreeBSD, SMP and Performance Speeds? In-Reply-To: <3C7E94BD.B22144BB@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org This is not so relevant because he is NOT RUNNING old hardware! (well not THAT old). On Thu, 28 Feb 2002, Terry Lambert wrote: > "Frost, Stephen C" wrote: > > This includes multiple configurations, incl: dual PIII 700s, dual PIII 800s, > > quad PIII Zeon 550s, etc... No old procs, per se. I'm running the released > > version of 4.5. Was a proc-specific fix implemented *after* its release? > > There is code in 4.5 that is incredibly slow on older > hardware. THis has been fixed in -current and -stable. > > Please see the list archives for the patch, if you can > not update to -stable. > > -- Terry > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 13:44:46 2002 Delivered-To: freebsd-smp@freebsd.org Received: from smtp.WPI.EDU (smtp.WPI.EDU [130.215.24.62]) by hub.freebsd.org (Postfix) with ESMTP id AAAEA37B402 for ; Thu, 28 Feb 2002 13:44:43 -0800 (PST) Received: from bert.WPI.EDU (root@bert.WPI.EDU [130.215.24.121]) by smtp.WPI.EDU (8.12.2/8.12.2) with ESMTP id g1SLig5D015562 for ; Thu, 28 Feb 2002 16:44:42 -0500 (EST) Received: from bert.WPI.EDU (jbrandt@localhost [127.0.0.1]) by bert.WPI.EDU (8.12.2/8.12.2) with ESMTP id g1SLigoM015760 for ; Thu, 28 Feb 2002 16:44:42 -0500 (EST) Received: (from jbrandt@localhost) by bert.WPI.EDU (8.12.2/8.12.2) id g1SLifaq026191 for freebsd-smp@freebsd.org; Thu, 28 Feb 2002 16:44:41 -0500 (EST) Date: Thu, 28 Feb 2002 16:44:41 -0500 From: Joshua T Brandt-- CCC Unix Admin To: freebsd-smp@freebsd.org Subject: ProFusion chipset Message-ID: <20020228214441.GV19005@bert.WPI.EDU> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.25i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Are systems using the ProFusion chipset, like the Dell 8450 multi P3Xeon systems supported? We have a budget (for once!), and I'd like to spend it. 8) Thanks... Josh -- jbrandt@wpi.edu Unix System Administrator Worcester Polytechnic Institute To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 15:15:40 2002 Delivered-To: freebsd-smp@freebsd.org Received: from artemis.drwilco.net (diana.drwilco.net [66.48.127.79]) by hub.freebsd.org (Postfix) with ESMTP id 677BC37B405; Thu, 28 Feb 2002 15:15:27 -0800 (PST) Received: from ceres.drwilco.net (docwilco.xs4all.nl [213.84.68.230]) by artemis.drwilco.net (8.11.6/8.11.6) with ESMTP id g1SNFKV52404 (using TLSv1/SSLv3 with cipher DES-CBC3-SHA (168 bits) verified NO); Thu, 28 Feb 2002 18:15:22 -0500 (EST) (envelope-from drwilco@drwilco.net) Message-Id: <5.1.0.14.0.20020301001536.01cd5610@mail.drwilco.net> X-Sender: lists@mail.drwilco.net X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 01 Mar 2002 00:26:06 +0100 To: "Frost, Stephen C" , "'freebsd-questions@freebsd.org'" , "'freebsd-hackers@freebsd.org'" , "'freebsd-smp@freebsd.org'" From: "Rogier R. Mulhuijzen" Subject: RE: FreeBSD, SMP and Performance Speeds? Cc: "Frost, Stephen C" In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org >Regarding my SMP query, Doc asks: > > What sort of throughput? What sort of processes are you > > running? Do you > > actually have multiple processes fighting for CPU? > >Yes, I'm using netperf, iperf or nttcp to measure TCP throughput using the >server (the box in question) in response to ten simultaneous clients. >Chariot allegedly did not show the performance hit. But then, even >measuring the process time to run a single simple script shows ~half the >speed with SMP enabled. I'm no expert but I'm going to have a shot at this anyways. Comments are welcome. =) When you run a benchmark or a process where network performance is the bottleneck instead of CPU time, you're not going to have SMP help you at all. Currently in the 4.X kernels the kernel can run on only one CPU at a given time. That means that when raw network performance is the bottleneck only one CPU is actually doing the work, and running in SMP mode gives you a lot of overhead. The same is true for a situation where a single single-threaded process is involved. A single-threaded process can only run on one CPU at a given time, so having a 2nd CPU only adds overhead. Have you tried running 4 jobs simultaneously and timing that? So what sort of application are you using exactly, is it multi-process, multithreaded, CPU intensive, network intensive? Where do you think the bottleneck in the performance lies at this moment? Doc To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 15:50:30 2002 Delivered-To: freebsd-smp@freebsd.org Received: from blount.mail.mindspring.net (blount.mail.mindspring.net [207.69.200.226]) by hub.freebsd.org (Postfix) with ESMTP id D3C9A37B42F; Thu, 28 Feb 2002 15:50:21 -0800 (PST) Received: from user-1120bnj.dsl.mindspring.com ([66.32.46.243] helo=europa2) by blount.mail.mindspring.net with smtp (Exim 3.33 #1) id 16gaJX-00020L-00; Thu, 28 Feb 2002 18:50:19 -0500 Message-Id: <3.0.6.32.20020228184845.00da5868@imatowns.com> X-Sender: ggombert@imatowns.com X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32) Date: Thu, 28 Feb 2002 18:48:45 -0500 To: John Baldwin , smp@FreeBSD.org From: Glenn Gombert Subject: Re: Some thoughts on Giant instrumentation In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org First a 're-cap' of the various kernel-debugging resources that are available and then a couple of suggestions from my own (somewhat limited) experience=85 There are basically four different types of kernel debugging 'schemes' that can be configured with what is commonly available in -Current: (1). Kernel debugging can be done (from user space) using the gdb debugging (gdb -k kernelxxx) which can be used to do such simple things as examine/set global variables,etc. (2). A serial console can be set up (using a remote machine) using the ddb debugger, and a kernel compiled on the host machine which had the 'Options ddb' and associated 'break_to_debugger' options enabled in the kernel. =20 (3). The third is a option to remote (over serial line(s)) perform remote kernel debugging using a 'stripped' kernel booted on a second machine and running ddb/gdb from two serial consoles on the second (debugging) machine. (4). The fourth option is to use a VMware virtual machine (running under Current) and use it (as described on the list here at various times) to do a versions of 'gdb debugging'. In my experience this arrangement is very slow but does have the advantage of using only on pc for as a development platform.=20 I think what needs to be added (and is sorely lacking) are some good syscalls (that can be called from 'userland' to perform such things as dump out the 'ktr' buffer from a user land program and show contents of some of the other kernel parameters (when using a test program from user land). It seems like a good set of 'debug syscalls' would be a good addition to the smp code that is -current and would make debugging/development efforts easier and more efficient for everyone :)) Glenn Gombert ggombert@imatowns.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 16: 0:50 2002 Delivered-To: freebsd-smp@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 7DF0F37B400 for ; Thu, 28 Feb 2002 16:00:40 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020301000040.IZPO2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Fri, 1 Mar 2002 00:00:40 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA08291; Thu, 28 Feb 2002 15:46:17 -0800 (PST) Date: Thu, 28 Feb 2002 15:46:16 -0800 (PST) From: Julian Elischer To: "Frost, Stephen C" Cc: "'freebsd-smp@freebsd.org'" Subject: RE: FreeBSD, SMP and Performance Speeds? In-Reply-To: <5.1.0.14.0.20020301001536.01cd5610@mail.drwilco.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Fri, 1 Mar 2002, Rogier R. Mulhuijzen wrote: > > Frost, Stephen C says: > >Regarding my SMP query, Doc asks: [...] > > > >Yes, I'm using netperf, iperf or nttcp to measure TCP throughput using the > >server (the box in question) in response to ten simultaneous clients. > >Chariot allegedly did not show the performance hit. But then, even > >measuring the process time to run a single simple script shows ~half the > >speed with SMP enabled. That sounds wrong if the system is being compared to a UP kernel. What is the script actually DOING? > > I'm no expert but I'm going to have a shot at this anyways. Comments are > welcome. =) > > When you run a benchmark or a process where network performance is the > bottleneck instead of CPU time, you're not going to have SMP help you at > all. Currently in the 4.X kernels the kernel can run on only one CPU at a > given time. That means that when raw network performance is the bottleneck > only one CPU is actually doing the work, and running in SMP mode gives you > a lot of overhead. > > The same is true for a situation where a single single-threaded process is > involved. A single-threaded process can only run on one CPU at a given > time, so having a 2nd CPU only adds overhead. > > Have you tried running 4 jobs simultaneously and timing that? > > So what sort of application are you using exactly, is it multi-process, > multithreaded, CPU intensive, network intensive? Where do you think the > bottleneck in the performance lies at this moment? To expand on this: WHile it may be that we can inprove the throughput of your TCP stack with some tuning, it is true that 4.x SMP will only show a gain for compute bound user processes running in parallel. This is the reason for all the work being done in 5.x. Stephen, If you could tell us a little more about th e exact tests you are seeing we may be able to improve them a little but it's unlikely that we can do much about network throughput.. This is the same problem that Linux had in their earlier MP kernels. It is possible that we can reduce the contention fot the kernel by tuning things if we knew exactly what the system, was doing, but it's also possible that we can't.. The 40% drop (or was that 40% of throughput?) seems a bit much so it's likely we can do SOMETHING. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 17:34: 4 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mail12.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by hub.freebsd.org (Postfix) with ESMTP id 275E137B402 for ; Thu, 28 Feb 2002 17:33:50 -0800 (PST) Received: (qmail 3225 invoked from network); 1 Mar 2002 01:33:46 -0000 Received: from unknown (HELO server.baldwin.cx) ([65.91.152.157]) (envelope-sender ) by mail12.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 1 Mar 2002 01:33:46 -0000 Received: from laptop.baldwin.cx (john@laptop.baldwin.cx [192.168.0.4]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g211XMG42462; Thu, 28 Feb 2002 20:33:22 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Thu, 28 Feb 2002 20:33:15 -0500 (EST) From: John Baldwin To: Robert Watson Subject: Re: Some thoughts on Giant instrumentation Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On 28-Feb-02 Robert Watson wrote: > > I'm not opposed to the idea of instrumentation, although that's probably > simply that so far we haven't managed to shoot ourselves quite enough to > have me wish I had it. One of my concerns with instrumentation has always > been the fine-grained issue: throwing Giant back into the mix will change > a fair amount about the codepath, and that if we adopt a phased locking of > a large structure (proc, for example), then we don't have all that much > granularity, suggesting that we'll probably want to look at something that > has more complexity than the current scheme. > > One of my particular concerns has been regarding the following: suppose a > structure is covered by three or four relevant locks. We decide to > selectively instrument one of those sets of locks (perhaps because we just > added it, having determined the others are stable). do we want to be able > to say: > > s = giant_instrument(PROCLOCK_INST | PGRPLOCK_INST | ALLPROC_INST); Yes. That is why in my example I had a syscall protected by "proc,vfs" meaning that both the proc variable and the vfs variable would be checked. > Where the test can check to see if any of the particular instrumented bits > of the mask require Giant, and then DTRT? This would help somewhat with > the "collapse the instrumentation" some, assuming the simplification that > we don't do too much instrumentation below the system call where it gets > complex. Which appears to be something we should avoid doing anyway so > that Giant doesn't end up in the wrong place in the lock order. *nod* This is pretty much what I said. :) -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 17:34:30 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id 1880237B417 for ; Thu, 28 Feb 2002 17:33:49 -0800 (PST) Received: (qmail 30633 invoked from network); 1 Mar 2002 01:33:40 -0000 Received: from unknown (HELO server.baldwin.cx) ([65.91.152.157]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 1 Mar 2002 01:33:40 -0000 Received: from laptop.baldwin.cx (john@laptop.baldwin.cx [192.168.0.4]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g211XHG42450; Thu, 28 Feb 2002 20:33:21 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <3.0.6.32.20020228184845.00da5868@imatowns.com> Date: Thu, 28 Feb 2002 20:33:10 -0500 (EST) From: John Baldwin To: Glenn Gombert Subject: Re: Some thoughts on Giant instrumentation Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On 28-Feb-02 Glenn Gombert wrote: > > > First a 're-cap' of the various kernel-debugging resources that are > available and then a couple of suggestions from my own (somewhat limited) > experience… > > There are basically four different types of kernel debugging 'schemes' > that can be configured with what is commonly available in -Current: > > (1). Kernel debugging can be done (from user space) using the gdb > debugging (gdb -k kernelxxx) which can be used to do such simple things as > examine/set global variables,etc. > > (2). A serial console can be set up (using a remote machine) using the ddb > debugger, and a kernel compiled on the host machine which had the 'Options > ddb' and associated 'break_to_debugger' options enabled in the kernel. > > (3). The third is a option to remote (over serial line(s)) perform remote > kernel debugging using a 'stripped' kernel booted on a second machine and > running ddb/gdb from two serial consoles on the second (debugging) machine. > > (4). The fourth option is to use a VMware virtual machine (running under > Current) and use it (as described on the list here at various times) to do > a versions of 'gdb debugging'. In my experience this arrangement is very > slow but does have the advantage of using only on pc for as a development > platform. > > I think what needs to be added (and is sorely lacking) are some good > syscalls (that can be called from 'userland' to perform such things as dump > out the 'ktr' buffer from a user land program and show contents of some of > the other kernel parameters (when using a test program from user land). It > seems like a good set of 'debug syscalls' would be a good addition to the > smp code that is -current and would make debugging/development efforts > easier and more efficient for everyone :)) Yep, being able to dump ktr to userland would be rather cool indeed if you'd like to tackle it. :) > Glenn Gombert > ggombert@imatowns.com > -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 19:37:34 2002 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 4E68837B417; Thu, 28 Feb 2002 19:37:28 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g213bQv40727; Thu, 28 Feb 2002 19:37:26 -0800 (PST) (envelope-from dillon) Date: Thu, 28 Feb 2002 19:37:26 -0800 (PST) From: Matthew Dillon Message-Id: <200203010337.g213bQv40727@apollo.backplane.com> To: Alfred Perlstein Cc: smp@FreeBSD.ORG, dillon@FreeBSD.ORG, jhb@FreeBSD.ORG, smp@FreeBSD.ORG, Chad David Subject: Re: select/poll locking wrong? References: <20020227111605.GH80761@elvis.mu.org> <20020227122606.A15980@colnta.acns.ab.ca> <20020227193059.GP80761@elvis.mu.org> <20020227124514.A27497@colnta.acns.ab.ca> <20020227202205.GT80761@elvis.mu.org> <20020227235736.A28776@colnta.acns.ab.ca> <20020228073743.GD80761@elvis.mu.org> <20020228011644.A28982@colnta.acns.ab.ca> <20020228095037.GJ80761@elvis.mu.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org You've lost me. I am going to try to rationalize your description. si_pid is set to 0 in selwakeup() and set to non-0 in selrecord(). Apparently, though, si_pid is also cached so if a select() returns and the event has not occured, si_pid will still be left set to the last process that requested the event. If the process later goes away another selrecord() will simply overwrite the cache, otherwise it uses the selwait conflict domain. The cache appears to be there so select(), when it is getting ready to return, can avoid traversing the list and removing the events that have not occured. This results in a potentially stale si_pid because now the process can exit without doing any further cleanups. This implies that we can add a TAILQ list to each thread and associate selinfo records with the thread (or process?) their si_pid has been assigned to. When the thread exits, the si_pid & si_thread fields can be wiped right then and there with a simple traversal so they aren't left stale. Is this what you are saying? I am still somewhat confused as to the distinction between thread and process in the context of select(). -Matt Matthew Dillon : One global lock for select/poll operations. (BSD/os style) : : cv_wait* interlocks with the global lock. : : struct selinfo is modified (and bloated :() to contain a pointer : to the thread as well as a TAILQ so all the selinfo's registered : to a particular thread are on a list hung off of that thread : struct. : : all the lists are protected by the select mutex. : : selrecord is modified to link the selinfo off the thread and set : the back pointer to the thread. (under select lock) : : selwakeup is modified to grab the select lock, do its checks. : : On return from from select/poll you walk the list and point all : the selinfos to null and remove them from yourself. : :basically, every time you see: : sip->si_pid = 0; :you want to remove that selinfo from the list hung off the proc. : :each time you see : sip->si_pid = mypid; :you want to link that selinfo into yourself. : :Because selwakeup is only called when someone is interested in :the socket/pipe/whatever it's not going to be a global contention :point other than with other threads heavily engaged in select/poll :activity. : : :I'd really appreciate some feedback, I'd like it even more if someone :did this work, failing that I'll take a shot at it. : :Or is this not going to work? :) : :thanks, :-- :-Alfred Perlstein [alfred@freebsd.org] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 19:58:37 2002 Delivered-To: freebsd-smp@freebsd.org Received: from thalia.fm.intel.com (fmfdns02.fm.intel.com [132.233.247.11]) by hub.freebsd.org (Postfix) with ESMTP id DF6D137B417 for ; Thu, 28 Feb 2002 19:58:32 -0800 (PST) Received: from fmsmsxvs043.fm.intel.com (fmsmsxv043-1.fm.intel.com [132.233.48.128]) by thalia.fm.intel.com (8.9.1a+p1/8.9.1/d: relay.m4,v 1.51 2002/02/19 21:12:32 root Exp $) with SMTP id DAA06326 for ; Fri, 1 Mar 2002 03:58:32 GMT Received: from fmsmsx28.fm.intel.com ([132.233.42.28]) by fmsmsxvs043.fm.intel.com (NAVGW 2.5.1.16) with SMTP id M2002022819573908171 ; Thu, 28 Feb 2002 19:57:39 -0800 Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Thu, 28 Feb 2002 19:58:32 -0800 Message-ID: From: "Frost, Stephen C" To: "'Julian Elischer'" Cc: "'freebsd-smp@freebsd.org'" , "Frost, Stephen C" , "Glick, Kevin" Subject: RE: FreeBSD, SMP and Performance Speeds? Date: Thu, 28 Feb 2002 19:58:29 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="ISO-8859-1" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > >But then, even > > >measuring the process time to run a single simple script > shows ~half the > > >speed with SMP enabled. > > That sounds wrong if the system is being compared to a UP kernel. > > What is the script actually DOING? This particular script was likely running a single process and not utilizing multi-proc functionality. > Stephen, If you could tell us a little more about th e exact > tests you > are seeing we may be able to improve them a little but it's > unlikely that > we can do much about network throughput.. This is the same > problem that > Linux had in their earlier MP kernels. The tests I am running are scripts that allow a tester to setup 10 separate processes on the server to run 10 separate netperf sessions simultaneously to ten clients, measuring the throughput speed of each session separate and summed. And again, when similar tests were run using iperf and nttcp, the same performance hits were noted with SMP running. > It is possible that we can reduce the contention fot the kernel > by tuning things if we knew exactly what the system, was > doing, but it's > also possible that we can't.. The 40% drop > (or was that 40% of throughput?) seems a bit much so it's > likely we can do > SOMETHING. It was a 40% drop in throughput, which surprised us. Your consideration is greatly appreciated. However, we are likely not in a position where tuning each box to maximize throughput is a viable option. These are, after all, test boxes, not 'performance' boxes. We are testing NICs and drivers here and the purpose of my original post was to determine if the performance drop we were seeing using SMP was expected. I'm still not clear on whether it is expected or not. It's rather important that we are testing with release versions of FreeBSD, as we are testing for 'the real world', although there are certainly times when we have the bandwidth to test upcoming releases. So, if I am reading you BSD gurus correctly, what I am seeing is (more or less) to be expected, and it may be in our best interest to wait for a release where this issue is resolved. Am I reading this correctly? Thanks again for your patience and guidance. Sincerely - -=C. Stephen Frost=- Intel Corp. ICG - Network Quality Labs Software Test Engineer 503.264.8300 Opinions are my own, not those of Intel Corp. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 20: 6: 8 2002 Delivered-To: freebsd-smp@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id 383D037B405; Thu, 28 Feb 2002 20:06:04 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id E6934AE27F; Thu, 28 Feb 2002 20:06:03 -0800 (PST) Date: Thu, 28 Feb 2002 20:06:03 -0800 From: Alfred Perlstein To: Matthew Dillon Cc: smp@FreeBSD.ORG, dillon@FreeBSD.ORG, jhb@FreeBSD.ORG, Chad David Subject: Re: select/poll locking wrong? Message-ID: <20020301040603.GE77980@elvis.mu.org> References: <20020227111605.GH80761@elvis.mu.org> <20020227122606.A15980@colnta.acns.ab.ca> <20020227193059.GP80761@elvis.mu.org> <20020227124514.A27497@colnta.acns.ab.ca> <20020227202205.GT80761@elvis.mu.org> <20020227235736.A28776@colnta.acns.ab.ca> <20020228073743.GD80761@elvis.mu.org> <20020228011644.A28982@colnta.acns.ab.ca> <20020228095037.GJ80761@elvis.mu.org> <200203010337.g213bQv40727@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200203010337.g213bQv40727@apollo.backplane.com> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * Matthew Dillon [020228 19:37] wrote: > You've lost me. I am going to try to rationalize your description. > > si_pid is set to 0 in selwakeup() and set to non-0 in selrecord(). > Apparently, though, si_pid is also cached so if a select() returns > and the event has not occured, si_pid will still be left set to the > last process that requested the event. If the process later goes > away another selrecord() will simply overwrite the cache, otherwise > it uses the selwait conflict domain. Yes. > > The cache appears to be there so select(), when it is getting ready > to return, can avoid traversing the list and removing the events that > have not occured. This results in a potentially stale si_pid because > now the process can exit without doing any further cleanups. Yes, this is why we must now walk the list when leaving select/poll. > This implies that we can add a TAILQ list to each thread and associate > selinfo records with the thread (or process?) Thread, not process. > their si_pid has been > assigned to. When the thread exits, the si_pid & si_thread fields > can be wiped right then and there with a simple traversal so they > aren't left stale. si_pid is going to go away, see below. > Is this what you are saying? I am still somewhat confused as to the > distinction between thread and process in the context of select(). Almost. There is no more si_pid, there will be a si_procp (back pointer to proc struct) and si_proclst (list of all selinfo associated with process). Yuo are correct, we now will have to traverse the list and null out the si_procp on return from select/poll. Forget thread/process, they are all threads. The only reason there is confustion is because si_pid is per-process that's why the thread pointer was added to the selinfo struct. Anyhow, hope that clarifies. Any other ideas or improvements? Any idea on how we can avoid the list traversal at the end of select? -- -Alfred Perlstein [alfred@freebsd.org] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Feb 28 20:41:35 2002 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 410FC37B402; Thu, 28 Feb 2002 20:41:32 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g214fUT41077; Thu, 28 Feb 2002 20:41:30 -0800 (PST) (envelope-from dillon) Date: Thu, 28 Feb 2002 20:41:30 -0800 (PST) From: Matthew Dillon Message-Id: <200203010441.g214fUT41077@apollo.backplane.com> To: Alfred Perlstein Cc: smp@FreeBSD.ORG, dillon@FreeBSD.ORG, jhb@FreeBSD.ORG, Chad David Subject: Re: select/poll locking wrong? References: <20020227111605.GH80761@elvis.mu.org> <20020227122606.A15980@colnta.acns.ab.ca> <20020227193059.GP80761@elvis.mu.org> <20020227124514.A27497@colnta.acns.ab.ca> <20020227202205.GT80761@elvis.mu.org> <20020227235736.A28776@colnta.acns.ab.ca> <20020228073743.GD80761@elvis.mu.org> <20020228011644.A28982@colnta.acns.ab.ca> <20020228095037.GJ80761@elvis.mu.org> <200203010337.g213bQv40727@apollo.backplane.com> <20020301040603.GE77980@elvis.mu.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org :> :> The cache appears to be there so select(), when it is getting ready :> to return, can avoid traversing the list and removing the events that :> have not occured. This results in a potentially stale si_pid because :> now the process can exit without doing any further cleanups. : :Yes, this is why we must now walk the list when leaving select/poll. : :.. :Anyhow, hope that clarifies. Any other ideas or improvements? : :Any idea on how we can avoid the list traversal at the end of :select? : :-- :-Alfred Perlstein [alfred@freebsd.org] Walking the whole list is one solution. Another would be to link selinfo into a thread-based list (the thread it has been asigned to) and just walk *that* list on exit in order to avoid having to lock the entire descriptor list on every select(). I'm still not sure which of these two approaches you are going for. I'm not sure how this would tie into KSE. It might not be efficient to walk the list on thread-exit since KSE creates and drops threads all the time. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Mar 1 0:20:13 2002 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 0600937B400; Fri, 1 Mar 2002 00:20:11 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g218KAF42627; Fri, 1 Mar 2002 00:20:10 -0800 (PST) (envelope-from dillon) Date: Fri, 1 Mar 2002 00:20:10 -0800 (PST) From: Matthew Dillon Message-Id: <200203010820.g218KAF42627@apollo.backplane.com> To: Robert Watson Cc: John Baldwin , smp@FreeBSD.ORG Subject: Re: Some thoughts on Giant instrumentation References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org :I'm not opposed to the idea of instrumentation, although that's probably :simply that so far we haven't managed to shoot ourselves quite enough to :have me wish I had it. One of my concerns with instrumentation has always :been the fine-grained issue: throwing Giant back into the mix will change :a fair amount about the codepath, and that if we adopt a phased locking of :a large structure (proc, for example), then we don't have all that much :granularity, suggesting that we'll probably want to look at something that :has more complexity than the current scheme. All I am advocating with Giant instrumentation is that when we have Giant in a particular place in the code and we believe it is safe to remove it, that instead of removing it we instrument it. That's it. There is nothing complicated about it. I think some people have blown Giant instrumentation completely out of proportion by trying to over-engineer the concept I put forth. Go back and look at the ucred patch I put forth two weeks ago (which still hasn't been resolved). All I did was to instrument Giant in a simple and straightforward way. :s = giant_instrument(PROCLOCK_INST | PGRPLOCK_INST | ALLPROC_INST); : :Where the test can check to see if any of the particular instrumented bits The current implmentation works like this: s = mtx_lock_giant(some_sysctl_global); ... mtx_unlock_giant(s). There is nothing preventing one from doing this: s = mtx_lock_giant(some_sysctl_global || some_other_sysctl_global); ... mtx_unlock_giant(s). I am not adverse to changing the API slightly to make it possible to inline and conditionally compile the individual sysctl capabilities or to conditionally compile the entire procedure, but I do not feel any paricular need to do so at this time because the calls simply do not eat up any significant amount of cpu. I am not adverse to someone else doing the work, though. As I told Peter, as long as it isn't turned into a mess of #define's he or someone else can conditionalize it. An inline, a kernel option/#defined conditional compilation mask, and a bitmask would be heavily optimizable by GCC and an excellent solution. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Mar 1 8:23:29 2002 Delivered-To: freebsd-smp@freebsd.org Received: from smtp.umr.edu (mrelay1.cc.umr.edu [131.151.1.120]) by hub.freebsd.org (Postfix) with ESMTP id CA2BB37B405; Fri, 1 Mar 2002 08:23:04 -0800 (PST) Received: from saucer.cc.umr.edu (root@saucer.cc.umr.edu [131.151.35.14]) via ESMTP by mrelay1.cc.umr.edu (8.12.1/) id g21GMxDU028991; Fri, 1 Mar 2002 10:22:59 -0600 Received: (from nkunkee@localhost) by saucer.cc.umr.edu (8.12.1/8.12.0.Beta7) id g21GMxYP025376; Fri, 1 Mar 2002 10:22:59 -0600 (CST) Date: Fri, 1 Mar 2002 10:22:58 -0600 From: Nathan Kunkee To: hackers@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: freebsd-hackers SMP performance question Message-ID: <20020301162258.GA24102@umr.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.25i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > I'm crossposting to freebsd-smp@freebsd.org, as per suggestion. > > My original post, edited: > > > > ... why any kernels compiled with SMP enabled seem > > > to be slowing the whole system down? Throughput goes down by 40%. > Tasks > > > take twice as long to run, etc, etc... > > > ... it appears to be system-wide. And is directly linked to > > > SMP: two kernels, identical EXCEPT that one has SMP enabled, the other > not. > > > The enabled kernel that *should* be fully utilizing multi-procs is > suddenly > > > effectively running at half speed. > > Thanks to all for replies. Since i am having the same problem, I'll tack my info next to yours and see if both of us can get an answer. I'm using a dual P120, 64M ram, built my own SMP kernel, and have noticed the same thing: performance/through put slows to nothing. my best example of this is when in X I move the mouse. no mouse motion, ~6% cpu usage. move the mouse, ~40-55% usage. Are all interrupts being mapped to a single cpu?? > > Regarding my SMP query, Doc asks: > > What sort of throughput? What sort of processes are you > > running? Do you > > actually have multiple processes fighting for CPU? > > Yes, I'm using netperf, iperf or nttcp to measure TCP throughput using the > server (the box in question) in response to ten simultaneous clients. > Chariot allegedly did not show the performance hit. But then, even > measuring the process time to run a single simple script shows ~half the > speed with SMP enabled. > Yes. does KDE with konqueror (and user ppp in background) count? konqueror is so slow it is nearly unusable. I figured that dual cpus would provide closer to my K6 233 performance, meaning comfortable interaction. building the kernel (make -j2 depend; make -j2; make -j2 install) seems to take as long as a single proc.. don't have actual wall clock time, got too bored. I also have some other quirks that i think are related. Occasionaly, my scsi drive (ahb eisa) will timeout while trying to reset. loading an smp kernel helped reduce this trouble, but not eliminate it. My sound card, which works fine AFAIK in other machines, has timeout trouble in this one. how can i determine if these are hardware troubles, or SMP related?? Is there a way to dynamicaly disable/enable a cpu? so that i can disable one and see how that affects performance?? I tried sysctl machdep.smp_active=1, or =2, but according to top, no difference. both procs are still getting programs and time slices. > Chris F. asks: > > Is this an old Pentium? If so, update to a recent -stable; > > a fix was committed a few weeks ago fixing a problem where > > the caches on both processors were not enabled on Pentiums. > > Otherwise, we have a few PII and PIII boxes here that work > > quite under 4.5. > > This includes multiple configurations, incl: dual PIII 700s, dual PIII 800s, > quad PIII Zeon 550s, etc... No old procs, per se. I'm running the released > version of 4.5. Was a proc-specific fix implemented *after* its release? > yes, p120. I will endeavor to setup and download the latest this weekend. will take some time since i'm on dialup.... > Greg L states: > > It would also be interesting to see if you get the same results > > running 5-CURRENT. While this version isn't suited to production use, > > it's based on a very different implementation, and the information > > would help us work out what's going on here. > I will endeavor to get this d/l as well. again, dialup will not make this quick or easy... > Unfortunately, I do not get a whole lot of time to get experimental due to > compressed testing schedules but, if a hole opens up, I will attempt to get > some testing done using 5-CURRENT. Will report any results to you. Thanks > for your interest. > > This scenario has been replicated on several (virtually any and all) test > boxes by multiple engineers. Any other tips are greatly appreciated. > > TIA - > > -=C. Stephen Frost=- > Intel Corp. > ICG - Network Quality Labs > Software Test Engineer > 503.264.8300 > > All opinions are my own, not those of Intel Corporation As well, thanks for all the information and help. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Mar 1 8:47:28 2002 Delivered-To: freebsd-smp@freebsd.org Received: from smtp10.atl.mindspring.net (smtp10.atl.mindspring.net [207.69.200.246]) by hub.freebsd.org (Postfix) with ESMTP id 633A837B41B; Fri, 1 Mar 2002 08:47:13 -0800 (PST) Received: from pool-63.49.209.169.troy.grid.net ([63.49.209.169] helo=europa2) by smtp10.atl.mindspring.net with smtp (Exim 3.33 #1) id 16gqBQ-0004oJ-00; Fri, 01 Mar 2002 11:47:01 -0500 Message-Id: <3.0.6.32.20020301114511.00dafba0@imatowns.com> X-Sender: ggombert@imatowns.com X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32) Date: Fri, 01 Mar 2002 11:45:11 -0500 To: Nathan Kunkee , hackers@FreeBSD.ORG, smp@FreeBSD.ORG From: Glenn Gombert Subject: Re: freebsd-hackers SMP performance question In-Reply-To: <20020301162258.GA24102@umr.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org There are *many* issue(s) in -Current right now realted to KDE operationg (or not operating) as the case manybe. Older versions of KDE (2.1 for example) seem to run much better than the current 2.2 version. I have much better luck when running X in FreeBSD/Current by using a 'lighter weight' wm such as blackbox or icewm..... GG At 10:22 AM 3/1/2002 -0600, Nathan Kunkee wrote: >> >> I'm crossposting to freebsd-smp@freebsd.org, as per suggestion. >> >> My original post, edited: >> >> > > ... why any kernels compiled with SMP enabled seem >> > > to be slowing the whole system down? Throughput goes down by 40%. >> Tasks >> > > take twice as long to run, etc, etc... >> > > ... it appears to be system-wide. And is directly linked to >> > > SMP: two kernels, identical EXCEPT that one has SMP enabled, the other >> not. >> > > The enabled kernel that *should* be fully utilizing multi-procs is >> suddenly >> > > effectively running at half speed. >> >> Thanks to all for replies. >Since i am having the same problem, I'll tack my info next to yours and see if >both of us can get an answer. > >I'm using a dual P120, 64M ram, built my own SMP kernel, and have noticed the >same thing: performance/through put slows to nothing. my best example of this >is when in X I move the mouse. no mouse motion, ~6% cpu usage. move the mouse, >~40-55% usage. Are all interrupts being mapped to a single cpu?? > > >> >> Regarding my SMP query, Doc asks: >> > What sort of throughput? What sort of processes are you >> > running? Do you >> > actually have multiple processes fighting for CPU? >> >> Yes, I'm using netperf, iperf or nttcp to measure TCP throughput using the >> server (the box in question) in response to ten simultaneous clients. >> Chariot allegedly did not show the performance hit. But then, even >> measuring the process time to run a single simple script shows ~half the >> speed with SMP enabled. >> >Yes. does KDE with konqueror (and user ppp in background) count? konqueror is >so slow it is nearly unusable. I figured that dual cpus would provide closer >to my K6 233 performance, meaning comfortable interaction. building the kernel >(make -j2 depend; make -j2; make -j2 install) seems to take as long as a single >proc.. don't have actual wall clock time, got too bored. > >I also have some other quirks that i think are related. Occasionaly, my scsi >drive (ahb eisa) will timeout while trying to reset. loading an smp kernel helped >reduce this trouble, but not eliminate it. My sound card, which works fine AFAIK >in other machines, has timeout trouble in this one. how can i determine if >these are hardware troubles, or SMP related?? > >Is there a way to dynamicaly disable/enable a cpu? so that i can disable one and >see how that affects performance?? I tried sysctl machdep.smp_active=1, or =2, >but according to top, no difference. both procs are still getting programs and >time slices. > >> Chris F. asks: >> > Is this an old Pentium? If so, update to a recent -stable; >> > a fix was committed a few weeks ago fixing a problem where >> > the caches on both processors were not enabled on Pentiums. >> > Otherwise, we have a few PII and PIII boxes here that work >> > quite under 4.5. >> >> This includes multiple configurations, incl: dual PIII 700s, dual PIII 800s, >> quad PIII Zeon 550s, etc... No old procs, per se. I'm running the released >> version of 4.5. Was a proc-specific fix implemented *after* its release? >> >yes, p120. I will endeavor to setup and download the latest this weekend. will >take some time since i'm on dialup.... > >> Greg L states: >> > It would also be interesting to see if you get the same results >> > running 5-CURRENT. While this version isn't suited to production use, >> > it's based on a very different implementation, and the information >> > would help us work out what's going on here. >> >I will endeavor to get this d/l as well. again, dialup will not make this quick >or easy... > >> Unfortunately, I do not get a whole lot of time to get experimental due to >> compressed testing schedules but, if a hole opens up, I will attempt to get >> some testing done using 5-CURRENT. Will report any results to you. Thanks >> for your interest. >> >> This scenario has been replicated on several (virtually any and all) test >> boxes by multiple engineers. Any other tips are greatly appreciated. >> >> TIA - >> >> -=C. Stephen Frost=- >> Intel Corp. >> ICG - Network Quality Labs >> Software Test Engineer >> 503.264.8300 >> >> All opinions are my own, not those of Intel Corporation > >As well, thanks for all the information and help. > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-smp" in the body of the message > Glenn Gombert ggombert@imatowns.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Mar 1 12:24: 9 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mass.dis.org (dhcp45-21.dis.org [216.240.45.21]) by hub.freebsd.org (Postfix) with ESMTP id 1819237B405 for ; Fri, 1 Mar 2002 12:24:06 -0800 (PST) Received: from mass.dis.org (localhost [127.0.0.1]) by mass.dis.org (8.11.6/8.11.6) with ESMTP id g21KNcK01173; Fri, 1 Mar 2002 12:23:39 -0800 (PST) (envelope-from msmith@mass.dis.org) Message-Id: <200203012023.g21KNcK01173@mass.dis.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Joshua T Brandt-- CCC Unix Admin Cc: freebsd-smp@freebsd.org Subject: Re: ProFusion chipset In-reply-to: Your message of "Thu, 28 Feb 2002 16:44:41 EST." <20020228214441.GV19005@bert.WPI.EDU> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 01 Mar 2002 12:23:38 -0800 From: Michael Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > Are systems using the ProFusion chipset, like the Dell 8450 multi P3Xeon > systems supported? Yes. > We have a budget (for once!), and I'd like to spend it. 8) IMO, you're better off spending the money on several smaller systems. I don't feel that the ProFusion machines scale well, although this is as much a gut feeling as anything. -- To announce that there must be no criticism of the president, or that we are to stand by the president, right or wrong, is not only unpatriotic and servile, but is morally treasonable to the American public. - Theodore Roosevelt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Mar 1 14: 0:40 2002 Delivered-To: freebsd-smp@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id D8F1537B402; Fri, 1 Mar 2002 14:00:33 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020301220033.PNOW1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Fri, 1 Mar 2002 22:00:33 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA13086; Fri, 1 Mar 2002 13:52:27 -0800 (PST) Date: Fri, 1 Mar 2002 13:52:26 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: Alfred Perlstein , smp@FreeBSD.ORG, dillon@FreeBSD.ORG, jhb@FreeBSD.ORG, Chad David Subject: Re: select/poll locking wrong? In-Reply-To: <200203010441.g214fUT41077@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Thu, 28 Feb 2002, Matthew Dillon wrote: > :> > :> The cache appears to be there so select(), when it is getting ready > :> to return, can avoid traversing the list and removing the events that > :> have not occured. This results in a potentially stale si_pid because > :> now the process can exit without doing any further cleanups. > : > :Yes, this is why we must now walk the list when leaving select/poll. > : > :.. > :Anyhow, hope that clarifies. Any other ideas or improvements? > : > :Any idea on how we can avoid the list traversal at the end of > :select? > : > :-- > :-Alfred Perlstein [alfred@freebsd.org] > > Walking the whole list is one solution. Another would be to link > selinfo into a thread-based list (the thread it has been asigned to) > and just walk *that* list on exit in order to avoid having to lock > the entire descriptor list on every select(). I'm still not sure > which of these two approaches you are going for. > > I'm not sure how this would tie into KSE. It might not be efficient > to walk the list on thread-exit since KSE creates and drops threads > all the time. I need to add a generation number to threads. And possibly a TID (like PID) (maybe the same thing?) The TID would be just an incrementing number taken from a field in the proc. thus the full TID would be PID:TID It hasn't been needed yet, but I can see it coming.. > > -Matt > Matthew Dillon > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Mar 2 10:16:13 2002 Delivered-To: freebsd-smp@freebsd.org Received: from posgate.acis.com.au (posgate.acis.com.au [203.14.230.14]) by hub.freebsd.org (Postfix) with ESMTP id 0E69437B400 for ; Sat, 2 Mar 2002 10:16:08 -0800 (PST) Received: from bullseye.apana.org.au (dialup-2.aaa.net.au [203.14.230.67]) by posgate.acis.com.au (8.11.6/8.11.6) with ESMTP id g22IG3u12855; Sun, 3 Mar 2002 05:16:04 +1100 Received: from bullseye.apana.org.au (tenring.andymac.org [203.9.107.238]) by bullseye.apana.org.au (8.11.6/8.11.6) with ESMTP id g22DTdd08227; Sun, 3 Mar 2002 00:29:39 +1100 (EST) (envelope-from andymac@bullseye.apana.org.au) Date: Sat, 2 Mar 2002 23:27:14 +1100 (EDT) From: Andrew MacIntyre To: Nathan Kunkee Cc: Subject: Re: freebsd-hackers SMP performance question In-Reply-To: <20020301162258.GA24102@umr.edu> Message-ID: X-X-Sender: andymac@bullseye.andymac.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Fri, 1 Mar 2002, Nathan Kunkee wrote: > I'm using a dual P120, 64M ram, built my own SMP kernel, and have noticed the > same thing: performance/through put slows to nothing. my best example of this > is when in X I move the mouse. no mouse motion, ~6% cpu usage. move the mouse, > ~40-55% usage. Are all interrupts being mapped to a single cpu?? A dual P120 would definitely be subject to the fix made post 4.5. Search the archives for a thread with the subject "P5 vs. SMP, part 2"; one followup contains a patch for a Pentium specific issue. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message