From owner-freebsd-smp Sun Dec 6 04:00:14 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id EAA17980 for freebsd-smp-outgoing; Sun, 6 Dec 1998 04:00:14 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from solaris.matti.ee (solaris.matti.ee [194.126.98.135]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id EAA17975 for ; Sun, 6 Dec 1998 04:00:11 -0800 (PST) (envelope-from vallo@myhakas.matti.ee) Received: from myhakas.matti.ee (myhakas [194.126.98.150]) by solaris.matti.ee (8.8.8/8.8.8.s) with ESMTP id OAA08592; Sun, 6 Dec 1998 14:00:05 +0200 (EET) Received: (from vallo@localhost) by myhakas.matti.ee (8.9.1/8.9.1) id OAA04672; Sun, 6 Dec 1998 14:00:03 +0200 (EET) (envelope-from vallo) Message-ID: <19981206140003.B4618@matti.ee> Date: Sun, 6 Dec 1998 14:00:03 +0200 From: Vallo Kallaste To: Sam Pigg , freebsd-smp@FreeBSD.ORG Subject: Re: Tyan Thunder 100 Reply-To: vallo@matti.ee References: <199812051635.IAA07988@phred.redbacknetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: <199812051635.IAA07988@phred.redbacknetworks.com>; from Sam Pigg on Sat, Dec 05, 1998 at 08:35:54AM -0800 Organization: =?iso-8859-1?Q?AS_Matti_B=FCrootehnika?= Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Sam Pigg wrote: > >> I'm using the S1836DLUAN with two 450Mhz processors. It's been working > >> fine. > > > >I can confirm. I'm using two 400Mhz processors. Onboard SCSI works > >well, too. > > Are any of you using it with >512 megs of ram? I still cannot get it > to function correctly with freebsd (still freezes, hangs etc). > Works fine with windblows. Yes I've checked the simms in different combinations > and even replaced all the memory. Any combination larger than 512M > will not work with freebsd. Unfortunately I don't have any plans to go over 256MB of memory, because it's my personal workstation. Probably it's an overkill already. Also I can't try because I don't have so much memory at hand, sorry :-| I think that Tyan hardware is okay, but there is some other mystery going on. YMMV 8-) Vallo Kallaste vallo@matti.ee To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Dec 6 08:25:58 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA11327 for freebsd-smp-outgoing; Sun, 6 Dec 1998 08:25:58 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from unox.student.tue.nl (unox.student.tue.nl [131.155.210.3]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA11321 for ; Sun, 6 Dec 1998 08:25:55 -0800 (PST) (envelope-from marcelk@stack.nl) Received: (from marcelk@localhost) by unox.student.tue.nl (8.9.1/8.8.8) id RAA11351 for smp@FreeBSD.ORG; Sun, 6 Dec 1998 17:25:49 +0100 Message-ID: <19981206172549.A11335@unox.student.tue.nl> Date: Sun, 6 Dec 1998 17:25:49 +0100 From: Marcel van Kervinck To: smp@FreeBSD.ORG Subject: Pthreads and SMP Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hello, Perhaps I've missed something obvious But: it seems to me that a process that uses multiple threads doesn't spread them over more processors? I tried on a dual PII, running fbsd 3.0, and a program that simply did a few pthread_create() calls. Kind regards, Marcel -- _ _ _| |_|_| |_ |_ marcelk@stack.nl |_| Marcel van Kervinck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Dec 6 08:41:51 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA13097 for freebsd-smp-outgoing; Sun, 6 Dec 1998 08:41:51 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from gjp.erols.com (alex-va-n008c079.moon.jic.com [206.156.18.89]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA13083 for ; Sun, 6 Dec 1998 08:41:48 -0800 (PST) (envelope-from gjp@gjp.erols.com) Received: from gjp.erols.com (localhost.erols.com [127.0.0.1]) by gjp.erols.com (8.9.1/8.8.7) with ESMTP id LAA51434; Sun, 6 Dec 1998 11:41:36 -0500 (EST) (envelope-from gjp@gjp.erols.com) X-Mailer: exmh version 2.0.1 12/23/97 To: Marcel van Kervinck cc: smp@FreeBSD.ORG From: "Gary Palmer" Subject: Re: Pthreads and SMP In-reply-to: Your message of "Sun, 06 Dec 1998 17:25:49 +0100." <19981206172549.A11335@unox.student.tue.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 06 Dec 1998 11:41:36 -0500 Message-ID: <51430.912962496@gjp.erols.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Marcel van Kervinck wrote in message ID <19981206172549.A11335@unox.student.tue.nl>: > Hello, > > Perhaps I've missed something obvious But: it seems to me that a process > that uses multiple threads doesn't spread them over more processors? > I tried on a dual PII, running fbsd 3.0, and a program that simply did > a few pthread_create() calls. Correct. No-one with enough skill to do the work necessary to get thread migration supported has had the time... Gary -- Gary Palmer FreeBSD Core Team Member FreeBSD: Turning PC's into workstations. See http://www.FreeBSD.ORG/ for info To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Dec 6 09:22:40 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id JAA16807 for freebsd-smp-outgoing; Sun, 6 Dec 1998 09:22:40 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from percy.xyz.com (c394172-a.potlnd1.or.home.com [24.0.48.175]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id JAA16800; Sun, 6 Dec 1998 09:22:35 -0800 (PST) (envelope-from nerd@percy.xyz.com) Received: from percy.xyz.com (localhost.xyz.com [127.0.0.1]) by percy.xyz.com (8.9.1/8.9.1) with ESMTP id JAA51729; Sun, 6 Dec 1998 09:19:45 -0800 (PST) (envelope-from nerd@percy.xyz.com) Message-Id: <199812061719.JAA51729@percy.xyz.com> To: "Gary Palmer" cc: Marcel van Kervinck , smp@FreeBSD.ORG Subject: Re: Pthreads and SMP In-reply-to: Your message of "Sun, 06 Dec 1998 11:41:36 EST." <51430.912962496@gjp.erols.com> Date: Sun, 06 Dec 1998 09:19:45 -0800 From: Michael Galassi Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Marcel van Kervinck wrote: >> Perhaps I've missed something obvious But: it seems to me that a process >> that uses multiple threads doesn't spread them over more processors? >> I tried on a dual PII, running fbsd 3.0, and a program that simply did >> a few pthread_create() calls. Gary Palmer wrote: >Correct. No-one with enough skill to do the work necessary to get thread >migration supported has had the time... Unless I'm wearing serious blinders I fail to see this as a real problem. Unless you have only a tiny number of procs on the system it would seem like keeping related threads together would be a good thing. If you had multiple closely related threads on seperate procs and they decided to acctually access their shared data I'm guessing you would see a lot of cache thrashing. Best keep the threads together if you can move whole procs elsewhere to take up the slack. All this is, of course, IMHO. -michael To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Dec 6 09:29:28 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id JAA17379 for freebsd-smp-outgoing; Sun, 6 Dec 1998 09:29:28 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from gjp.erols.com (alex-va-n008c079.moon.jic.com [206.156.18.89]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id JAA17373 for ; Sun, 6 Dec 1998 09:29:26 -0800 (PST) (envelope-from gjp@gjp.erols.com) Received: from gjp.erols.com (localhost.erols.com [127.0.0.1]) by gjp.erols.com (8.9.1/8.8.7) with ESMTP id MAA52129; Sun, 6 Dec 1998 12:29:01 -0500 (EST) (envelope-from gjp@gjp.erols.com) X-Mailer: exmh version 2.0.1 12/23/97 To: Michael Galassi cc: Marcel van Kervinck , smp@FreeBSD.ORG From: "Gary Palmer" Subject: Re: Pthreads and SMP In-reply-to: Your message of "Sun, 06 Dec 1998 09:19:45 PST." <199812061719.JAA51729@percy.xyz.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 06 Dec 1998 12:29:01 -0500 Message-ID: <52125.912965341@gjp.erols.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Michael Galassi wrote in message ID <199812061719.JAA51729@percy.xyz.com>: > Unless I'm wearing serious blinders I fail to see this as a real > problem. Unless you have only a tiny number of procs on the system it > would seem like keeping related threads together would be a good > thing. If you had multiple closely related threads on seperate procs > and they decided to acctually access their shared data I'm guessing > you would see a lot of cache thrashing. Then your threaded program is badly designed :) Seriously, there are advantages to thread migration and disadvantages. I believe Solaris only does thread migration when the CPU that the process is on is overcomitted. So unless you were totally CPU bound (in which case the process shouldn't have been threaded anyhow, as threads get their advantage from being I/O or network bound), there shouldn't really be a problem... Gary -- Gary Palmer FreeBSD Core Team Member FreeBSD: Turning PC's into workstations. See http://www.FreeBSD.ORG/ for info To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Dec 6 10:09:29 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id KAA20894 for freebsd-smp-outgoing; Sun, 6 Dec 1998 10:09:29 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from ns1.adsu.bellsouth.com (ns1.adsu.bellsouth.com [205.152.173.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id KAA20888; Sun, 6 Dec 1998 10:09:21 -0800 (PST) (envelope-from ck@ns1.adsu.bellsouth.com) Received: (from ck@localhost) by ns1.adsu.bellsouth.com (8.9.1a/8.9.1) id NAA03200; Sun, 6 Dec 1998 13:09:18 -0500 (EST) Message-ID: <19981206130918.I24178@ns1.adsu.bellsouth.com> Date: Sun, 6 Dec 1998 13:09:18 -0500 From: Christian Kuhtz To: Gary Palmer , Michael Galassi Cc: Marcel van Kervinck , smp@FreeBSD.ORG Subject: Re: Pthreads and SMP References: <199812061719.JAA51729@percy.xyz.com> <52125.912965341@gjp.erols.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: <52125.912965341@gjp.erols.com>; from Gary Palmer on Sun, Dec 06, 1998 at 12:29:01PM -0500 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sun, Dec 06, 1998 at 12:29:01PM -0500, Gary Palmer wrote: > Michael Galassi wrote in message ID > <199812061719.JAA51729@percy.xyz.com>: > > Unless I'm wearing serious blinders I fail to see this as a real > > problem. Unless you have only a tiny number of procs on the system it > > would seem like keeping related threads together would be a good > > thing. If you had multiple closely related threads on seperate procs > > and they decided to acctually access their shared data I'm guessing > > you would see a lot of cache thrashing. > > Then your threaded program is badly designed :) I agree. > Seriously, there are advantages to thread migration and disadvantages. I > believe Solaris only does thread migration when the CPU that the process is on > is overcomitted. So unless you were totally CPU bound (in which case the > process shouldn't have been threaded anyhow, as threads get their advantage > from being I/O or network bound), there shouldn't really be a problem... If you have n CPUs, and n threads in a given process, each CPU bound, it may still make for better application design to have the process be threaded than to have descrete processes running. Suggested reading includes the various papers and books on Mach ukernels. It also depends tremendously on the actual multiprocessor architecture (shared memory, distributed memory etc etc) as much as the application design to make a final judgement call. Regardless, there should be a way to bind any given process or thread to a particular CPU manually, like the Slowlaris pbind command utility. Cheers, Chris To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Dec 6 20:43:20 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id UAA14511 for freebsd-smp-outgoing; Sun, 6 Dec 1998 20:43:20 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from spinner.netplex.com.au (spinner.netplex.com.au [202.12.86.3]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id UAA14497; Sun, 6 Dec 1998 20:43:10 -0800 (PST) (envelope-from peter@netplex.com.au) Received: from spinner.netplex.com.au (localhost [127.0.0.1]) by spinner.netplex.com.au (8.9.1/8.9.1/Netplex) with ESMTP id MAA17869; Mon, 7 Dec 1998 12:43:04 +0800 (WST) (envelope-from peter@spinner.netplex.com.au) Message-Id: <199812070443.MAA17869@spinner.netplex.com.au> X-Mailer: exmh version 2.0.2 2/24/98 To: "Gary Palmer" cc: Marcel van Kervinck , smp@FreeBSD.ORG Subject: Re: Pthreads and SMP In-reply-to: Your message of "Sun, 06 Dec 1998 11:41:36 EST." <51430.912962496@gjp.erols.com> Date: Mon, 07 Dec 1998 12:43:02 +0800 From: Peter Wemm Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org "Gary Palmer" wrote: > Marcel van Kervinck wrote in message ID > <19981206172549.A11335@unox.student.tue.nl>: > > Hello, > > > > Perhaps I've missed something obvious But: it seems to me that a process > > that uses multiple threads doesn't spread them over more processors? > > I tried on a dual PII, running fbsd 3.0, and a program that simply did > > a few pthread_create() calls. > > Correct. No-one with enough skill to do the work necessary to get thread > migration supported has had the time... It's not so much a question of skill, it's just a moderately big job with a fairly large kernel vm/pmap layer impact and nobody (so far) has had the time to do it. And then there's the issue of connecting the kernel support up to a thread library to implement the posix interfaces to it. > Gary Cheers, -Peter -- Peter Wemm Netplex Consulting To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Dec 7 12:12:59 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id MAA10342 for freebsd-smp-outgoing; Mon, 7 Dec 1998 12:12:59 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from post.mail.demon.net (post-12.mail.demon.net [194.217.242.41]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id MAA10312; Mon, 7 Dec 1998 12:12:47 -0800 (PST) (envelope-from james@westongold.com) Received: from [158.152.96.124] (helo=wgp01.wgold.demon.co.uk) by post.mail.demon.net with esmtp (Exim 2.054 #1) id 0zn71L-0001Q7-00; Mon, 7 Dec 1998 20:12:40 +0000 Received: by WGP01 with Internet Mail Service (5.5.1960.3) id ; Mon, 7 Dec 1998 19:59:15 -0000 Message-ID: <32BABEF63EAED111B2C5204C4F4F50201835@WGP01> From: James Mansion To: Gary Palmer , Michael Galassi Cc: Marcel van Kervinck , smp@FreeBSD.ORG Subject: RE: Pthreads and SMP Date: Mon, 7 Dec 1998 19:59:07 -0000 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.1960.3) Content-Type: text/plain Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I think what you wrote is not representative in many cases. Its not representative of ANY of my threaded development. I normally expect (and desire) threads to be spread over the system's CPUs. If I want to do a lot of IO, I'd rather set it up as aio_ or lio_ calls. Running a thread per client is very dangerous unless you KNOW that you will only run on systems that have an N:1 (ugh!) or M:N structure. Its not guaranteed by POSIX at all. While its clear that working on a hot structure from multiple threads can give your caches headaches, its by far the best way to handle concurrent queries against general data structures. Reducing contention is clearly a design issue of threaded development. It does seem that FreeBSD is some way from being leading edge as a platform for threaded or AIO applications - hopefully this will only be a temporary thing. James > -----Original Message----- > From: Gary Palmer [mailto:gpalmer@FreeBSD.ORG] ... > Seriously, there are advantages to thread migration and > disadvantages. I > believe Solaris only does thread migration when the CPU that > the process is on > is overcomitted. So unless you were totally CPU bound (in > which case the > process shouldn't have been threaded anyhow, as threads get > their advantage > from being I/O or network bound), there shouldn't really be a > problem... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Dec 7 14:17:31 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA23709 for freebsd-smp-outgoing; Mon, 7 Dec 1998 14:17:31 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA23681; Mon, 7 Dec 1998 14:17:17 -0800 (PST) (envelope-from tlambert@usr06.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id PAA27128; Mon, 7 Dec 1998 15:17:01 -0700 (MST) Received: from usr06.primenet.com(206.165.6.206) via SMTP by smtp01.primenet.com, id smtpd026979; Mon Dec 7 15:16:54 1998 Received: (from tlambert@localhost) by usr06.primenet.com (8.8.5/8.8.5) id PAA16574; Mon, 7 Dec 1998 15:16:22 -0700 (MST) From: Terry Lambert Message-Id: <199812072216.PAA16574@usr06.primenet.com> Subject: Re: Pthreads and SMP To: peter@netplex.com.au (Peter Wemm) Date: Mon, 7 Dec 1998 22:16:22 +0000 (GMT) Cc: gpalmer@FreeBSD.ORG, marcelk@stack.nl, smp@FreeBSD.ORG In-Reply-To: <199812070443.MAA17869@spinner.netplex.com.au> from "Peter Wemm" at Dec 7, 98 12:43:02 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > Perhaps I've missed something obvious But: it seems to me that a process > > > that uses multiple threads doesn't spread them over more processors? > > > I tried on a dual PII, running fbsd 3.0, and a program that simply did > > > a few pthread_create() calls. > > > > Correct. No-one with enough skill to do the work necessary to get thread > > migration supported has had the time... > > It's not so much a question of skill, it's just a moderately big job with a > fairly large kernel vm/pmap layer impact and nobody (so far) has had the > time to do it. And then there's the issue of connecting the kernel > support up to a thread library to implement the posix interfaces to it. One reason I haven't piped up on this (I *do* have the scheduler changes for CPU affinity working, actually) is that apart from that, there's still the issue of the per thread stack. I am not very satisfied nor happy with the allocation of stack in the same VM space in each thread (for reasons of stack autogrowth), and I would want to address that before anything else. Stack issues pushed aside (i.e., ignored), I think there is a serious bug in the current implementation of the sfork call, which one can see as a vfork bug when trying to send a reply with a message file using the vacation program (the file is prematurely closed instead of using the pipefd, and you end up getting empty message bodies as a result). Avoiding vfork is possible, if one is willing to lose a (to my mind) critical part of the error reporting; but the point is that there is cruft in there. Finally, apart from affinity, kernel threads are relatively useless as an implementation without cooperative user space scheduling. The only OS which I am aware of having addressed these isses at this point is Solaris. So I agree that it's a big job, but there *is* a certain amount of skill required, as well, especially if you want to meet the spirit of the POSIX standard, and not just the letter. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Dec 7 14:20:44 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA24287 for freebsd-smp-outgoing; Mon, 7 Dec 1998 14:20:44 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA24275 for ; Mon, 7 Dec 1998 14:20:41 -0800 (PST) (envelope-from tlambert@usr06.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.8.8/8.8.8) id PAA22338; Mon, 7 Dec 1998 15:20:33 -0700 (MST) Received: from usr06.primenet.com(206.165.6.206) via SMTP by smtp04.primenet.com, id smtpd022094; Mon Dec 7 15:20:21 1998 Received: (from tlambert@localhost) by usr06.primenet.com (8.8.5/8.8.5) id PAA16916; Mon, 7 Dec 1998 15:19:58 -0700 (MST) From: Terry Lambert Message-Id: <199812072219.PAA16916@usr06.primenet.com> Subject: Re: Pthreads and SMP To: marcelk@stack.nl (Marcel van Kervinck) Date: Mon, 7 Dec 1998 22:19:57 +0000 (GMT) Cc: smp@FreeBSD.ORG In-Reply-To: <19981206172549.A11335@unox.student.tue.nl> from "Marcel van Kervinck" at Dec 6, 98 05:25:49 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Perhaps I've missed something obvious But: it seems to me that a process > that uses multiple threads doesn't spread them over more processors? That is correct. The threads are user space threads, which execute in a single process context. > I tried on a dual PII, running fbsd 3.0, and a program that simply did > a few pthread_create() calls. There are L1 and L2 cache coherency issues that impact bus based arbitration and invalidation bottlenecks, there are scheduler issues (both user and kernel space), there are stack issues, there are address space issues, memory model issues, and a slew more issues. These problems are not as trivially solvable, as the poor Linux and SVR4 kernel threading implementations, or the EGCS thread support assumptions would have you believe. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Dec 7 14:43:18 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA26574 for freebsd-smp-outgoing; Mon, 7 Dec 1998 14:43:18 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from poboxer.pobox.com (port95.prairietech.net [208.141.230.95] (may be forged)) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA26561; Mon, 7 Dec 1998 14:43:03 -0800 (PST) (envelope-from alk@poboxer.pobox.com) Received: (from alk@localhost) by poboxer.pobox.com (8.9.1/8.9.1) id QAA60351; Mon, 7 Dec 1998 16:41:05 -0600 (CST) (envelope-from alk) From: Tony Kimball MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Mon, 7 Dec 1998 16:41:04 -0600 (CST) X-Face: \h9Jg:Cuivl4S*UP-)gO.6O=T]]@ncM*tn4zG);)lk#4|lqEx=*talx?.Gk,dMQU2)ptPC17cpBzm(l'M|H8BUF1&]dDCxZ.c~Wy6-j,^V1E(NtX$FpkkdnJixsJHE95JlhO 5\M3jh'YiO7KPCn0~W`Ro44_TB@&JuuqRqgPL'0/{):7rU-%.*@/>q?1&Ed Reply-To: alk@pobox.com To: tlambert@primenet.com Cc: peter@netplex.com.au, gpalmer@FreeBSD.ORG, marcelk@stack.nl, smp@FreeBSD.ORG Subject: Re: Pthreads and SMP References: <199812070443.MAA17869@spinner.netplex.com.au> <199812072216.PAA16574@usr06.primenet.com> X-Mailer: VM 6.43 under 20.4 "Emerald" XEmacs Lucid Message-ID: <13932.22397.480607.310372@avalon.east> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Quoth Terry Lambert on Mon, 7 December: : One reason I haven't piped up on this (I *do* have the scheduler : changes for CPU affinity working, actually) is that apart from : that, there's still the issue of the per thread stack. : : I am not very satisfied nor happy with the allocation of stack in : the same VM space in each thread (for reasons of stack autogrowth), : and I would want to address that before anything else. Yikes! Surely you are not proposing to place the stacks of distinct threads in distinct address spaces?! This would impact application portability negatively. : Finally, apart from affinity, kernel threads are relatively useless : as an implementation without cooperative user space scheduling. : The only OS which I am aware of having addressed these isses at : this point is Solaris. Give me 1-1 threads, thank you very much, and I will be most happy to manage my own task pools, as this is the only way to write portably in the current state of play. 'Relatively useless' is an overstatement of the case. Relatively useless, perhaps, for the case of Solaris applications relying upon the peculiarities of Solaris thread semantics. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Dec 8 00:22:38 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id AAA28120 for freebsd-smp-outgoing; Tue, 8 Dec 1998 00:22:38 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from sv01.cet.co.jp (sv01.cet.co.jp [210.171.56.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id AAA28090; Tue, 8 Dec 1998 00:22:32 -0800 (PST) (envelope-from michaelh@cet.co.jp) Received: from localhost (michaelh@localhost) by sv01.cet.co.jp (8.8.8/8.8.8) with SMTP id IAA00509; Tue, 8 Dec 1998 08:22:01 GMT (envelope-from michaelh@cet.co.jp) Date: Tue, 8 Dec 1998 17:22:01 +0900 (JST) From: Michael Hancock To: Tony Kimball cc: tlambert@primenet.com, peter@netplex.com.au, gpalmer@FreeBSD.ORG, marcelk@stack.nl, smp@FreeBSD.ORG Subject: Re: Pthreads and SMP In-Reply-To: <13932.22397.480607.310372@avalon.east> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 7 Dec 1998, Tony Kimball wrote: > Quoth Terry Lambert on Mon, 7 December: > : One reason I haven't piped up on this (I *do* have the scheduler > : changes for CPU affinity working, actually) is that apart from > : that, there's still the issue of the per thread stack. > : > : I am not very satisfied nor happy with the allocation of stack in > : the same VM space in each thread (for reasons of stack autogrowth), > : and I would want to address that before anything else. > > Yikes! Surely you are not proposing to place the stacks of > distinct threads in distinct address spaces?! This would > impact application portability negatively. I think he's talking about stacks for kernel execution contexts (kernel threads). > : Finally, apart from affinity, kernel threads are relatively useless > : as an implementation without cooperative user space scheduling. > : The only OS which I am aware of having addressed these isses at > : this point is Solaris. > > Give me 1-1 threads, thank you very much, and I will be most > happy to manage my own task pools, as this is the only way to write > portably in the current state of play. 'Relatively useless' is an > overstatement of the case. Relatively useless, perhaps, for the case > of Solaris applications relying upon the peculiarities of Solaris > thread semantics. Do you mean peculiarities that require you to do things like...? #ifdef SOLARIS25 thr_setconcurrency(2); #endif Digital UNIX is m-n and doesn't have these peculiarities and I thought they were resolved in Solaris 2.6. Regards, Mike Hancock To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Dec 8 02:07:10 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id CAA07548 for freebsd-smp-outgoing; Tue, 8 Dec 1998 02:07:10 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from public.hk.hi.cn (public.hk.hi.cn [202.100.192.111]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id CAA07533 for ; Tue, 8 Dec 1998 02:07:00 -0800 (PST) (envelope-from hahalee@163.net) Received: from ns ([202.100.219.65]) by public.hk.hi.cn (8.8.8+Sun/8.8.8) with SMTP id SAA16782 for ; Tue, 8 Dec 1998 18:06:02 +0800 (CST) Message-ID: <05af01be2292$5c720b20$f910830a@ns.hihkptt.net.cn> From: "hahalee(李凌)" To: "freebsd-smp" Subject: 3.0-REL/Compaq PL3000/NICs Help Date: Tue, 8 Dec 1998 18:06:01 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MIMEOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi,dear gurus, I'm trying to install 3.0-REL on a Compaq PL3000 server: PII-300x2/256M. The symbios SCSI controllers worked just flawlessly.Now my trouble is: NICs don't work! The original nic is a TI ThunderLan: Compaq Netelligent 10/100 PCI UTP. but 3.0-REL cannot identify it. I soon find out that It's compaq's fault: Wrong/old dated PCI-ids. (They sold us a server this year with a 1996 NIC!). I gave up this card. Now I'm trying an Accton 1207(Digital 21143), It works on my UP machine under ervery version of FreeBSD. But this time something strange occured: the install diskette cannot identify it too! though it's an UP kernel! So what's the right NIC should I use on this compaq beast? And is there some SMP related issues on NICs? Or it's just because of compaq's weird SMP? great TIA! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Dec 8 02:15:09 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id CAA08209 for freebsd-smp-outgoing; Tue, 8 Dec 1998 02:15:09 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from hq.cninfo.net ([202.100.219.68]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id CAA08200 for ; Tue, 8 Dec 1998 02:15:06 -0800 (PST) (envelope-from sendmail@163.net) Received: from ns ([10.131.16.249]) by hq.cninfo.net yeshu.com (951211.SGI.8.6.12.PATCH1502/950213.SGI.AUTOCF) via SMTP id SAA01367 for ; Tue, 8 Dec 1998 18:14:39 -0800 Message-ID: <05c401be2293$80a22740$f910830a@ns.hihkptt.net.cn> From: "hahalee(李凌)" To: "freebsd-smp" Subject: 3.0-REL/Compaq PL3000/NICs Help Date: Tue, 8 Dec 1998 18:13:58 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MIMEOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi,dear gurus, I'm trying to install 3.0-REL on a Compaq PL3000 server: PII-300x2/256M. The symbios SCSI controllers worked just flawlessly.Now my trouble is: NICs don't work! The original nic is a TI ThunderLan: Compaq Netelligent 10/100 PCI UTP. but 3.0-REL cannot identify it. I soon find out that It's compaq's fault: Wrong/old dated PCI-ids. (They sold us a server this year with a 1996 NIC!). I gave up this card. Now I'm trying an Accton 1207(Digital 21143), It works on my UP machine under ervery version of FreeBSD. But this time something strange occured: the install diskette cannot identify it too! though it's an UP kernel! So what's the right NIC should I use on this compaq beast? And is there some SMP related issues on NICs? Or it's just because of compaq's weird SMP? great TIA! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Dec 8 05:37:02 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id FAA25477 for freebsd-smp-outgoing; Tue, 8 Dec 1998 05:37:02 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from verdi.nethelp.no (verdi.nethelp.no [158.36.41.162]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id FAA25472 for ; Tue, 8 Dec 1998 05:36:57 -0800 (PST) (envelope-from sthaug@nethelp.no) From: sthaug@nethelp.no Received: (qmail 6151 invoked by uid 1001); 8 Dec 1998 13:36:44 +0000 (GMT) To: hahalee@163.net Cc: freebsd-smp@FreeBSD.ORG Subject: Re: 3.0-REL/Compaq PL3000/NICs Help In-Reply-To: Your message of "Tue, 8 Dec 1998 18:06:01 +0800" References: <05af01be2292$5c720b20$f910830a@ns.hihkptt.net.cn> X-Mailer: Mew version 1.05+ on Emacs 19.34.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Tue, 08 Dec 1998 14:36:44 +0100 Message-ID: <6149.913124204@verdi.nethelp.no> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I'm trying to install 3.0-REL on a Compaq PL3000 server: > PII-300x2/256M. The symbios SCSI controllers worked just > flawlessly.Now my trouble is: > > NICs don't work! > > The original nic is a TI ThunderLan: Compaq Netelligent 10/100 > PCI UTP. but 3.0-REL cannot identify it. I soon find out that > It's compaq's fault: Wrong/old dated PCI-ids. (They sold us a > server this year with a 1996 NIC!). I gave up this card. > > Now I'm trying an Accton 1207(Digital 21143), It works on my > UP machine under ervery version of FreeBSD. But this time > something strange occured: the install diskette cannot identify > it too! though it's an UP kernel! > > So what's the right NIC should I use on this compaq beast? > And is there some SMP related issues on NICs? > Or it's just because of compaq's weird SMP? It may be Compaq. I had basically the same trouble trying to install 3.0 on a Proliant 3000. No NICs were seen. I ended up installing 2.2.7, using an Intel Pro 100/B card, and *then* configuring a kernel with the tl (Thunderlan) driver. The machine is rock stable with 2.2.7, but is using only one processor. Performance is still very good! Below is a suggestion on what you can do with the Proliant to make it run FreeBSD smoothly. I haven't tried this myself yet - will probably get a chance to do this next week. Steinar Haug, Nethelp consulting, sthaug@nethelp.no ---------------------------------------------------------------------- Date: Sun, 29 Nov 1998 15:36:44 +0200 From: Eugene Vedistchev Reply-To: scaner@belabm.by Organization: Global One in Belarus To: Gus Bourg , freebsd-smp@FreeBSD.ORG Subject: Re: 25 pin apic Hello. The problem with Compaq SMP servers is: Compaq configuration utility have several OS presets (NW,SCO,NT,OS/2,Vines), which ajusts some chipset, CPU and SMP mode settings. Although no one of thise presets is suitable for FreeBSD and Linux. To switch Compaq box to suitable mode you need to run configuration utility from SmartStart CD or from system partition, than, when you'll see textmode(not graphics fonts) menu with the following items - configure system, configure Smart Array... press Control-A sequence. The utility say that you'll enter in Advanced (not documented) mode. Then go to system configuration, set MPTABLE option into FULL mode, save and go on. FreeBSD will run perfectly on your Compaq server. WBR, Eugene PS. Sorry for bad english :) Gus Bourg wrote: > > I read an earlier message regarding 25 interrupt pins. It seems FreeBSD at > the moment only supports 24, is that still true? I have a couple of Compaq > Proliant 3000s here that I want to get up and running with FreeBSD/SMP, > but I get the same error, panic: assign_apic_irq: inconsistent table. Are > there any known workarounds for this? Are there plans to support more than > 24pins? > > Thanks! > Gus Bourg > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Dec 8 05:38:33 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id FAA25807 for freebsd-smp-outgoing; Tue, 8 Dec 1998 05:38:33 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from hq.cninfo.net ([202.100.219.68]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id FAA25799 for ; Tue, 8 Dec 1998 05:38:28 -0800 (PST) (envelope-from sendmail@163.net) Received: from ns ([10.131.16.249]) by hq.cninfo.net yeshu.com (951211.SGI.8.6.12.PATCH1502/950213.SGI.AUTOCF) via SMTP id VAA05758; Tue, 8 Dec 1998 21:37:36 -0800 Message-ID: <066901be22af$e00ca9a0$f910830a@ns.hihkptt.net.cn> From: "hahalee(李凌)" To: "Matthew Patton" Cc: "freebsd-smp" Subject: Re: 3.0-REL/Compaq PL3000/NICs Help Date: Tue, 8 Dec 1998 21:36:32 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MIMEOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Yep! I did set the "OS" to "UNIX", not work. damn compaq mobos! but according to Linux docs,maybe I should set it to "Unixware", they linux guys donknow why, but they say: "it works,don't ask why!". Thanks, I'd keep trying... If it works, I'll tell you. "cause PL3000 is such a terrible thing to waste" :-) -----Original Message----- Matthew Patton hahalee Re: 3.0-REL/Compaq PL3000/NICs Help >wow, what smartstart 'OS' choice did you make? I haven't tried FreeBSD yet >on my 5 servers but I wanted to. I couldn't get the scsi chain in >particular to probe. > >-------- >OpenBSD - Because security matters... (http://www.openbsd.org/) > >"There is one terrifying word in the world of nuclear physics, Oops." > - Tom Servo, Mystery Science Theator 3000 > > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Dec 8 13:33:42 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id NAA12972 for freebsd-smp-outgoing; Tue, 8 Dec 1998 13:33:42 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id NAA12944; Tue, 8 Dec 1998 13:33:37 -0800 (PST) (envelope-from tlambert@usr09.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.8.8/8.8.8) id OAA02473; Tue, 8 Dec 1998 14:33:13 -0700 (MST) Received: from usr09.primenet.com(206.165.6.209) via SMTP by smtp02.primenet.com, id smtpd002297; Tue Dec 8 14:33:10 1998 Received: (from tlambert@localhost) by usr09.primenet.com (8.8.5/8.8.5) id OAA25044; Tue, 8 Dec 1998 14:32:43 -0700 (MST) From: Terry Lambert Message-Id: <199812082132.OAA25044@usr09.primenet.com> Subject: Re: Pthreads and SMP To: alk@pobox.com Date: Tue, 8 Dec 1998 21:32:43 +0000 (GMT) Cc: tlambert@primenet.com, peter@netplex.com.au, gpalmer@FreeBSD.ORG, marcelk@stack.nl, smp@FreeBSD.ORG In-Reply-To: <13932.22397.480607.310372@avalon.east> from "Tony Kimball" at Dec 7, 98 04:41:04 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > : I am not very satisfied nor happy with the allocation of stack in > : the same VM space in each thread (for reasons of stack autogrowth), > : and I would want to address that before anything else. > > Yikes! Surely you are not proposing to place the stacks of > distinct threads in distinct address spaces?! This would > impact application portability negatively. No. That was in fact the major Mistake Microsoft made that caused the need for CreateFreeThreadedMarshallar() (sp?) in order to transit objects between thread address spaces. > : Finally, apart from affinity, kernel threads are relatively useless > : as an implementation without cooperative user space scheduling. > : The only OS which I am aware of having addressed these isses at > : this point is Solaris. > > Give me 1-1 threads, thank you very much, and I will be most > happy to manage my own task pools, as this is the only way to write > portably in the current state of play. 'Relatively useless' is an > overstatement of the case. Relatively useless, perhaps, for the case > of Solaris applications relying upon the peculiarities of Solaris > thread semantics. OK; not useless for interoperability, only useless for actually getting more work done than you would get done without kernel threads; is that better? The problem with kernel threads is that, barring everything else, the threads will migrate randomly between processors, cache-busting as they go, and you won't get significant benefit from having multiple processors at all. Only multiple processes really gain benefit from SMP given the current scheduler. If you really want kernel threads, you should look at the -current list archives, and take the John Dyson assembly glue and use it, since you could have kernel threads out your nose for less than 10 minutes worth of work. Unfortunately, that's not a very interesting thing to do until it would actually buy you benefit of some kind (which explains why it hasn't been packaged up and integrated yet). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Dec 8 13:52:44 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id NAA15811 for freebsd-smp-outgoing; Tue, 8 Dec 1998 13:52:44 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id NAA15805; Tue, 8 Dec 1998 13:52:37 -0800 (PST) (envelope-from tlambert@usr09.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id OAA07587; Tue, 8 Dec 1998 14:52:16 -0700 (MST) Received: from usr09.primenet.com(206.165.6.209) via SMTP by smtp01.primenet.com, id smtpd007090; Tue Dec 8 14:51:42 1998 Received: (from tlambert@localhost) by usr09.primenet.com (8.8.5/8.8.5) id OAA25595; Tue, 8 Dec 1998 14:51:01 -0700 (MST) From: Terry Lambert Message-Id: <199812082151.OAA25595@usr09.primenet.com> Subject: Re: Pthreads and SMP To: michaelh@cet.co.jp (Michael Hancock) Date: Tue, 8 Dec 1998 21:50:55 +0000 (GMT) Cc: alk@pobox.com, tlambert@primenet.com, peter@netplex.com.au, gpalmer@FreeBSD.ORG, marcelk@stack.nl, smp@FreeBSD.ORG In-Reply-To: from "Michael Hancock" at Dec 8, 98 05:22:01 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > : One reason I haven't piped up on this (I *do* have the scheduler > > : changes for CPU affinity working, actually) is that apart from > > : that, there's still the issue of the per thread stack. > > : > > : I am not very satisfied nor happy with the allocation of stack in > > : the same VM space in each thread (for reasons of stack autogrowth), > > : and I would want to address that before anything else. > > > > Yikes! Surely you are not proposing to place the stacks of > > distinct threads in distinct address spaces?! This would > > impact application portability negatively. > > I think he's talking about stacks for kernel execution contexts (kernel > threads). No; that's a problem, too, but it's less of a problem. If you think in terms of an async call gate, all blocking calls in a reentrant kernel would be made on a kernel stack from a kernel stack pool, with guard pages and the ability to manage the VM space for the call by fault (to grow the kernel stack). In other words, the kernel stack is not associated with the kernel process (scheduling context, actually, since threads in a kernel thread group should have the same "thread group ID" [PID]). In the current implementation using the glue code, each kernel thread is a process that shares a number of identical proc struct pointers with its nominal "parent" (peer thread that created it). > #ifdef SOLARIS25 > thr_setconcurrency(2); > #endif > > Digital UNIX is m-n and doesn't have these peculiarities and I thought > they were resolved in Solaris 2.6. They are; the usage model is such that there is a sperate thread for reception of a signal causing the spawning of an additional kernel thread. This allows them to avoid the m:n "n blocking calls" deadlock by spawning another kernel thread to handle the load. It's what Microsoft calls their "apartment threading" model, where user space threads are allowed to go into the kernel, and kernel threads are instances to back them while they are there, as necessary. Kind of like the cube visiting flatland. 8-). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Dec 8 19:11:29 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id TAA21538 for freebsd-smp-outgoing; Tue, 8 Dec 1998 19:11:29 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from public.hk.hi.cn (public.hk.hi.cn [202.100.192.111]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id TAA21531 for ; Tue, 8 Dec 1998 19:11:18 -0800 (PST) (envelope-from hahalee@163.net) Received: from ns ([202.100.219.15]) by public.hk.hi.cn (8.8.8+Sun/8.8.8) with SMTP id LAA11984; Wed, 9 Dec 1998 11:10:23 +0800 (CST) Message-ID: <092701be2321$7305b780$f910830a@ns.hihkptt.net.cn> From: "hahalee(李凌)" To: "freebsd-smp" Cc: "Matthew Patton" , "Steinar Haug" Subject: Compaq PL3000/NICs, Again. Date: Wed, 9 Dec 1998 11:10:15 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MIMEOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Dear list, Matthew, Steinar, After a sleepless night, I realized what's the problem: The Compaq PL3000 is a PCI-EISA-ISA mixture, and has TWO PCI buses, Mobo buildin components are attached on bus0, Symbios dual SCSI and NICs are attached on bus1, the second one. You can use HWINFO (diag utils for dos) to find this. BTW, in Advanced (the control_a ) mode, I tried every combinations, APIC = {Full w/ mapped, Full, None} FreeBSD still fails to find NIC/SCSI. Linux(debian20) does find the Accton NIC, but can not send/recv packet. Is there any way to solve this? Is there any one runs FreeBSD-3.0 SMP on PL3000? regards TIA To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Dec 9 22:06:15 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA06212 for freebsd-smp-outgoing; Wed, 9 Dec 1998 22:06:15 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from chen.ml.org (luoqi.watermarkgroup.com [207.202.73.170]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA06207 for ; Wed, 9 Dec 1998 22:06:12 -0800 (PST) (envelope-from luoqi@chen.ml.org) Received: (from luoqi@localhost) by chen.ml.org (8.9.1/8.9.1) id AAA07472 for smp@freebsd.org; Thu, 10 Dec 1998 00:46:56 -0500 (EST) (envelope-from luoqi) Date: Thu, 10 Dec 1998 00:46:56 -0500 (EST) From: Luoqi Chen Message-Id: <199812100546.AAA07472@chen.ml.org> To: smp@FreeBSD.ORG Subject: address space sharing Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In the last week or so, I have been working on vmspace sharing for SMP. It was an experiment helping myself understand better issues involved in kernal thread implementation. Now I have a kernel stable enought to survive a make -j8 world (up to the point gdb build failed because of some kernel data structure changes). If anyone is interested, I've placed a patch file at http://www.freebsd.org/~luoqi/pmap.diff . -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Dec 9 23:13:55 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id XAA12302 for freebsd-smp-outgoing; Wed, 9 Dec 1998 23:13:55 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from fast.cs.utah.edu (fast.cs.utah.edu [155.99.212.1]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id XAA12297 for ; Wed, 9 Dec 1998 23:13:54 -0800 (PST) (envelope-from vanmaren@fast.cs.utah.edu) Received: (from vanmaren@localhost) by fast.cs.utah.edu (8.9.1/8.9.1) id AAA27687; Thu, 10 Dec 1998 00:13:45 -0700 (MST) Date: Thu, 10 Dec 1998 00:13:45 -0700 (MST) From: Kevin Van Maren Message-Id: <199812100713.AAA27687@fast.cs.utah.edu> To: freebsd-smp@FreeBSD.ORG, hahalee@163.net Subject: Re: Compaq PL3000/NICs, Again. Cc: patton@sysnet.net, sthaug@nethelp.no Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Is there any way to solve this? Sure! Sounds like the second PCI bus isn't getting probed. Can you send the `mptable -verbose -dmesg` output (boot -v) [after applying the patch in kern/8898]? If it doesn't list PCI bus #1, then SMP probably won't work with those devices until you get an updated bios. The PL3000 is an interesting case; when you mentioned two PCI buses, I thought it would have used the Orion (PPro) chipset. But it is a Pentium II. What chipset does it use to support up to 3GB ram? The only ones from Intel that can do that are the Orion (PPro, 66Mhz only) and the 450NX (Xeon only). Most likely what is happening is that the host-pci (ala the Orion or 450NX chipset or the Micron Samurai) or the Pci-Pci bridge (ala most everything else with multiple PCI busses) isn't getting probed properly and only the first PCI bus is getting probed. Anyway, if we can get the programming info, it should be easy enough to fix. You can hack a quick fix by changing pci_bushigh() in src/sys/pci/pci.c to return 2 (or maybe try up to 255). Kevin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Dec 10 00:29:28 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id AAA21072 for freebsd-smp-outgoing; Thu, 10 Dec 1998 00:29:28 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from NS1.HomeMortgageUSA.Com (Shell.Bourg.Net [207.229.69.135]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id AAA21065 for ; Thu, 10 Dec 1998 00:29:26 -0800 (PST) (envelope-from gus@Bourg.Net) Received: from localhost (gus@localhost) by NS1.HomeMortgageUSA.Com (8.9.1/8.8.7) with SMTP id AAA04666; Thu, 10 Dec 1998 00:33:13 -0800 (PST) (envelope-from gus@Bourg.Net) Date: Thu, 10 Dec 1998 00:33:13 -0800 (PST) From: Gus Bourg X-Sender: gus@NS1.HomeMortgageUSA.Com To: "hahalee(李凌)" cc: freebsd-smp , Matthew Patton , Steinar Haug Subject: Re: Compaq PL3000/NICs, Again. In-Reply-To: <092701be2321$7305b780$f910830a@ns.hihkptt.net.cn> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from QUOTED-PRINTABLE to 8bit by hub.freebsd.org id AAA21066 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I have several PL3000s running. Heres what I did: I personally could not get into the advanced mode, so as I was about to throw the compaqs off the 18th floor, I noticed SCO Unix was an option so I selected it. This fixed the SMP kernel crashes, however it would only probe the first PCI bus. My machines don't have built in ethernet, they have OEM'd ether express cards. The first pci bus starts at the bottom pci slot (the ones that are shared ISA), and extends to at least the third one. I just moved the network card around. I chose not to use the symbios cards, and opted for DPT raid cards as my stuff is semi-mission critical. I'm not sure if the toying I had to do applies to all cards, but the DPTs were a pain to get to work in the compaq box and their tech support was as bad as dealing with M$. The solution for it was to turn on the EBDA relocation option. If anyone happens to have a good solution for the second pci bus problem I'd like toear what they did. I'm not sure if I mentioned this, but I also had to manually set the MAXMEM option in my kernel config as the bios apparently sends extended information. Also when compiling the kernel, you may need to set NINTR to 52. I hope this helps. :-) Gus Bourg On Wed, 9 Dec 1998, hahalee(李凌) wrote: > Dear list, > Matthew, > Steinar, > > After a sleepless night, I realized what's the problem: > > The Compaq PL3000 is a PCI-EISA-ISA mixture, and > has TWO PCI buses, Mobo buildin components are attached > on bus0, Symbios dual SCSI and NICs are attached on > bus1, the second one. > > You can use HWINFO (diag utils for dos) to find this. > > BTW, in Advanced (the control_a ) mode, I tried every > combinations, APIC = {Full w/ mapped, Full, None} > FreeBSD still fails to find NIC/SCSI. Linux(debian20) does > find the Accton NIC, but can not send/recv packet. > > Is there any way to solve this? > Is there any one runs FreeBSD-3.0 SMP on PL3000? > > regards > TIA > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Dec 10 09:27:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id JAA18116 for freebsd-smp-outgoing; Thu, 10 Dec 1998 09:27:34 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from fleming.cs.strath.ac.uk (fleming.cs.strath.ac.uk [130.159.196.126]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id JAA18082; Thu, 10 Dec 1998 09:27:08 -0800 (PST) (envelope-from roger@cs.strath.ac.uk) Received: from muir-10 (roger@muir-10.cs.strath.ac.uk [130.159.148.10]) by fleming.cs.strath.ac.uk (8.8.8/8.8.8) with SMTP id RAA04143 Thu, 10 Dec 1998 17:26:51 GMT Message-ID: <3670045D.2781@cs.strath.ac.uk> Date: Thu, 10 Dec 1998 17:26:53 +0000 From: Roger Hardiman Organization: University of Strathclyde X-Mailer: Mozilla 3.04Gold (X11; I; OSF1 V4.0 alpha) MIME-Version: 1.0 To: mobile@FreeBSD.ORG, smp@FreeBSD.ORG Subject: SMP and PCMCIA card controllers give kernel panic Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, Does the 3.0-RELEASE PCMCIA controller code work with SMP kernels? It keeps crashing the kernel during the boot sequence on my server. I have a desktop PC with an ISA PCMCIA adapter controller card which has two PCMCIA modem cards plugged in. It was a Dual Pentium II motherboard, but until now, I only had one CPU. I have just added a 2nd Pentium II to my server and built a SMP kernel. But this new SMP kernel crashes during the boot sequence if I include the PCMCIA controller in my kernel config file. Without PCMCIA support, the machine boots fine with SMP support. Does anyone know anything about this? If someone is offering to help, I'll build a debug kernel and hook up a serial line to another FreeBSD machine and find the crash point. Bye Roger To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Dec 10 10:28:15 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id KAA25604 for freebsd-smp-outgoing; Thu, 10 Dec 1998 10:28:15 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from ns.mt.sri.com (sri-gw.MT.net [206.127.105.141]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id KAA25599; Thu, 10 Dec 1998 10:28:12 -0800 (PST) (envelope-from nate@mt.sri.com) Received: from mt.sri.com (rocky.mt.sri.com [206.127.76.100]) by ns.mt.sri.com (8.8.8/8.8.8) with SMTP id LAA10119; Thu, 10 Dec 1998 11:28:05 -0700 (MST) (envelope-from nate@rocky.mt.sri.com) Received: by mt.sri.com (SMI-8.6/SMI-SVR4) id LAA23387; Thu, 10 Dec 1998 11:28:04 -0700 Date: Thu, 10 Dec 1998 11:28:04 -0700 Message-Id: <199812101828.LAA23387@mt.sri.com> From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Roger Hardiman Cc: mobile@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: SMP and PCMCIA card controllers give kernel panic In-Reply-To: <3670045D.2781@cs.strath.ac.uk> References: <3670045D.2781@cs.strath.ac.uk> X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Does the 3.0-RELEASE PCMCIA controller code work with SMP kernels? It was never designed with this in mind. I think very little code has SMP locks around it. > It keeps crashing the kernel during the boot sequence on my server. Finding out where it's crashing would make it easier to determine if we could have a 'quick fix' for it. Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Dec 10 10:49:53 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id KAA28749 for freebsd-smp-outgoing; Thu, 10 Dec 1998 10:49:53 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from cynix.ecn.purdue.edu (cynix.ecn.purdue.edu [128.46.108.200]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id KAA28740; Thu, 10 Dec 1998 10:49:48 -0800 (PST) (envelope-from splite@purdue.edu) Received: (from splite@localhost) by cynix.ecn.purdue.edu (8.8.8/8.8.8) id NAA03655; Thu, 10 Dec 1998 13:48:24 -0500 (EST) Message-ID: <19981210134824.A3259@cynix.ecn.purdue.edu> Date: Thu, 10 Dec 1998 13:48:24 -0500 From: splite To: Roger Hardiman , mobile@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: SMP and PCMCIA card controllers give kernel panic References: <3670045D.2781@cs.strath.ac.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii In-Reply-To: <3670045D.2781@cs.strath.ac.uk>; from Roger Hardiman on Thu, Dec 10, 1998 at 05:26:53PM +0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, Dec 10, 1998 at 05:26:53PM +0000, Roger Hardiman wrote: > Hi, > > Does the 3.0-RELEASE PCMCIA controller code work with SMP kernels? > It keeps crashing the kernel during the boot sequence on my server. > > > I have a desktop PC with an ISA PCMCIA adapter controller card > which has two PCMCIA modem cards plugged in. > It was a Dual Pentium II motherboard, but until now, I only had > one CPU. > > I have just added a 2nd Pentium II to my server and built a SMP > kernel. > But this new SMP kernel crashes during the boot sequence if I > include the PCMCIA controller in my kernel config file. > Without PCMCIA support, the machine boots fine with SMP support. > > Does anyone know anything about this? > If someone is offering to help, I'll build a debug kernel and hook > up a serial line to another FreeBSD machine and find the crash > point. I've seen the same problem. Dual PII/266, Tyan Thunder-2, i365-clone PCMCIA adapter, 3.0-current ELF from about a month ago. I didn't have time to mess with it, so I just removed the adpater. Just so you don't feel alone. :-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Dec 10 13:04:08 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id NAA16028 for freebsd-smp-outgoing; Thu, 10 Dec 1998 13:04:08 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from hromeo.algonet.se (hromeo.algonet.se [194.213.74.10]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id NAA16017 for ; Thu, 10 Dec 1998 13:04:05 -0800 (PST) (envelope-from mal@algonet.se) Received: (qmail 24269 invoked from network); 10 Dec 1998 22:03:45 +0100 Received: from kairos.algonet.se (194.213.74.18) by hromeo.algonet.se with SMTP; 10 Dec 1998 22:03:45 +0100 Received: (mal@localhost) by kairos.algonet.se (8.8.8+Sun/8.6.12) id WAA24473; Thu, 10 Dec 1998 22:03:44 +0100 (MET) Date: Thu, 10 Dec 1998 22:03:44 +0100 (MET) Message-Id: <199812102103.WAA24473@kairos.algonet.se> X-Authentication-Warning: kairos.algonet.se: mal set sender to mal@kairos.algonet.se using -f From: Mats Lofkvist To: tlambert@primenet.com CC: alk@pobox.com, tlambert@primenet.com, peter@netplex.com.au, gpalmer@FreeBSD.ORG, marcelk@stack.nl, smp@FreeBSD.ORG In-reply-to: <199812082132.OAA25044@usr09.primenet.com> (message from Terry Lambert on Tue, 8 Dec 1998 21:32:43 +0000 (GMT)) Subject: Re: Pthreads and SMP References: <199812082132.OAA25044@usr09.primenet.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Terry Lambert wrote: The problem with kernel threads is that, barring everything else, the threads will migrate randomly between processors, cache-busting as they go, and you won't get significant benefit from having multiple processors at all. Only multiple processes really gain benefit from SMP given the current scheduler. (Sorry for the delay, but I have been thinking about this on and off for a few days now and I still don't get it :-) Are you saying that processes migrating randomly between processors, cache busting as they go, is less of a problem for a given parallell application than kernel threads doing the same? Or that the scheduler is able to keep processes mostly on the same processor but would not easily be made able to do the same for kernel threads? _ Mats Lofkvist mal@algonet.se To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Dec 10 17:51:32 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id RAA24049 for freebsd-smp-outgoing; Thu, 10 Dec 1998 17:51:32 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id RAA24035; Thu, 10 Dec 1998 17:51:24 -0800 (PST) (envelope-from tlambert@usr09.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id SAA17023; Thu, 10 Dec 1998 18:51:16 -0700 (MST) Received: from usr09.primenet.com(206.165.6.209) via SMTP by smtp01.primenet.com, id smtpd016818; Thu Dec 10 18:51:09 1998 Received: (from tlambert@localhost) by usr09.primenet.com (8.8.5/8.8.5) id SAA09778; Thu, 10 Dec 1998 18:50:46 -0700 (MST) From: Terry Lambert Message-Id: <199812110150.SAA09778@usr09.primenet.com> Subject: Re: Pthreads and SMP To: mal@algonet.se (Mats Lofkvist) Date: Fri, 11 Dec 1998 01:50:46 +0000 (GMT) Cc: tlambert@primenet.com, alk@pobox.com, peter@netplex.com.au, gpalmer@FreeBSD.ORG, marcelk@stack.nl, smp@FreeBSD.ORG In-Reply-To: <199812102103.WAA24473@kairos.algonet.se> from "Mats Lofkvist" at Dec 10, 98 10:03:44 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > The problem with kernel threads is that, barring everything else, > the threads will migrate randomly between processors, cache-busting > as they go, and you won't get significant benefit from having > multiple processors at all. Only multiple processes really gain > benefit from SMP given the current scheduler. > > (Sorry for the delay, but I have been thinking about this > on and off for a few days now and I still don't get it :-) > > Are you saying that processes migrating randomly between processors, > cache busting as they go, is less of a problem for a given parallell > application than kernel threads doing the same? Or that the scheduler > is able to keep processes mostly on the same processor but would not > easily be made able to do the same for kernel threads? The latter, and not on purpose, but merely as an artifact of how the scheduler works. You can get process migration, but it just doesn't happen very often in practice. The use of a (mostly) shared VM space between kernel threads is a major source of cache line invalidation/update and IPI's between the processors in an SMP system, and this adds to the problem. Seperate processes would not require invalidation,/update since the address space is only in one cache at a time, not two. It's an interesting problem. The limits that everyone quotes for scalability of SMP (4 or 8 processors) are really artifacts of interprocessor contention traffic getting to the point it may exceed the "useful" traffic of actually getting work done. A number of smart people at companies like Sequent resolved these problems for 32 (or more) processors years ago, using per processor resource pools and CPU affinity. It looks like a lot of "modern" threading architecture is about to undo a lot of good work in this area by glomming the contention domain back into one unit (the multithreaded process) once again. 8-(. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Dec 10 19:07:20 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id TAA02401 for freebsd-smp-outgoing; Thu, 10 Dec 1998 19:07:20 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from poboxer.pobox.com (port7.prairietech.net [208.141.230.84]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id TAA02396 for ; Thu, 10 Dec 1998 19:07:18 -0800 (PST) (envelope-from alk@poboxer.pobox.com) Received: (from alk@localhost) by poboxer.pobox.com (8.9.1/8.9.1) id VAA08200; Thu, 10 Dec 1998 21:07:03 -0600 (CST) (envelope-from alk) From: Tony Kimball MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Thu, 10 Dec 1998 21:07:03 -0600 (CST) X-Face: \h9Jg:Cuivl4S*UP-)gO.6O=T]]@ncM*tn4zG);)lk#4|lqEx=*talx?.Gk,dMQU2)ptPC17cpBzm(l'M|H8BUF1&]dDCxZ.c~Wy6-j,^V1E(NtX$FpkkdnJixsJHE95JlhO 5\M3jh'YiO7KPCn0~W`Ro44_TB@&JuuqRqgPL'0/{):7rU-%.*@/>q?1&Ed Reply-To: alk@pobox.com To: tlambert@primenet.com Cc: smp@FreeBSD.ORG Subject: Re: Pthreads and SMP References: <199812102103.WAA24473@kairos.algonet.se> <199812110150.SAA09778@usr09.primenet.com> X-Mailer: VM 6.43 under 20.4 "Emerald" XEmacs Lucid Message-ID: <13936.35389.443798.776169@avalon.east> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Quoth Terry Lambert on Fri, 11 December: : The latter, and not on purpose, but merely as an artifact of how : the scheduler works. I guess it's relative. I can sit here and watch top ping back and forth between 2 CPUs roughly 50% of the time. A pretty crude data source, admittedly, but one that indicates an absence of affinity. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Dec 11 05:54:25 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id FAA08360 for freebsd-smp-outgoing; Fri, 11 Dec 1998 05:54:25 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from fleming.cs.strath.ac.uk (fleming.cs.strath.ac.uk [130.159.196.126]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id FAA08352; Fri, 11 Dec 1998 05:54:11 -0800 (PST) (envelope-from roger@cs.strath.ac.uk) Received: from muir-10 (roger@muir-10.cs.strath.ac.uk [130.159.148.10]) by fleming.cs.strath.ac.uk (8.8.8/8.8.8) with SMTP id NAA00993 Fri, 11 Dec 1998 13:53:27 GMT Message-ID: <367123D9.41C6@cs.strath.ac.uk> Date: Fri, 11 Dec 1998 13:53:30 +0000 From: Roger Hardiman Organization: University of Strathclyde X-Mailer: Mozilla 3.04Gold (X11; I; OSF1 V4.0 alpha) MIME-Version: 1.0 To: splite , Nate Williams CC: mobile@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: SMP and PCMCIA card controllers give kernel panic References: <3670045D.2781@cs.strath.ac.uk> <19981210134824.A3259@cynix.ecn.purdue.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, Well I did some kernel hacking and tracked down the problem to the IRQ probe code. The error is in build_freelist() in /sys/pccard/pcic.c It tries to ask the kernel which IRQs are free to be allocated to A) the PCMCIA controller board and B) the PCMCIA cards. This probe does not work correctly with SMP systems. It seems that on my hardware, the kernel Frees up some IRQs, giving these messages in my boot sequence. Freeing (NOT implemented) redirected ISA irq 11. Freeing (NOT implemented) redirected ISA irq 10. Freeing (NOT implemented) redirected ISA irq 9. Freeing (NOT implemented) redirected ISA irq 11. The IRQ probe scans IRQs from 1 to 15. When it gets to IRQ9 it causes a kernel panic, reporting a clash in the IRQ allocation. (I did not capture the exact error). Now IRQs 9,10 and 11 are what my BIOS assignes to my PCI cards and the SMP kernel does reasign these to IRQs 16 onwards in the magical APIC_IO way. So, I got around the crash by hacking the build_freelist() function to only scan from IRQ 1 to 7 and to ignore the rest. I also added my hack which prevents the controller allocating itself a IRQ too, but commenting some other code. So, it all seems to work. PCCARDD detects the two PCMCIA modems and gives me SIO2 and SIO3. I plugged in my two phones and dialed from one to the other and it all worked just great. So, the proper fix is for the build_freelist to be changed to probe correctly under SMP. Who do we need to hastle in the SMP development team. Bye Roger To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Dec 11 10:43:36 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id KAA10108 for freebsd-smp-outgoing; Fri, 11 Dec 1998 10:43:36 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from Ilsa.StevesCafe.com (Ilsa.StevesCafe.com [205.168.119.129]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id KAA10098; Fri, 11 Dec 1998 10:43:29 -0800 (PST) (envelope-from fbsd@Ilsa.StevesCafe.com) Received: from Ilsa.StevesCafe.com (localhost.StevesCafe.com [127.0.0.1]) by Ilsa.StevesCafe.com (8.8.8/8.8.5) with ESMTP id LAA03619; Fri, 11 Dec 1998 11:47:21 -0700 (MST) Message-Id: <199812111847.LAA03619@Ilsa.StevesCafe.com> X-Mailer: exmh version 2.0.2 2/24/98 From: Steve Passe To: Roger Hardiman cc: splite , Nate Williams , mobile@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: SMP and PCMCIA card controllers give kernel panic In-reply-to: Your message of "Fri, 11 Dec 1998 13:53:30 GMT." <367123D9.41C6@cs.strath.ac.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 11 Dec 1998 11:47:21 -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, > Hi, > > Well I did some kernel hacking and tracked down the problem to the > IRQ probe code. The error is in build_freelist() in /sys/pccard/pcic.c > > It tries to ask the kernel which IRQs are free to be > allocated to A) the PCMCIA controller board and B) the PCMCIA cards. > > This probe does not work correctly with SMP systems. > It seems that on my hardware, the kernel Frees up some IRQs, giving > these messages in my boot sequence. > Freeing (NOT implemented) redirected ISA irq 11. > Freeing (NOT implemented) redirected ISA irq 10. > Freeing (NOT implemented) redirected ISA irq 9. > Freeing (NOT implemented) redirected ISA irq 11. > > The IRQ probe scans IRQs from 1 to 15. > When it gets to IRQ9 it causes a kernel panic, reporting a clash > in the IRQ allocation. (I did not capture the exact error). > > Now IRQs 9,10 and 11 are what my BIOS assignes to my PCI cards and > the SMP kernel does reasign these to IRQs 16 onwards in the magical > APIC_IO way. Need more detailed error messages from your failure. The line: Freeing (NOT implemented) redirected ISA irq 9. ^^^^^^^^^^^^^^^ implies that the SMP kernel did direct IRQ9 thru the APIC. The "NOT implimented" part is a reminder that the ISA section has not been reprogrammed to allow use of IRQ 9 by cards on the ISA bus. (didn't) have time to figure out how when I wrote that code...). So the problem is in this code: /* See if the IRQ is free. */ if (register_intr(irq, 0, 0, nullfunc, NULL, irq) == 0) { /* Give it back, but add it to the mask */ INTRMASK(freemask, mask); unregister_intr(irq, nullfunc); } Since the SMP kernel actually registered an INT from 16 up, but didn't unhook IRQ9 from the bus, nor register it, it looks like its free, when it really isn't. Its been a long time since I was in this code, so I'm not sure of what the cure should be. Would be nice if someone could find time to impliment the "freeing of ISA INT paths" for this situation, but I don't think this would fix this problem. It appears (need the exact error message!) that it is a programmed panic, based on some perceived allocation problem, but I have no idea where without seeing the error message. (note that I used an older SNAP for source to discuss this issue, could be the panic is coming from the current version of this function, but that I am not up to date.) -- Steve Passe | powered by smp@csn.net | Symmetric MultiProcessor FreeBSD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Dec 11 18:21:26 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id SAA01182 for freebsd-smp-outgoing; Fri, 11 Dec 1998 18:21:26 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from NS1.HomeMortgageUSA.Com (Shell.Bourg.Net [207.229.69.135]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id SAA01150; Fri, 11 Dec 1998 18:21:08 -0800 (PST) (envelope-from gus@Bourg.Net) Received: from localhost (gus@localhost) by NS1.HomeMortgageUSA.Com (8.9.1/8.8.7) with SMTP id SAA21764; Fri, 11 Dec 1998 18:25:50 -0800 (PST) (envelope-from gus@Bourg.Net) Date: Fri, 11 Dec 1998 18:25:50 -0800 (PST) From: Gus Bourg X-Sender: gus@NS1.HomeMortgageUSA.Com To: current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: gettimeofday/named Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I have some FreeBSD-Current machines running named. I'm having a little bit of a challenege with them, though. Named is taking up all of the CPU on the machine. Some of these machines are running SMP kernels and some aren't. 79892 root 105 0 15348K 15128K CPU1 0 18.9H 98.63% 98.63% named 15816 root 93 0 9772K 9512K RUN 176:14 99.22% 99.22% named As you can see here, named isn't being very nice. :-) I did a ktrace on it, and then a dump. Heres what I got: 79892 named CALL gettimeofday(0xefbfda84,0) 79892 named RET gettimeofday 0 79892 named CALL sendto(0x4,0x83fba80,0x1e,0,0x850d07c,0x10) 79892 named GIO fd 4 wrote 30 bytes "\M^B\^T\0\0\0\^A\0\0\0\0\0\0\bnocharge\^Cnet\0\0\^O\0\^A" 79892 named RET sendto 30/0x1e 79892 named CALL gettimeofday(0x8084b04,0) 79892 named RET gettimeofday 0 79892 named CALL gettimeofday(0xefbfda84,0) 79892 named RET gettimeofday 0 79892 named CALL sendto(0x4,0x83efe80,0x1e,0,0x8bcebec,0x10) 79892 named GIO fd 4 wrote 30 bytes "\M^B\^S\0\0\0\^A\0\0\0\0\0\0\bnocharge\^Cnet\0\0\^O\0\^A" Anyone know whats wrong? Thanks Gus Bourg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Dec 12 01:00:26 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id BAA04250 for freebsd-smp-outgoing; Sat, 12 Dec 1998 01:00:26 -0800 (PST) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from verdi.nethelp.no (verdi.nethelp.no [158.36.41.162]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id BAA04229 for ; Sat, 12 Dec 1998 01:00:23 -0800 (PST) (envelope-from sthaug@nethelp.no) From: sthaug@nethelp.no Received: (qmail 24046 invoked by uid 1001); 12 Dec 1998 09:00:10 +0000 (GMT) To: gus@Bourg.Net Cc: current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: gettimeofday/named In-Reply-To: Your message of "Fri, 11 Dec 1998 18:25:50 -0800 (PST)" References: X-Mailer: Mew version 1.05+ on Emacs 19.34.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Sat, 12 Dec 1998 10:00:10 +0100 Message-ID: <24044.913453210@verdi.nethelp.no> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I have some FreeBSD-Current machines running named. I'm having a little > bit of a challenege with them, though. Named is taking up all of the CPU > on the machine. Some of these machines are running SMP kernels and some > aren't. Are you running the standard FreeBSD named (ie 8.1.2) from -current? How many zones do you have? > As you can see here, named isn't being very nice. :-) I did a ktrace on > it, and then a dump. Heres what I got: > 79892 named CALL gettimeofday(0xefbfda84,0) > 79892 named RET gettimeofday 0 > 79892 named CALL sendto(0x4,0x83fba80,0x1e,0,0x850d07c,0x10) > 79892 named GIO fd 4 wrote 30 bytes > "\M^B\^T\0\0\0\^A\0\0\0\0\0\0\bnocharge\^Cnet\0\0\^O\0\^A" > 79892 named RET sendto 30/0x1e > 79892 named CALL gettimeofday(0x8084b04,0) > 79892 named RET gettimeofday 0 > 79892 named CALL gettimeofday(0xefbfda84,0) > 79892 named RET gettimeofday 0 > 79892 named CALL sendto(0x4,0x83efe80,0x1e,0,0x8bcebec,0x10) > 79892 named GIO fd 4 wrote 30 bytes > "\M^B\^S\0\0\0\^A\0\0\0\0\0\0\bnocharge\^Cnet\0\0\^O\0\^A" I see long sequences of gettimeofday() right after a named restart - that's when it tries do get the SOA of every zone it's secondary for, to check that the zone is current. After it's finished verifying the zones, things get nice and quiet. You need to look at some longer traces, I think. Is named spending its time on system calls, or in userland? How many gettimeofday() calls do you have per second? Steinar Haug, Nethelp consulting, sthaug@nethelp.no To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message